<jupyter_start><jupyter_text>Part 4 : Deploy, Run Inference, Interpret Inference  [Overview](./0-AutoClaimFraudDetection.ipynb)* [Notebook 0 : Overview, Architecture and Data Exploration](./0-AutoClaimFraudDetection.ipynb)* [Notebook 1: Data Prep, Process, Store Features](./1-data-prep-e2e.ipynb)* [Notebook 2: Train, Check Bias, Tune, Record Lineage, and Register a Model](./2-lineage-train-assess-bias-tune-registry-e2e.ipynb)* [Notebook 3: Mitigate Bias, Train New Model, Store in Registry](./3-mitigate-bias-train-model2-registry-e2e.ipynb)* **[Notebook 4: Deploy Model, Run Predictions](./4-deploy-run-inference-e2e.ipynb)**  * **[Architecture](deploy)**  * **[Deploy an approved model and Run Inference via Feature Store](deploy-model)**  * **[Create a Predictor](predictor)**  * **[Run Predictions from Online FeatureStore](run-predictions)*** [Notebook 5 : Create and Run an End-to-End Pipeline to Deploy the Model](./5-pipeline-e2e.ipynb) In this section of the end to end use case, we will deploy the mitigated model that is the end-product of this fraud detection use-case. We will show how to run inference and also how to use Clarify to interpret or "explain" the model.   Architecture for this ML Lifecycle Stage : Train, Check Bias, Tune, Record Lineage, Register Model[overview](overview-4)![train-assess-tune-register](./images/part4.png)  Install required and/or update third-party libraries<jupyter_code>!python -m pip install -Uq pip
!python -m pip install -q awswrangler imbalanced-learn<jupyter_output>/opt/conda/lib/python3.7/site-packages/secretstorage/dhcrypto.py:16: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
/opt/conda/lib/python3.7/site-packages/secretstorage/util.py:25: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv[0m
/opt/conda/lib/python3.7/site-packages/secretstorage/dhcrypto.py:16: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
  from cryptography.utils import int_from_bytes
/opt/conda/lib/python3.7/site-packages/secretstorage/util.py:25: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
 [...]<jupyter_text>Load stored variablesRun the cell below to load any prevously created variables. You should see a print-out of the existing variables. If you don't see anything you may need to create them again or it may be your first time running this notebook.<jupyter_code>%store -r
%store<jupyter_output>Stored variables and their in-db values:
bucket                              -> 'sagemaker-us-east-1-875692608981'
claims_fg_name                      -> 'fraud-detect-demo-claims'
claims_preprocessed                 ->       policy_id  incident_severity  num_vehicles_i
claims_table                        -> 'fraud-detect-demo-claims-1637200704'
clarify_bias_job_1_name             -> 'Clarify-Bias-2021-11-18-02-16-43-192'
clarify_bias_job_2_name             -> 'Clarify-Bias-2021-11-18-02-37-10-321'
clarify_expl_job_name               -> 'Clarify-Explainability-2021-11-18-02-49-06-695'
col_order                           -> ['fraud', 'customer_gender_male', 'customer_educat
context_name                        -> 'fraud-detect-1637201755'
customers_fg_name                   -> 'fraud-detect-demo-customers'
customers_preprocessed              ->       policy_id  customer_age  customer_education 
customers_table                     -> 'fraud-detect-demo-customers-1637200706'
database_name [...]<jupyter_text>**Important: You must have run the previous sequential notebooks to retrieve variables using the StoreMagic command.**  Import libraries<jupyter_code>import json
import time
import boto3
import sagemaker
import numpy as np
import pandas as pd
import awswrangler as wr<jupyter_output><empty_output><jupyter_text>Set region, boto3 and SageMaker SDK variables<jupyter_code># You can change this to a region of your choice
region = sagemaker.Session().boto_region_name
print("Using AWS Region: {}".format(region))
boto3.setup_default_session(region_name=region)
boto_session = boto3.Session(region_name=region)
s3_client = boto3.client("s3", region_name=region)
sagemaker_client = boto_session.client("sagemaker")
sagemaker_session = sagemaker.session.Session(
    boto_session=boto_session, sagemaker_client=sagemaker_client
)
sagemaker_role = sagemaker.get_execution_role()
print(f"Sagemaker Role: {sagemaker_role }")
account_id = boto3.client("sts").get_caller_identity()["Account"]
# variables used for parameterizing the notebook run
endpoint_name = f"{model_2_name}-endpoint"
endpoint_instance_count = 1
endpoint_instance_type = "ml.m4.xlarge"

predictor_instance_count = 1
predictor_instance_type = "ml.c5.xlarge"
batch_transform_instance_count = 1
batch_transform_instance_type = "ml.c5.xlarge"<jupyter_output><empty_output><jupyter_text>Deploy an approved model and make prediction via Feature Store[overview](overview-4)  Approve the second modelIn the real-life MLOps lifecycle, a model package gets approved after evaluation by data scientists, subject matter experts and auditors. ![train-assess-tune-register](./images/bestmodel.png)<jupyter_code>sagemaker_client.list_model_packages(ModelPackageGroupName=mpg_name)["ModelPackageSummaryList"][0]
second_model_package = sagemaker_client.list_model_packages(ModelPackageGroupName=mpg_name)["ModelPackageSummaryList"][0]
model_package_update = {
    "ModelPackageArn": second_model_package["ModelPackageArn"],
    "ModelApprovalStatus": "Approved",
}

update_response = sagemaker_client.update_model_package(**model_package_update)
update_response<jupyter_output><empty_output><jupyter_text>Create an endpoint config and an endpointDeploy the endpoint. This might take about 8minutes. ![train-assess-tune-register](./images/endpoint.png)<jupyter_code>primary_container = {'ModelPackageName': second_model_package['ModelPackageArn']}
endpoint_config_name=f'{model_2_name}-endpoint-config'
existing_configs = len(sagemaker_client.list_endpoint_configs(NameContains=endpoint_config_name, MaxResults = 30)['EndpointConfigs'])

if existing_configs == 0:
    create_ep_config_response = sagemaker_client.create_endpoint_config(
        EndpointConfigName=endpoint_config_name,
        ProductionVariants=[{
            'InstanceType': endpoint_instance_type,
            'InitialVariantWeight': 1,
            'InitialInstanceCount': endpoint_instance_count,
            'ModelName': model_2_name,
            'VariantName': 'AllTraffic'
        }]
    )
    %store endpoint_config_name
    print(f"Endpoint Config name: {endpoint_config_name}")
existing_endpoints = sagemaker_client.list_endpoints(NameContains=endpoint_name, MaxResults = 30)['Endpoints']
if not existing_endpoints:
    create_endpoint_response = sagemaker_client.create_endpoint(
        EndpointName=endpoint_name,
        EndpointConfigName=endpoint_config_name)
    %store endpoint_name
    print(f"Endpoint name: {endpoint_name}")

endpoint_info = sagemaker_client.describe_endpoint(EndpointName=endpoint_name)
endpoint_status = endpoint_info['EndpointStatus']

while endpoint_status == 'Creating':
    endpoint_info = sagemaker_client.describe_endpoint(EndpointName=endpoint_name)
    endpoint_status = endpoint_info['EndpointStatus']
    print('Endpoint status:', endpoint_status)
    if endpoint_status == 'Creating':
        time.sleep(60)<jupyter_output>Stored 'endpoint_name' (str)
Endpoint name: fraud-detect-demo-xgboost-post-smote-endpoint
Endpoint status: Creating
Endpoint status: Creating
Endpoint status: Creating
Endpoint status: Creating
Endpoint status: Creating
Endpoint status: InService<jupyter_text>Create a predictor<jupyter_code>predictor = sagemaker.predictor.Predictor(
    endpoint_name=endpoint_name, sagemaker_session=sagemaker_session
)

predictor.enable_data_capture()<jupyter_output>------------------!<jupyter_text>Split Dataset<jupyter_code>dataset = pd.read_csv("data/dataset.csv")
train = dataset.sample(frac=0.8, random_state=0)
test = dataset.drop(train.index)
test.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 1000 entries, 0 to 4997
Data columns (total 48 columns):
 #   Column                           Non-Null Count  Dtype  
---  ------                           --------------  -----  
 0   Unnamed: 0                       1000 non-null   int64  
 1   policy_id                        1000 non-null   int64  
 2   customer_gender_male             1000 non-null   int64  
 3   customer_education               1000 non-null   int64  
 4   incident_type_collision          1000 non-null   int64  
 5   fraud                            1000 non-null   int64  
 6   num_injuries                     1000 non-null   int64  
 7   policy_liability                 1000 non-null   int64  
 8   customer_gender_female           1000 non-null   int64  
 9   incident_severity                1000 non-null   int64  
 10  authorities_contacted_fire       1000 non-null   int64  
 11  collision_type_front             1000 non-null   int64  
 12  incident_type_theft [...]<jupyter_text>Get sample's claim data from online feature storeThis will simulate getting data in real-time from a customer's insurance claim submission. ![train-assess-tune-register](./images/endpoint2.png)<jupyter_code>featurestore_runtime = boto_session.client(
    service_name="sagemaker-featurestore-runtime", region_name=region
)

feature_store_session = sagemaker.Session(
    boto_session=boto_session,
    sagemaker_client=sagemaker_client,
    sagemaker_featurestore_runtime_client=featurestore_runtime,
)
feature_store_session<jupyter_output><empty_output><jupyter_text>Run Predictions on Multiple Claims[overview](overview-4)<jupyter_code>import datetime as datetime

timer = []
MAXRECS = 100


def barrage_of_inference():
    sample_policy_id = int(test.sample(1)["policy_id"])

    temp_fg_name = "fraud-detect-demo-claims"

    claims_response = featurestore_runtime.get_record(
        FeatureGroupName=temp_fg_name, RecordIdentifierValueAsString=str(sample_policy_id)
    )

    if claims_response.get("Record"):
        claims_record = claims_response["Record"]
        claims_df = pd.DataFrame(claims_record).set_index("FeatureName")
    else:
        print("No Record returned / Record Key  \n")

    t0 = datetime.datetime.now()

    customers_response = featurestore_runtime.get_record(
        FeatureGroupName=customers_fg_name, RecordIdentifierValueAsString=str(sample_policy_id)
    )

    t1 = datetime.datetime.now()

    customer_record = customers_response["Record"]
    customer_df = pd.DataFrame(customer_record).set_index("FeatureName")

    blended_df = pd.concat([claims_df, customer_df]).loc[col_order].drop("fraud")
    data_input = ",".join(blended_df["ValueAsString"])

    results = predictor.predict(data_input, initial_args={"ContentType": "text/csv"})
    prediction = json.loads(results)
    # print (f'Probablitity the claim from policy {int(sample_policy_id)} is fraudulent:', prediction)

    arr = t1 - t0
    minutes, seconds = divmod(arr.total_seconds(), 60)

    timer.append(seconds)
    # print (prediction, " done in {} ".format(seconds))

    return sample_policy_id, prediction, arr


for i in range(MAXRECS):
    sample_policy_id, prediction, arr = barrage_of_inference()
    print(f"Probablitity the claim from policy {int(sample_policy_id)} is fraudulent:, {prediction}  Time {str(arr).split(':')[2]}")<jupyter_output>Probablitity the claim from policy 4516 is fraudulent:, 0.01880461908876896  Time 00.039854
Probablitity the claim from policy 823 is fraudulent:, 0.01880461908876896  Time 00.012036
Probablitity the claim from policy 3286 is fraudulent:, 0.06872951984405518  Time 00.009671
Probablitity the claim from policy 2154 is fraudulent:, 0.01635090820491314  Time 00.010776
Probablitity the claim from policy 3658 is fraudulent:, 0.053575288504362106  Time 00.009784
Probablitity the claim from policy 4683 is fraudulent:, 0.06872951984405518  Time 00.010459
Probablitity the claim from policy 2121 is fraudulent:, 0.01635090820491314  Time 00.014014
Probablitity the claim from policy 446 is fraudulent:, 0.13182105123996735  Time 00.014302
Probablitity the claim from policy 3428 is fraudulent:, 0.1972830593585968  Time 00.029294
Probablitity the claim from policy 1846 is fraudulent:, 0.01880461908876896  Time 00.011701
Probablitity the claim from policy 2992 is fraudulent:, 0.014692461118102074  Time[...]<jupyter_text>Note: the above "timer" records the first call and then subsequent calls to the online Feature Store<jupyter_code>import statistics
import numpy as np

statistics.mean(timer)


arr = np.array(timer)
print(
    "p95: {}, p99: {}, mean: {} for {} distinct feature store gets".format(
        np.percentile(arr, 95), np.percentile(arr, 99), np.mean(arr), MAXRECS
    )
)<jupyter_output>p95: 0.029443599999999993, p99: 0.04704072000000004, mean: 0.012613419999999997 for 100 distinct feature store gets<jupyter_text>Pull customer data from Customers feature groupWhen a customer submits an insurance claim online for instant approval, the insurance company will need to pull customer-specific data from the online feature store to add to the claim data as input for a model prediction.  Sample a claim from the test data<jupyter_code>sample_policy_id = int(test.sample(1)["policy_id"])
print(f"Sample Policy ID: {sample_policy_id}")
customers_response = featurestore_runtime.get_record(
    FeatureGroupName=customers_fg_name, RecordIdentifierValueAsString=str(sample_policy_id)
)

customer_record = customers_response["Record"]
customer_df = pd.DataFrame(customer_record).set_index("FeatureName")


claims_response = featurestore_runtime.get_record(
    FeatureGroupName=claims_fg_name, RecordIdentifierValueAsString=str(sample_policy_id)
)

claims_record = claims_response["Record"]
claims_df = pd.DataFrame(claims_record).set_index("FeatureName")
claims_df<jupyter_output><empty_output><jupyter_text>Format the datapointThe datapoint must match the exact input format as the model was trained--with all features in the correct order. In this example, the `col_order` variable was saved when you created the train and test datasets earlier in the guide.<jupyter_code>blended_df = pd.concat([claims_df, customer_df]).loc[col_order].drop("fraud")
blended_df
data_input = ",".join(blended_df["ValueAsString"])
data_input<jupyter_output><empty_output><jupyter_text>Make prediction<jupyter_code>results = predictor.predict(data_input, initial_args={"ContentType": "text/csv"})
prediction = json.loads(results)
print(f"Probablitity the claim from policy {int(sample_policy_id)} is fraudulent:", prediction)<jupyter_output>Probablitity the claim from policy 4374 is fraudulent: 0.06872951984405518
StarcoderdataJupyterStructuredCleanDedup