<jupyter_start><jupyter_text>Run your first InferenceService> A tutorial on building and deploying a model using the KServe Python SDKThe InferenceService custom resource is the primary interface that is used for deploying models on KServe. Inside an InferenceService, users can specify multiplecomponents that are used for handling inference requests. These components are the predictor, transformer, and explainer. Learn more [here](https://kserve.github.io/website/0.7/modelserving/data_plane/).In this tutorial, you will deploy an InferenceService with a predictor that will load a scikit-learn model trained with the [iris](https://archive.ics.uci.edu/ml/datasets/iris) dataset.This dataset has three output class: Iris Setosa, Iris Versicolour, and Iris Virginica.You will then send an inference request to your deployed model in order to get a prediction for the class of iris plant your request corresponds to. Before you beginFirst, install the KServe SDK using the following command. If you run this command in a Jupyter notebook, restart the kernel after installing the SDK.<jupyter_code>!pip install kserve==0.7.0<jupyter_output><empty_output><jupyter_text>Import `kubernetes.client` and `kserve` packages<jupyter_code>from kubernetes import client 
from kserve import KServeClient
from kserve import constants
from kserve import utils
from kserve import V1beta1InferenceService
from kserve import V1beta1InferenceServiceSpec
from kserve import V1beta1PredictorSpec
from kserve import V1beta1SKLearnSpec<jupyter_output><empty_output><jupyter_text>Declare NamespaceThis will retrieve the current namespace of your Kubernetes context. The InferenceService will be deployed in this namespace.<jupyter_code>namespace = utils.get_default_target_namespace()<jupyter_output><empty_output><jupyter_text>Define InferenceService Next, define the InferenceService based on several key parameters. In the `predictor` parameter, a `V1beta1PredictorSpec` object with an embedded `V1beta1SKLearnSpec` object is created.Inside the `V1beta1SKLearnSpec` object, a storage URI is provided, pointing to the location of the trained iris model in cloud storage.<jupyter_code>name='sklearn-iris'
kserve_version='v1beta1'
api_version = constants.KSERVE_GROUP + '/' + kserve_version

isvc = V1beta1InferenceService(api_version=api_version,
                               kind=constants.KSERVE_KIND,
                               metadata=client.V1ObjectMeta(
                                   name=name, namespace=namespace, annotations={'sidecar.istio.io/inject':'false'}),
                               spec=V1beta1InferenceServiceSpec(
                               predictor=V1beta1PredictorSpec(
                               sklearn=(V1beta1SKLearnSpec(
                                   storage_uri="gs://kfserving-samples/models/sklearn/iris"))))
)<jupyter_output><empty_output><jupyter_text>Create InferenceServiceNow, with the InferenceService defined, you can now create it by calling the `create` method of the `KServeClient`.<jupyter_code>KServe = KServeClient()
KServe.create(isvc)<jupyter_output><empty_output><jupyter_text>Check the InferenceServiceRun the following command to watch the InferenceService until it is ready (or times out).<jupyter_code>KServe.get(name, namespace=namespace, watch=True, timeout_seconds=120)<jupyter_output><empty_output><jupyter_text>Perform InferenceNext, you can try sending an inference request to the deployed model in order to get predictions. This notebook assumes that you runningit in your Kubeflow cluster and will use the internal URL of the InferenceService.The Python `requests` library will be used to send a POST request containing your payload.<jupyter_code>import requests

isvc_resp = KServe.get(name, namespace=namespace)
isvc_url = isvc_resp['status']['address']['url']

print(isvc_url)

inference_input = {
  'instances': [
    [6.8,  2.8,  4.8,  1.4],
    [6.0,  3.4,  4.5,  1.6]
  ]
}

response = requests.post(isvc_url, json=inference_input)
print(response.text)<jupyter_output><empty_output><jupyter_text>You should see two predictions returned (i.e. `{"predictions": [1, 1]}`). Both sets of data points sent for inference correspond to the flower with index `1`.In this case, the model predicts that both flowers are "Iris Versicolour".To learn more about sending inference requests, please check out the [KServe guide](https://kserve.github.io/website/0.7/get_started/first_isvc/3-determine-the-ingress-ip-and-ports).  Run Performance Test (Optional)If you want to load test the deployed model, try deploying the Kubernetes Job to drive load to the InferenceService.<jupyter_code>!kubectl create -f https://raw.githubusercontent.com/kserve/kserve/release-0.7/docs/samples/v1beta1/sklearn/v1/perf.yaml -n kubeflow-user-example-com<jupyter_output><empty_output><jupyter_text>Get Job Name<jupyter_code>!kubectl get pods --namespace=kubeflow-user-example-com | grep load<jupyter_output><empty_output><jupyter_text>Check the Job Logs<jupyter_code>!kubectl logs <job-name> -n kubeflow-user-example-com<jupyter_output><empty_output><jupyter_text>The output should look like similar to the following:```Requests      [total, rate, throughput]         30000, 500.02, 499.99Duration      [total, attack, wait]             1m0s, 59.998s, 3.336msLatencies     [min, mean, 50, 90, 95, 99, max]  1.743ms, 2.748ms, 2.494ms, 3.363ms, 4.091ms, 7.749ms, 46.354msBytes In      [total, mean]                     690000, 23.00Bytes Out     [total, mean]                     2460000, 82.00Success       [ratio]                           100.00%Status Codes  [code:count]                      200:30000Error Set:```  Delete InferenceServiceWhen you are done with your InferenceService, you can delete it by running the following.<jupyter_code>KServe.delete(name, namespace=namespace)<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup