#
Deploy with your own container

If the available Inference Engines don’t meet your requirements, you can deploy your own custom solution as a Docker container and run it on Inference Endpoints. You can use public images like `tensorflow/serving:2.7.3` or private images hosted on Docker Hub, AWS ECR, Azure ACR, or Google GCR.

The creation flow of your image artifacts from a custom image is the same as the base image. This means Inference Endpoints will create a unique image artifact derived from your provided image, including all model artifacts.

The model artifacts (weights) are stored under `/repository`. For example, if you use `tensorflow/serving` as your custom image, then you have to set `model_base_path=“/repository”:

    tensorflow_model_server \
      --rest_api_port=5000 \
      --model_name=my_model \
      --model_base_path="/repository"

Update on GitHub