#
Configuration

This section describes the configuration options available when creating a new inference endpoint. Each section of the interface allows fine-grained control over how the model is deployed, accessed, and scaled.

##
Endpoint name, model and organization

In the top left you can:

- change the name of the inference endpoint
- verify to which organization you’re deploying this model
- verify which model you are deploying
- and which Hugging Face Hub repo you are deploying this model from

##
Hardware Configuration

The Hardware Configuration section allows you to choose the compute backend used to host the model. You can select from three major cloud providers:

- Amazon Web Services (AWS)
- Microsoft Azure
- Google Cloud Platform

You must also choose an accelerator type:

- CPU
- GPU
- INF2 (AWS Inferentia)

Additionally, you can select the deployment region (e.g., East US) using the dropdown menu. Once the provider, accelerator, and region are chosen, a list of available instance types is displayed. Each instance tile includes:

- GPU Type and Count
- Memory (e.g., 48 GB)
- vCPUs and RAM
- Hourly Pricing (e.g., $1.80 / h)

You can select a tile to choose that instance type for your deployment. Instances that are incompatible or unavailable in the selected region are grayed out and unclickable.

##
Security Level

This section determines who can access your deployed endpoint. Available options are:

- Protected (default): Accessible only to members of your Hugging Face organization using personal access tokens. The endpoint is secured with TLS/SSL.
- Public: Anyone can access your endpoint, without authentication.
- HF Restricted: Anyone with a Hugging Face account can access it, using a personal Hugging Face Token generated from their account.

Additionally, if you deploy your Inference Endpoint in AWS, you can use AWS Private Link for an intra-region secured connection to your AWS VPN.

##
Autoscaling

The Autoscaling section configures how many replicas of your model run and whether the system scales down to zero during periods of inactivity. For more information we recommend reading the in-depth guide on autoscaling.

- Automatic Scale-to-Zero: A dropdown lets you choose how long the system should wait after the last request before scaling down to zero. Default is after 1 hour with no activity.
- Number of Replicas:
  - Min: Minimum number of replicas to keep running. Note that enabling automatic scale-to-zero requires setting this to 0.
  - Max: Maximum number of replicas allowed (e.g., 1)
- Autoscaling strategy:
  - Based on hardware usage: For example, a scale up will be triggered if the average hardware utilisation (%) exceeds this threshold for more than 20 seconds.
  - Pending requests: A scale up event will be triggered if the average number of pending requests exceeds this threshold for more than 20 seconds.

##
Container Configuration

This section allows you to specify how the container hosting your model behaves. This setting depends on the selected inference engine. For configuration details, please read the Inference Engine section.

##
Environment Variables

Environment variables can be provided to customize container behavior or pass secrets.

- Default Env: Key-value pairs passed as plain environment variables.
- Secret Env: Key-value pairs stored securely and injected at runtime.

Each section allows you to add multiple entries using the Add button.

##
Endpoint Tags

You can label endpoints with tags (e.g., for-testing) to help organize and manage deployments across environments or teams. In the dashboard you will be able to filter and sort endpoints based on these tags. Tags are plain text labels added via the Add button.

##
Advanced Settings

Advanced Settings offer more fine-grained control over deployment.

- Commit Revision: Optionally specify a commit hash to which revision of the model repository on the Hugging Face Hub you want to download the model artifacts from
- Task: Defines the type of model task. This is usually inferred from the model repository.
- Container Arguments: Pass CLI-style arguments to the container entrypoint.
- Container Command: Override the container entrypoint entirely.
- Download Pattern: Defines which model files are downloaded.

< > Update on GitHub