# ONNX Runtime Pipelines[[optimum.onnxruntime.pipeline]]

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>optimum.onnxruntime.pipeline</name><anchor>optimum.onnxruntime.pipeline</anchor><source>https://github.com/huggingface/optimum-onnx/blob/vr_82/optimum/onnxruntime/pipelines.py#L150</source><parameters>[{"name": "task", "val": ": str | None = None"}, {"name": "model", "val": ": str | ORTModel | None = None"}, {"name": "config", "val": ": str | PretrainedConfig | None = None"}, {"name": "tokenizer", "val": ": str | PreTrainedTokenizer | PreTrainedTokenizerFast | None = None"}, {"name": "feature_extractor", "val": ": str | FeatureExtractionMixin | None = None"}, {"name": "image_processor", "val": ": str | BaseImageProcessor | None = None"}, {"name": "processor", "val": ": str | ProcessorMixin | None = None"}, {"name": "revision", "val": ": str | None = None"}, {"name": "use_fast", "val": ": bool = True"}, {"name": "token", "val": ": str | bool | None = None"}, {"name": "device", "val": ": int | str | torch.device | None = None"}, {"name": "trust_remote_code", "val": ": bool | None = None"}, {"name": "model_kwargs", "val": ": dict[str, Any] | None = None"}, {"name": "pipeline_class", "val": ": Any | None = None"}, {"name": "**kwargs", "val": ": Any"}]</parameters><paramsdesc>- **task** (`str`) --
  The task defining which pipeline will be returned. Currently accepted tasks are:

  - `"audio-classification"`: will return a `AudioClassificationPipeline`.
  - `"automatic-speech-recognition"`: will return a `AutomaticSpeechRecognitionPipeline`.
  - `"depth-estimation"`: will return a `DepthEstimationPipeline`.
  - `"document-question-answering"`: will return a `DocumentQuestionAnsweringPipeline`.
  - `"feature-extraction"`: will return a `FeatureExtractionPipeline`.
  - `"fill-mask"`: will return a `FillMaskPipeline`:.
  - `"image-classification"`: will return a `ImageClassificationPipeline`.
  - `"image-feature-extraction"`: will return an `ImageFeatureExtractionPipeline`.
  - `"image-segmentation"`: will return a `ImageSegmentationPipeline`.
  - `"image-text-to-text"`: will return a `ImageTextToTextPipeline`.
  - `"image-to-image"`: will return a `ImageToImagePipeline`.
  - `"image-to-text"`: will return a `ImageToTextPipeline`.
  - `"mask-generation"`: will return a `MaskGenerationPipeline`.
  - `"object-detection"`: will return a `ObjectDetectionPipeline`.
  - `"question-answering"`: will return a `QuestionAnsweringPipeline`.
  - `"summarization"`: will return a `SummarizationPipeline`.
  - `"table-question-answering"`: will return a `TableQuestionAnsweringPipeline`.
  - `"text2text-generation"`: will return a `Text2TextGenerationPipeline`.
  - `"text-classification"` (alias `"sentiment-analysis"` available): will return a
    `TextClassificationPipeline`.
  - `"text-generation"`: will return a `TextGenerationPipeline`:.
  - `"text-to-audio"` (alias `"text-to-speech"` available): will return a `TextToAudioPipeline`:.
  - `"token-classification"` (alias `"ner"` available): will return a `TokenClassificationPipeline`.
  - `"translation"`: will return a `TranslationPipeline`.
  - `"translation_xx_to_yy"`: will return a `TranslationPipeline`.
  - `"video-classification"`: will return a `VideoClassificationPipeline`.
  - `"visual-question-answering"`: will return a `VisualQuestionAnsweringPipeline`.
  - `"zero-shot-classification"`: will return a `ZeroShotClassificationPipeline`.
  - `"zero-shot-image-classification"`: will return a `ZeroShotImageClassificationPipeline`.
  - `"zero-shot-audio-classification"`: will return a `ZeroShotAudioClassificationPipeline`.
  - `"zero-shot-object-detection"`: will return a `ZeroShotObjectDetectionPipeline`.

- **model** (`str` or `ORTModel`, *optional*) --
  The model that will be used by the pipeline to make predictions. This can be a model identifier or an
  actual instance of a ONNX Runtime model inheriting from `ORTModel`.

  If not provided, the default for the `task` will be loaded.
- **config** (`str` or `PretrainedConfig`, *optional*) --
  The configuration that will be used by the pipeline to instantiate the model. This can be a model
  identifier or an actual pretrained model configuration inheriting from `PretrainedConfig`.

  If not provided, the default configuration file for the requested model will be used. That means that if
  `model` is given, its default configuration will be used. However, if `model` is not supplied, this
  `task`'s default model's config is used instead.
- **tokenizer** (`str` or `PreTrainedTokenizer`, *optional*) --
  The tokenizer that will be used by the pipeline to encode data for the model. This can be a model
  identifier or an actual pretrained tokenizer inheriting from `PreTrainedTokenizer`.

  If not provided, the default tokenizer for the given `model` will be loaded (if it is a string). If `model`
  is not specified or not a string, then the default tokenizer for `config` is loaded (if it is a string).
  However, if `config` is also not given or not a string, then the default tokenizer for the given `task`
  will be loaded.
- **feature_extractor** (`str` or `PreTrainedFeatureExtractor`, *optional*) --
  The feature extractor that will be used by the pipeline to encode data for the model. This can be a model
  identifier or an actual pretrained feature extractor inheriting from `PreTrainedFeatureExtractor`.

  Feature extractors are used for non-NLP models, such as Speech or Vision models as well as multi-modal
  models. Multi-modal models will also require a tokenizer to be passed.

  If not provided, the default feature extractor for the given `model` will be loaded (if it is a string). If
  `model` is not specified or not a string, then the default feature extractor for `config` is loaded (if it
  is a string). However, if `config` is also not given or not a string, then the default feature extractor
  for the given `task` will be loaded.
- **image_processor** (`str` or `BaseImageProcessor`, *optional*) --
  The image processor that will be used by the pipeline to preprocess images for the model. This can be a
  model identifier or an actual image processor inheriting from `BaseImageProcessor`.

  Image processors are used for Vision models and multi-modal models that require image inputs. Multi-modal
  models will also require a tokenizer to be passed.

  If not provided, the default image processor for the given `model` will be loaded (if it is a string). If
  `model` is not specified or not a string, then the default image processor for `config` is loaded (if it is
  a string).
- **processor** (`str` or `ProcessorMixin`, *optional*) --
  The processor that will be used by the pipeline to preprocess data for the model. This can be a model
  identifier or an actual processor inheriting from `ProcessorMixin`.

  Processors are used for multi-modal models that require multi-modal inputs, for example, a model that
  requires both text and image inputs.

  If not provided, the default processor for the given `model` will be loaded (if it is a string). If `model`
  is not specified or not a string, then the default processor for `config` is loaded (if it is a string).
- **revision** (`str`, *optional*, defaults to `"main"`) --
  When passing a task name or a string model identifier: The specific model version to use. It can be a
  branch name, a tag name, or a commit id, since we use a git-based system for storing models and other
  artifacts on huggingface.co, so `revision` can be any identifier allowed by git.
- **use_fast** (`bool`, *optional*, defaults to `True`) --
  Whether or not to use a Fast tokenizer if possible (a `PreTrainedTokenizerFast`).
- **use_auth_token** (`str` or *bool*, *optional*) --
  The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated
  when running `hf auth login` (stored in `~/.huggingface`).
- **device** (`int` or `str` or `torch.device`) --
  Defines the device (*e.g.*, `"cpu"`, `"cuda:1"`, `"mps"`, or a GPU ordinal rank like `1`) on which this
  pipeline will be allocated.
- **device_map** (`str` or `dict[str, Union[int, str, torch.device]`, *optional*) --
  Sent directly as `model_kwargs` (just a simpler shortcut). When `accelerate` library is present, set
  `device_map="auto"` to compute the most optimized `device_map` automatically (see
  [here](https://huggingface.co/docs/accelerate/main/en/package_reference/big_modeling#accelerate.cpu_offload)
  for more information).

  <Tip warning={true}>

  Do not use `device_map` AND `device` at the same time as they will conflict

  </Tip>

- **torch_dtype** (`str` or `torch.dtype`, *optional*) --
  Sent directly as `model_kwargs` (just a simpler shortcut) to use the available precision for this model
  (`torch.float16`, `torch.bfloat16`, ... or `"auto"`).
- **trust_remote_code** (`bool`, *optional*, defaults to `False`) --
  Whether or not to allow for custom code defined on the Hub in their own modeling, configuration,
  tokenization or even pipeline files. This option should only be set to `True` for repositories you trust
  and in which you have read the code, as it will execute code present on the Hub on your local machine.
- **model_kwargs** (`dict[str, Any]`, *optional*) --
  Additional dictionary of keyword arguments passed along to the model's `from_pretrained(...,
  **model_kwargs)` function.
- **kwargs** (`dict[str, Any]`, *optional*) --
  Additional keyword arguments passed along to the specific pipeline init (see the documentation for the
  corresponding pipeline class for possible values).</paramsdesc><paramgroups>0</paramgroups><rettype>`Pipeline`</rettype><retdesc>A suitable pipeline for the task.</retdesc></docstring>
Utility factory method to build a `Pipeline` with an ONNX Runtime model, similar to `transformers.pipeline`.

A pipeline consists of:

- One or more components for pre-processing model inputs, such as a [tokenizer](tokenizer),
[image_processor](image_processor), [feature_extractor](feature_extractor), or [processor](processors).
- A [model](model) that generates predictions from the inputs.
- Optional post-processing steps to refine the model's output, which can also be handled by processors.

<Tip>
While there are such optional arguments as `tokenizer`, `feature_extractor`, `image_processor`, and `processor`,
they shouldn't be specified all at once. If these components are not provided, `pipeline` will try to load
required ones automatically. In case you want to provide these components explicitly, please refer to a
specific pipeline in order to get more details regarding what components are required.
</Tip>







<ExampleCodeBlock anchor="optimum.onnxruntime.pipeline.example">

Examples:
```python
>>> from optimum.onnxruntime import pipeline

>>> # Sentiment analysis pipeline
>>> analyzer = pipeline("sentiment-analysis")

>>> # Question answering pipeline, specifying the checkpoint identifier
>>> oracle = pipeline(
...     "question-answering", model="distilbert/distilbert-base-cased-distilled-squad", tokenizer="google-bert/bert-base-cased"
... )

>>> # Named entity recognition pipeline, passing in a specific model and tokenizer
>>> model = ORTModelForTokenClassification.from_pretrained("dbmdz/bert-large-cased-finetuned-conll03-english")
>>> tokenizer = AutoTokenizer.from_pretrained("google-bert/bert-base-cased")
>>> recognizer = pipeline("ner", model=model, tokenizer=tokenizer)
```

</ExampleCodeBlock>


</div>