# Export functions

You can export models to ONNX from PyTorch. 
There is an export function for PyTorch models [export_pytorch()](/docs/optimum/pr_68/en/onnx/package_reference/export#optimum.exporters.onnx.convert.export_pytorch), 
but the recommended way of using those is via the main export function `~optimum.exporters.main_export`, 
which will take care of using the proper exporting function according to the available framework, 
check that the exported model is valid, and provide extended options to run optimizations on the exported model.

## Main functions[[optimum.exporters.onnx.main_export]]

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>optimum.exporters.onnx.main_export</name><anchor>optimum.exporters.onnx.main_export</anchor><source>https://github.com/huggingface/optimum-onnx/blob/vr_68/optimum/exporters/onnx/__main__.py#L57</source><parameters>[{"name": "model_name_or_path", "val": ": str"}, {"name": "output", "val": ": str | Path"}, {"name": "task", "val": ": str = 'auto'"}, {"name": "opset", "val": ": int | None = None"}, {"name": "device", "val": ": str = 'cpu'"}, {"name": "dtype", "val": ": str | None = None"}, {"name": "optimize", "val": ": str | None = None"}, {"name": "monolith", "val": ": bool = False"}, {"name": "no_post_process", "val": ": bool = False"}, {"name": "framework", "val": ": str | None = 'pt'"}, {"name": "atol", "val": ": float | None = None"}, {"name": "pad_token_id", "val": ": int | None = None"}, {"name": "subfolder", "val": ": str = ''"}, {"name": "revision", "val": ": str = 'main'"}, {"name": "force_download", "val": ": bool = False"}, {"name": "local_files_only", "val": ": bool = False"}, {"name": "trust_remote_code", "val": ": bool = False"}, {"name": "cache_dir", "val": ": str = '/home/runner/.cache/huggingface/hub'"}, {"name": "token", "val": ": bool | str | None = None"}, {"name": "do_validation", "val": ": bool = True"}, {"name": "model_kwargs", "val": ": dict[str, Any] | None = None"}, {"name": "custom_onnx_configs", "val": ": dict[str, OnnxConfig] | None = None"}, {"name": "fn_get_submodels", "val": ": Callable | None = None"}, {"name": "use_subprocess", "val": ": bool = False"}, {"name": "_variant", "val": ": str = 'default'"}, {"name": "library_name", "val": ": str | None = None"}, {"name": "no_dynamic_axes", "val": ": bool = False"}, {"name": "do_constant_folding", "val": ": bool = True"}, {"name": "slim", "val": ": bool = False"}, {"name": "**kwargs_shapes", "val": ""}]</parameters><paramsdesc></paramsdesc><paramsdesc1title>Required parameters</paramsdesc1title><paramsdesc1>

- **model_name_or_path** (`str`) --
  Model ID on huggingface.co or path on disk to the model repository to export. Example: `model_name_or_path="BAAI/bge-m3"` or `mode_name_or_path="/path/to/model_folder`.
- **output** (`Union[str, Path]`) --
  Path indicating the directory where to store the generated ONNX model.

</paramsdesc1><paramsdesc2title>Optional parameters</paramsdesc2title><paramsdesc2>

- **task** (`Optional[str]`, defaults to `None`) --
  The task to export the model for. If not specified, the task will be auto-inferred based on the model. For decoder models,
  use `xxx-with-past` to export the model using past key values in the decoder.
- **opset** (`Optional[int]`, defaults to `None`) --
  If specified, ONNX opset version to export the model with. Otherwise, the default opset for the given model architecture
  will be used.
- **device** (`str`, defaults to `"cpu"`) --
  The device to use to do the export. Defaults to "cpu".
- **dtype** (`Optional[str]`, defaults to `None`) --
  The floating point precision to use for the export. Supported options: `"fp32"` (float32), `"fp16"` (float16), `"bf16"` (bfloat16). Defaults to `"fp32"`.
- **optimize** (`Optional[str]`, defaults to `None`) --
  Allows to run ONNX Runtime optimizations directly during the export. Some of these optimizations are specific to
  ONNX Runtime, and the resulting ONNX will not be usable with other runtime as OpenVINO or TensorRT.
  Available options: `"O1", "O2", "O3", "O4"`. Reference: [AutoOptimizationConfig](/docs/optimum/pr_68/en/onnxruntime/package_reference/configuration#optimum.onnxruntime.AutoOptimizationConfig)
- **monolith** (`bool`, defaults to `False`) --
  Forces to export the model as a single ONNX file.
- **no_post_process** (`bool`, defaults to `False`) --
  Allows to disable any post-processing done by default on the exported ONNX models.
- **framework** (`Optional[str]`, defaults to `None`) --
  The framework to use for the ONNX export (`"pt"`). If not provided, will attempt to automatically detect the framework for the checkpoint.
- **atol** (`Optional[float]`, defaults to `None`) --
  If specified, the absolute difference tolerance when validating the model. Otherwise, the default atol for the model will be used.
- **cache_dir** (`Optional[str]`, defaults to `None`) --
  Path indicating where to store cache. The default Hugging Face cache path will be used by default.
- **trust_remote_code** (`bool`, defaults to `False`) --
  Allows to use custom code for the modeling hosted in the model repository. This option should only be set for repositories
  you trust and in which you have read the code, as it will execute on your local machine arbitrary code present in the
  model repository.
- **pad_token_id** (`Optional[int]`, defaults to `None`) --
  This is needed by some models, for some tasks. If not provided, will attempt to use the tokenizer to guess it.
- **subfolder** (`str`, defaults to `""`) --
  In case the relevant files are located inside a subfolder of the model repo either locally or on huggingface.co, you can
  specify the folder name here.
- **revision** (`str`, defaults to `"main"`) --
  Revision is the specific model version to use. It can be a branch name, a tag name, or a commit id.
- **force_download** (`bool`, defaults to `False`) --
  Whether or not to force the (re-)download of the model weights and configuration files, overriding the
  cached versions if they exist.
- **local_files_only** (`Optional[bool]`, defaults to `False`) --
  Whether or not to only look at local files (i.e., do not try to download the model).
- **token** (`Optional[Union[bool,str]]`, defaults to `None`) --
  The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated
  when running `huggingface-cli login` (stored in `huggingface_hub.constants.HF_TOKEN_PATH`).
- **do_validation** (`bool`, defaults to `True`) --
  Whether or not to validate the exported ONNX model by running inference on it.
- **model_kwargs** (`Optional[Dict[str, Any]]`, defaults to `None`) --
  Experimental usage: keyword arguments to pass to the model during
  the export. This argument should be used along the `custom_onnx_configs` argument
  in case, for example, the model inputs/outputs are changed (for example, if
  `model_kwargs={"output_attentions": True}` is passed).
- **custom_onnx_configs** (`Optional[Dict[str, OnnxConfig]]`, defaults to `None`) --
  Experimental usage: override the default ONNX config used for the given model. This argument may be useful for advanced users that desire a finer-grained control on the export. An example is available [here](https://huggingface.co/docs/optimum/main/en/exporters/onnx/usage_guides/export_a_model).
- **fn_get_submodels** (`Optional[Callable]`, defaults to `None`) --
  Experimental usage: Override the default submodels that are used at the export. This is
  especially useful when exporting a custom architecture that needs to split the ONNX (e.g. encoder-decoder). If unspecified with custom models, optimum will try to use the default submodels used for the given task, with no guarantee of success.
- **use_subprocess** (`bool`, defaults to `False`) --
  Do the ONNX exported model validation in subprocesses. This is especially useful when
  exporting on CUDA device, where ORT does not release memory at inference session
  destruction. When set to `True`, the `main_export` call should be guarded in
  `if __name__ == "__main__":` block.
- **_variant** (`str`, defaults to `default`) --
  Specify the variant of the ONNX export to use.
- **library_name** (`Optional[str]`, defaults to `None`) --
  The library of the model (`"transformers"` or `"diffusers"` or `"timm"` or `"sentence_transformers"`). If not provided, will attempt to automatically detect the library name for the checkpoint.
- **no_dynamic_axes** (bool, defaults to `False`) --
  If True, disables the use of dynamic axes during ONNX export.
- **do_constant_folding** (bool, defaults to `True`) --
  PyTorch-specific argument. If `True`, the PyTorch ONNX export will fold constants into adjacent nodes, if possible.
- **slim** (bool, defaults to `False`) --
  PyTorch-specific argument. If `True`, use onnxslim to optimize the ONNX model.
- ****kwargs_shapes** (`Dict`) --
  Shapes to use during inference. This argument allows to override the default shapes used during the ONNX export.</paramsdesc2><paramgroups>2</paramgroups></docstring>
Full-suite ONNX export function, exporting **from a model ID on Hugging Face Hub or a local model repository**.



<ExampleCodeBlock anchor="optimum.exporters.onnx.main_export.example">

Example usage:
```python
>>> from optimum.exporters.onnx import main_export

>>> main_export("gpt2", output="gpt2_onnx/")
```

</ExampleCodeBlock>


</div>

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>optimum.exporters.onnx.onnx_export_from_model</name><anchor>optimum.exporters.onnx.onnx_export_from_model</anchor><source>https://github.com/huggingface/optimum-onnx/blob/vr_68/optimum/exporters/onnx/convert.py#L801</source><parameters>[{"name": "model", "val": ": PreTrainedModel | DiffusionPipeline"}, {"name": "output", "val": ": str | Path"}, {"name": "opset", "val": ": int | None = None"}, {"name": "optimize", "val": ": str | None = None"}, {"name": "monolith", "val": ": bool = False"}, {"name": "no_post_process", "val": ": bool = False"}, {"name": "atol", "val": ": float | None = None"}, {"name": "do_validation", "val": ": bool = True"}, {"name": "model_kwargs", "val": ": dict[str, Any] | None = None"}, {"name": "custom_onnx_configs", "val": ": dict[str, OnnxConfig] | None = None"}, {"name": "fn_get_submodels", "val": ": Callable | None = None"}, {"name": "_variant", "val": ": str = 'default'"}, {"name": "preprocessors", "val": ": list | None = None"}, {"name": "device", "val": ": str = 'cpu'"}, {"name": "no_dynamic_axes", "val": ": bool = False"}, {"name": "task", "val": ": str | None = None"}, {"name": "use_subprocess", "val": ": bool = False"}, {"name": "do_constant_folding", "val": ": bool = True"}, {"name": "slim", "val": ": bool = False"}, {"name": "**kwargs_shapes", "val": ""}]</parameters><paramsdesc></paramsdesc><paramsdesc1title>Required parameters</paramsdesc1title><paramsdesc1>

- **model** (`Union["PreTrainedModel", "DiffusionPipeline"]`) --
  PyTorch model to export to ONNX.
- **output** (`Union[str, Path]`) --
  Path indicating the directory where to store the generated ONNX model.

</paramsdesc1><paramsdesc2title>Optional parameters</paramsdesc2title><paramsdesc2>

- **task** (`Optional[str]`, defaults to `None`) --
  The task to export the model for. If not specified, the task will be auto-inferred based on the model.
- **opset** (`Optional[int]`, defaults to `None`) --
  If specified, ONNX opset version to export the model with. Otherwise, the default opset for the given model architecture
  will be used.
- **device** (`str`, defaults to `"cpu"`) --
  The device to use to do the export. Defaults to "cpu".
- **optimize** (`Optional[str]`, defaults to `None`) --
  Allows to run ONNX Runtime optimizations directly during the export. Some of these optimizations are specific to
  ONNX Runtime, and the resulting ONNX will not be usable with other runtime as OpenVINO or TensorRT.
  Available options: `"O1", "O2", "O3", "O4"`. Reference: [AutoOptimizationConfig](/docs/optimum/pr_68/en/onnxruntime/package_reference/configuration#optimum.onnxruntime.AutoOptimizationConfig)
- **monolith** (`bool`, defaults to `False`) --
  Forces to export the model as a single ONNX file.
- **no_post_process** (`bool`, defaults to `False`) --
  Allows to disable any post-processing done by default on the exported ONNX models.
- **atol** (`Optional[float]`, defaults to `None`) --
  If specified, the absolute difference tolerance when validating the model. Otherwise, the default atol for the model will be used.
- **do_validation** (`bool`, defaults to `True`) --
  If `True`, the exported ONNX model will be validated against the original PyTorch model.
- **model_kwargs** (`Optional[Dict[str, Any]]`, defaults to `None`) --
  Experimental usage: keyword arguments to pass to the model during
  the export. This argument should be used along the `custom_onnx_configs` argument
  in case, for example, the model inputs/outputs are changed (for example, if
  `model_kwargs={"output_attentions": True}` is passed).
- **custom_onnx_configs** (`Optional[Dict[str, OnnxConfig]]`, defaults to `None`) --
  Experimental usage: override the default ONNX config used for the given model. This argument may be useful for advanced users that desire a finer-grained control on the export. An example is available [here](https://huggingface.co/docs/optimum/main/en/exporters/onnx/usage_guides/export_a_model).
- **fn_get_submodels** (`Optional[Callable]`, defaults to `None`) --
  Experimental usage: Override the default submodels that are used at the export. This is
  especially useful when exporting a custom architecture that needs to split the ONNX (e.g. encoder-decoder). If unspecified with custom models, optimum will try to use the default submodels used for the given task, with no guarantee of success.
- **use_subprocess** (`bool`, defaults to `False`) --
  Do the ONNX exported model validation in subprocesses. This is especially useful when
  exporting on CUDA device, where ORT does not release memory at inference session
  destruction. When set to `True`, the `main_export` call should be guarded in
  `if __name__ == "__main__":` block.
- **_variant** (`str`, defaults to `default`) --
  Specify the variant of the ONNX export to use.
- **preprocessors** (`Optional[List]`, defaults to `None`) --
  List of preprocessors to use for the ONNX export.
- **no_dynamic_axes** (bool, defaults to `False`) --
  If True, disables the use of dynamic axes during ONNX export.
- **do_constant_folding** (bool, defaults to `True`) --
  PyTorch-specific argument. If `True`, the PyTorch ONNX export will fold constants into adjacent nodes, if possible.
- **slim** (bool, defaults to `False`) --
  Use onnxslim to optimize the ONNX model.
- ****kwargs_shapes** (`Dict`) --
  Shapes to use during inference. This argument allows to override the default shapes used during the ONNX export.</paramsdesc2><paramgroups>2</paramgroups></docstring>
Full-suite ONNX export function, exporting **from a pre-loaded PyTorch model**. This function is especially useful in case one needs to do modifications on the model, as overriding a forward call, before exporting to ONNX.



<ExampleCodeBlock anchor="optimum.exporters.onnx.onnx_export_from_model.example">

Example usage:
```python
>>> from transformers import AutoModelForCausalLM

>>> model = AutoModelForCausalLM.from_pretrained("gpt2")
>>> # At this point, we could override some submodules, forward methods, weights, etc. from the model.

>>> onnx_export_from_model(model, output="gpt2_onnx/")
```

</ExampleCodeBlock>


</div>

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>optimum.exporters.onnx.export</name><anchor>optimum.exporters.onnx.export</anchor><source>https://github.com/huggingface/optimum-onnx/blob/vr_68/optimum/exporters/onnx/convert.py#L698</source><parameters>[{"name": "model", "val": ": PreTrainedModel | ModelMixin"}, {"name": "config", "val": ": OnnxConfig"}, {"name": "output", "val": ": Path"}, {"name": "opset", "val": ": int | None = None"}, {"name": "device", "val": ": str = 'cpu'"}, {"name": "input_shapes", "val": ": dict | None = None"}, {"name": "disable_dynamic_axes_fix", "val": ": bool | None = False"}, {"name": "dtype", "val": ": str | None = None"}, {"name": "no_dynamic_axes", "val": ": bool = False"}, {"name": "do_constant_folding", "val": ": bool = True"}, {"name": "model_kwargs", "val": ": dict[str, Any] | None = None"}]</parameters><paramsdesc>- **model** (`PreTrainedModel` or `ModelMixin`) --
  The model to export.
- **config** ([OnnxConfig](/docs/optimum/pr_68/en/onnx/package_reference/configuration#optimum.exporters.onnx.OnnxConfig)) --
  The ONNX configuration associated with the exported model.
- **output** (`Path`) --
  Directory to store the exported ONNX model.
- **opset** (`Optional[int]`, defaults to `None`) --
  The version of the ONNX operator set to use.
- **device** (`Optional[str]`, defaults to `"cpu"`) --
  The device on which the ONNX model will be exported. Either `cpu` or `cuda`. Only PyTorch is supported for
  export on CUDA devices.
- **input_shapes** (`Optional[Dict]`, defaults to `None`) --
  If specified, allows to use specific shapes for the example input provided to the ONNX exporter.
- **disable_dynamic_axes_fix** (`Optional[bool]`, defaults to `False`) --
  Whether to disable the default dynamic axes fixing.
- **dtype** (`Optional[str]`, defaults to `None`) --
  Data type to remap the model inputs to. PyTorch-only. Only `fp16` is supported.
- **no_dynamic_axes** (bool, defaults to `False`) --
  If True, disables the use of dynamic axes during ONNX export.
- **do_constant_folding** (bool, defaults to `True`) --
  PyTorch-specific argument. If `True`, the PyTorch ONNX export will fold constants into adjacent nodes, if possible.
- **model_kwargs** (`Optional[Dict[str, Any]]`, defaults to `None`) --
  Experimental usage: keyword arguments to pass to the model during
  the export. This argument should be used along the `custom_onnx_config` argument
  in case, for example, the model inputs/outputs are changed (for example, if
  `model_kwargs={"output_attentions": True}` is passed).</paramsdesc><paramgroups>0</paramgroups><rettype>`Tuple[List[str], List[str]]`</rettype><retdesc>A tuple with an ordered list of the model's inputs, and the named outputs from
the ONNX configuration.</retdesc></docstring>
Exports a Pytorch model to an ONNX Intermediate Representation.








</div>

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>optimum.exporters.onnx.convert.export_pytorch</name><anchor>optimum.exporters.onnx.convert.export_pytorch</anchor><source>https://github.com/huggingface/optimum-onnx/blob/vr_68/optimum/exporters/onnx/convert.py#L462</source><parameters>[{"name": "model", "val": ": PreTrainedModel | ModelMixin"}, {"name": "config", "val": ": OnnxConfig"}, {"name": "opset", "val": ": int"}, {"name": "output", "val": ": Path"}, {"name": "device", "val": ": str = 'cpu'"}, {"name": "input_shapes", "val": ": dict | None = None"}, {"name": "no_dynamic_axes", "val": ": bool = False"}, {"name": "do_constant_folding", "val": ": bool = True"}, {"name": "model_kwargs", "val": ": dict[str, Any] | None = None"}]</parameters><paramsdesc>- **model** (`PreTrainedModel`) --
  The model to export.
- **config** ([OnnxConfig](/docs/optimum/pr_68/en/onnx/package_reference/configuration#optimum.exporters.onnx.OnnxConfig)) --
  The ONNX configuration associated with the exported model.
- **opset** (`int`) --
  The version of the ONNX operator set to use.
- **output** (`Path`) --
  Path to save the exported ONNX file to.
- **device** (`str`, defaults to `"cpu"`) --
  The device on which the ONNX model will be exported. Either `cpu` or `cuda`. Only PyTorch is supported for
  export on CUDA devices.
- **input_shapes** (`Optional[Dict]`, defaults to `None`) --
  If specified, allows to use specific shapes for the example input provided to the ONNX exporter.
- **no_dynamic_axes** (bool, defaults to `False`) --
  If True, disables the use of dynamic axes during ONNX export.
- **do_constant_folding** (bool, defaults to `True`) --
  PyTorch-specific argument. If `True`, the PyTorch ONNX export will fold constants into adjacent nodes, if possible.
- **model_kwargs** (`Optional[Dict[str, Any]]`, defaults to `None`) --
  Experimental usage: keyword arguments to pass to the model during
  the export. This argument should be used along the `custom_onnx_config` argument
  in case, for example, the model inputs/outputs are changed (for example, if
  `model_kwargs={"output_attentions": True}` is passed).</paramsdesc><paramgroups>0</paramgroups><rettype>`Tuple[List[str], List[str]]`</rettype><retdesc>A tuple with an ordered list of the model's inputs, and the named outputs from
the ONNX configuration.</retdesc></docstring>
Exports a PyTorch model to an ONNX Intermediate Representation.








</div>

## Utility functions[[optimum.exporters.utils.check_dummy_inputs_are_allowed]]

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>optimum.exporters.utils.check_dummy_inputs_are_allowed</name><anchor>optimum.exporters.utils.check_dummy_inputs_are_allowed</anchor><source>https://github.com/huggingface/optimum-onnx/blob/vr_68/optimum/exporters/utils.py#L621</source><parameters>[{"name": "model", "val": ": typing.Union[ForwardRef('PreTrainedModel'), ForwardRef('ModelMixin')]"}, {"name": "dummy_input_names", "val": ": typing.Iterable[str]"}]</parameters><paramsdesc>- **model** (`PreTrainedModel` or `ModelMixin`) --
  The model instance.
- **model_inputs** (`Iterable[str]`) --
  The model input names.</paramsdesc><paramgroups>0</paramgroups></docstring>

Checks that the dummy inputs from the ONNX config is a subset of the allowed inputs for `model`.



</div>

<div class="docstring border-l-2 border-t-2 pl-4 pt-3.5 border-gray-100 rounded-tl-xl mb-6 mt-8">


<docstring><name>optimum.exporters.onnx.validate_model_outputs</name><anchor>optimum.exporters.onnx.validate_model_outputs</anchor><source>https://github.com/huggingface/optimum-onnx/blob/vr_68/optimum/exporters/onnx/convert.py#L166</source><parameters>[{"name": "config", "val": ": OnnxConfig"}, {"name": "reference_model", "val": ": PreTrainedModel | ModelMixin"}, {"name": "onnx_model", "val": ": Path"}, {"name": "onnx_named_outputs", "val": ": list[str]"}, {"name": "atol", "val": ": float | None = None"}, {"name": "input_shapes", "val": ": dict | None = None"}, {"name": "device", "val": ": str = 'cpu'"}, {"name": "use_subprocess", "val": ": bool | None = True"}, {"name": "model_kwargs", "val": ": dict[str, Any] | None = None"}]</parameters><paramsdesc>- **config** (`~OnnxConfig`) --
  The configuration used to export the model.
- **reference_model** (`Union["PreTrainedModel", "ModelMixin"]`) --
  The model used for the export.
- **onnx_model** (`Path`) --
  The path to the exported model.
- **onnx_named_outputs** (`List[str]`) --
  The names of the outputs to check.
- **atol** (`Optional[float]`, defaults to `None`) --
  The absolute tolerance in terms of outputs difference between the reference and the exported model.
- **input_shapes** (`Optional[Dict]`, defaults to `None`) --
  If specified, allows to use specific shapes to validate the ONNX model on.
- **device** (`str`, defaults to `"cpu"`) --
  The device on which the ONNX model will be validated. Either `cpu` or `cuda`. Validation on a CUDA device is supported only for PyTorch.
- **use_subprocess** (`Optional[bool]`, defaults to `True`) --
  Launch validation of each exported model in a subprocess.
- **model_kwargs** (`Optional[Dict[str, Any]]`, defaults to `None`) --
  Experimental usage: keyword arguments to pass to the model during
  the export and validation.</paramsdesc><paramgroups>0</paramgroups><raises>- ``ValueError`` -- If the outputs shapes or values do not match between the reference and the exported model.</raises><raisederrors>``ValueError``</raisederrors></docstring>
Validates the export by checking that the outputs from both the reference and the exported model match.








</div>