# Nemotron-Labs-Audex-2B Overview

## Description:  
Nemotron-Labs-Audex-2B is an audio language model built on a dense-2B LLM with strong audio abilities and no degradation on text intelligence. It uses NV-Whisper as audio encoder and outputs both text tokens and audio tokens from XCodec and XCodec-2. The model leverages Cascaded-RL based on Nemotron Cascade 2 for strong text reasoning abilities and operates in both thinking and instruct modes.  
Nemotron-Labs-Audex-2B was developed by NVIDIA as a part of Nemotron.  
_This model is for research and development only._  


### License/Terms of Use:
NVIDIA OneWay Noncommercial License

Audex is derived from outputs of XCodec2 (CC-BY-NC 4.0) and XCodec (Apache-2.0) and is subject to the same non-commercial restrictions. See [`license/THIRD_PARTY_NOTICES.md`](license/THIRD_PARTY_NOTICES.md) for third-party license and attribution details.


### Deployment Geography:  
Global


### Use Case:  
Developers building AI agent systems, chatbots, and applications requiring advanced reasoning and long-context understanding in both text and audio domains.  
  

### Release Date:  
Hugging Face [June 8, 2026] via [HF Model Link](https://huggingface.co/nvidia/Nemotron-Labs-Audex-2B)

  

## Reference(s):
[Unified Audio Intelligence Without Regressing on Text Intelligence](https://arxiv.org/)
[UALM: Unified Audio Language Model for Understanding, Generation and Reasoning](https://arxiv.org/abs/2510.12000)  
[Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation](https://arxiv.org/abs/2603.19220)  

## Model Architecture:   
**Architecture Type:** Transformer   
**Network Architecture:** Dense GPT  
**This model was developed based on GPT.**  
**Number of model parameters:** 2B (3.0*10^9)  
  
 
## Input:  
**Input Type(s):** Text, Audio  
**Input Format:** String, .wav  
**Input Parameters:** One-Dimensional (1D), One-Dimensional (1D)  
**Other Properties Related to Input:** Text input is provided as prompts or instructions with context length 1M. Audio input uses NV-Whisper as the audio encoder.  
  

## Output:  
**Output Type(s):** Text, Audio  
**Output Format:** String, Tensor  
**Output Parameters:** One-Dimensional (1D), One-Dimensional (1D)  
**Other Properties Related to Output:** Supports text token generation and audio token generation from XCodec and XCodec-2. Text generation supports 1M context lengths. Audio token generation supports 15 minutes of audio at 16kHz sampling rate.  
   

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

## Software Integration:
**Runtime Engine(s):**
* vLLM
* Transformers

**Supported Hardware Microarchitecture Compatibility:**
* NVIDIA Blackwell
* NVIDIA Hopper

**Supported Operating System(s):** Linux  

The model can be integrated using various checkpoints and APIs: text-only vLLM checkpoint with vLLM inference API, full vLLM checkpoint with vLLM inference API for audio generation, and full HF checkpoint with HF inference API for audio understanding. Conversion scripts are provided.  

The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.  


## Model Version(s): 
Nemotron-Labs-Audex-2B  
 
## Training, Testing, and Evaluation Datasets:  


## Training Dataset:

**Data Modality:**
* Text  
* Audio  


**Audio Training Data Size:** 10,000 to 1 Million Hours  
**Text Training Data Size:** 1 Billion to 1 Trillion Tokens  
**Data Collection Method by dataset:** Hybrid: Human/Synthetic  
**Labeling Method by dataset:** Hybrid: Human/Synthetic  
**Properties (Quantity, Dataset Descriptions, Sensor(s)):** Training data is composed of 477.9B tokens in total sourced from public commercial and academic datasets. It includes a mix of Nemotron-Cascade-2 data, Parakeet data, Magpie-TTS data, Audio Flamingo 3 data, and SAO data for various tasks the model supports.

### Testing Dataset:

**Data Split:** Not Applicable: All testing benchmarks are separate, standard research benchmarks
**Data Collection Method by dataset:** Hybrid: Human/Synthetic  
**Labeling Method by dataset:** Hybrid: Human/Synthetic  
**Properties (Quantity, Dataset Descriptions, Sensor(s)):** Not Applicable: All testing benchmarks are separate, standard research benchmarks 

### Evaluation Dataset:

<img src="assets/result_text.png" alt="Description" style="width: 75%;">
<br></br>


**Data Collection Method by dataset:** Undisclosed  
**Labeling Method by dataset:** Undisclosed  
**Properties (Quantity, Dataset Descriptions, Sensor(s)):** Evaluated on diverse reasoning benchmarks including mathematical problem solving, code generation challenges, question answering tasks, and instruction following assessments.



## Inference:
**Acceleration Engine:** vLLM
**Test Hardware:** NVIDIA Hopper (H100)

## Ethical Considerations:
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. Developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.  

Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail).