Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

Misc

Inference Endpoints

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

47,162

Base only

Active filters: 4-bit

Statuo/MN-12b-ArliAI-RPMax-EXL2-4bpw

Updated Sep 10, 2024 • 1 • 4

Qwen/Qwen2.5-14B-Instruct-GPTQ-Int4

Text Generation • 15B • Updated Oct 9, 2024 • 90.9k • 27

MaziyarPanahi/Qwen2.5-1.5B-Instruct-GGUF

Text Generation • 2B • Updated Sep 18, 2024 • 86.2k • 11

unsloth/Llama-3.2-11B-Vision-Instruct-bnb-4bit

Image-Text-to-Text • 11B • Updated Dec 10, 2024 • 2.48k • 82

mlx-community/Qwen2.5-7B-Instruct-Uncensored-4bit

Text Generation • Updated Oct 21, 2024 • 796 • 6

MaziyarPanahi/L3-8B-Sunfall-v0.5-Stheno-v3.2-GGUF

Text Generation • 8B • Updated Oct 29, 2024 • 1.65k • 7

MaziyarPanahi/BioMistral-Clinical-7B-GGUF

Text Generation • 7B • Updated Nov 6, 2024 • 40 • 2

lmstudio-community/Qwen2.5-14B-Instruct-MLX-4bit

Text Generation • 2B • Updated Nov 13, 2024 • 2.46k • 2

mlx-community/DeepSeek-R1-Distill-Llama-70B-4bit

Text Generation • 11B • Updated Feb 26, 2025 • 2.86k • 10

mlx-community/dolphin3.0-llama3.2-3B-4Bit

Updated Jan 22, 2025 • 1.79k • 1

stelterlab/Mistral-Small-24B-Instruct-2501-AWQ

Text Generation • 24B • Updated Mar 30, 2025 • 216k • 30

unsloth/Qwen2.5-VL-3B-Instruct-unsloth-bnb-4bit

Image-Text-to-Text • Updated May 12, 2025 • 7.53k • 16

hfl/Qwen2.5-VL-7B-Instruct-GPTQ-Int4

Image-Text-to-Text • 8B • Updated Mar 20, 2025 • 2.16k • 10

Qwen/Qwen2.5-VL-32B-Instruct-AWQ

Image-Text-to-Text • 33B • Updated Apr 6, 2025 • 232k • 64

mlx-community/Llama-3.1-Nemotron-8B-UltraLong-4M-Instruct-4bit

Text Generation • 1B • Updated Apr 17, 2025 • 93 • 2

mlx-community/Qwen3-8B-4bit

Text Generation • Updated Apr 28, 2025 • 31.5k • 11

mlx-community/MiMo-7B-RL-4bit

Text Generation • Updated May 2, 2025 • 45 • 1

Qwen/Qwen3-4B-AWQ

Text Generation • 4B • Updated May 21, 2025 • 548k • 31

Qwen/Qwen2.5-Omni-7B-AWQ

Any-to-Any • 11B • Updated May 15, 2025 • 122k • 19

Qwen/Qwen3-30B-A3B-MLX-4bit

Text Generation • 4B • Updated Jul 7, 2025 • 542 • 12

huynguyendbs/Qwen3-Embedding-8B-4bit-MLX

Text Generation • 1B • Updated Jun 12, 2025 • 279 • 2

mlx-community/deepseek-coder-33b-instruct

Text Generation • Updated Jun 21, 2025 • 141 • 2

mlx-community/Llama-3.1-Nemotron-8B-UltraLong-1M-Instruct-4bit

Text Generation • 1B • Updated Jun 30, 2025 • 95 • 2

mlx-community/Dolphin-Mistral-24B-Venice-Edition-DWQ

Text Generation • 24B • Updated Jul 7, 2025 • 291 • 4

mlx-community/SmolLM3-3B-4bit

Text Generation • Updated Jul 8, 2025 • 3.6k • 7

mlx-community/Kimi-Dev-72B-4bit-DWQ

Text Generation • 73B • Updated Jul 12, 2025 • 1.92k • 20

wikeeyang/Nexus-GenV2-nf4-fp8

Any-to-Any • 8B • Updated Jul 22, 2025 • 15 • 5

lmstudio-community/Qwen3-Coder-30B-A3B-Instruct-MLX-4bit

Text Generation • 31B • Updated Jul 31, 2025 • 174k • 34

nightmedia/gpt-oss-20b-q4-hi-mlx

Text Generation • 21B • Updated Aug 6, 2025 • 112 • 3

mlx-community/Qwen3-Coder-30B-A3B-Instruct-4bit-dwq-v2

Text Generation • Updated Aug 9, 2025 • 1.31k • 10