Instructions to use AtomicChat/Qwen3-Coder-30B-A3B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use AtomicChat/Qwen3-Coder-30B-A3B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AtomicChat/Qwen3-Coder-30B-A3B-GGUF:UD-Q4_K_XL
# Run inference directly in the terminal:
llama cli -hf AtomicChat/Qwen3-Coder-30B-A3B-GGUF:UD-Q4_K_XL

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AtomicChat/Qwen3-Coder-30B-A3B-GGUF:UD-Q4_K_XL
# Run inference directly in the terminal:
llama cli -hf AtomicChat/Qwen3-Coder-30B-A3B-GGUF:UD-Q4_K_XL

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf AtomicChat/Qwen3-Coder-30B-A3B-GGUF:UD-Q4_K_XL
# Run inference directly in the terminal:
./llama-cli -hf AtomicChat/Qwen3-Coder-30B-A3B-GGUF:UD-Q4_K_XL

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf AtomicChat/Qwen3-Coder-30B-A3B-GGUF:UD-Q4_K_XL
# Run inference directly in the terminal:
./build/bin/llama-cli -hf AtomicChat/Qwen3-Coder-30B-A3B-GGUF:UD-Q4_K_XL

Use Docker

docker model run hf.co/AtomicChat/Qwen3-Coder-30B-A3B-GGUF:UD-Q4_K_XL

LM Studio
Jan

vLLM

How to use AtomicChat/Qwen3-Coder-30B-A3B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AtomicChat/Qwen3-Coder-30B-A3B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AtomicChat/Qwen3-Coder-30B-A3B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/AtomicChat/Qwen3-Coder-30B-A3B-GGUF:UD-Q4_K_XL

Ollama
How to use AtomicChat/Qwen3-Coder-30B-A3B-GGUF with Ollama:
```
ollama run hf.co/AtomicChat/Qwen3-Coder-30B-A3B-GGUF:UD-Q4_K_XL
```

Unsloth Studio

How to use AtomicChat/Qwen3-Coder-30B-A3B-GGUF with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AtomicChat/Qwen3-Coder-30B-A3B-GGUF to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for AtomicChat/Qwen3-Coder-30B-A3B-GGUF to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for AtomicChat/Qwen3-Coder-30B-A3B-GGUF to start chatting

How to use AtomicChat/Qwen3-Coder-30B-A3B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AtomicChat/Qwen3-Coder-30B-A3B-GGUF:UD-Q4_K_XL

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AtomicChat/Qwen3-Coder-30B-A3B-GGUF:UD-Q4_K_XL"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use AtomicChat/Qwen3-Coder-30B-A3B-GGUF with Docker Model Runner:
```
docker model run hf.co/AtomicChat/Qwen3-Coder-30B-A3B-GGUF:UD-Q4_K_XL
```

Lemonade

How to use AtomicChat/Qwen3-Coder-30B-A3B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull AtomicChat/Qwen3-Coder-30B-A3B-GGUF:UD-Q4_K_XL

Run and chat with the model

lemonade run user.Qwen3-Coder-30B-A3B-GGUF-UD-Q4_K_XL

List all available models

lemonade list

Hermes Agent

How to use AtomicChat/Qwen3-Coder-30B-A3B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AtomicChat/Qwen3-Coder-30B-A3B-GGUF:UD-Q4_K_XL

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AtomicChat/Qwen3-Coder-30B-A3B-GGUF:UD-Q4_K_XL

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use AtomicChat/Qwen3-Coder-30B-A3B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AtomicChat/Qwen3-Coder-30B-A3B-GGUF:UD-Q4_K_XL

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "AtomicChat/Qwen3-Coder-30B-A3B-GGUF:UD-Q4_K_XL" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

AlexAtomic commited on Jun 17

Commit

b800d5d

0 Parent(s):

Duplicate from AlexAtomic/qwen3-coder-30b-a3b-GGUF

Browse files

Files changed (18) hide show

.gitattributes +48 -0
README.md +135 -0
hero.png +3 -0
pill_atomic_v3.png +0 -0
pill_discord_v3.png +0 -0
pill_github_v3.png +0 -0
qwen3-coder-30b-a3b-IQ3_M.gguf +3 -0
qwen3-coder-30b-a3b-IQ4_XS.gguf +3 -0
qwen3-coder-30b-a3b-Q2_K.gguf +3 -0
qwen3-coder-30b-a3b-Q3_K_L.gguf +3 -0
qwen3-coder-30b-a3b-Q3_K_M.gguf +3 -0
qwen3-coder-30b-a3b-Q4_K_M.gguf +3 -0
qwen3-coder-30b-a3b-Q4_K_S.gguf +3 -0
qwen3-coder-30b-a3b-Q5_K_M.gguf +3 -0
qwen3-coder-30b-a3b-Q5_K_S.gguf +3 -0
qwen3-coder-30b-a3b-Q6_K.gguf +3 -0
qwen3-coder-30b-a3b-Q8_0.gguf +3 -0
qwen3-coder-30b-a3b-UD-Q4_K_XL.gguf +3 -0

.gitattributes ADDED Viewed

	@@ -0,0 +1,48 @@

+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+qwen3-coder-30b-a3b-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-coder-30b-a3b-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-coder-30b-a3b-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-coder-30b-a3b-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-coder-30b-a3b-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-coder-30b-a3b-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-coder-30b-a3b-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-coder-30b-a3b-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-coder-30b-a3b-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-coder-30b-a3b-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-coder-30b-a3b-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
+qwen3-coder-30b-a3b-UD-Q4_K_XL.gguf filter=lfs diff=lfs merge=lfs -text
+hero.png filter=lfs diff=lfs merge=lfs -text

README.md ADDED Viewed

	@@ -0,0 +1,135 @@

+---
+license: apache-2.0
+license_link: https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct/blob/main/LICENSE
+thumbnail: https://huggingface.co/AlexAtomic/qwen3-coder-30b-a3b-GGUF/resolve/main/hero.png
+base_model:
+- Qwen/Qwen3-Coder-30B-A3B-Instruct
+base_model_relation: quantized
+quantized_by: AlexAtomic
+pipeline_tag: text-generation
+library_name: gguf
+tags:
+- atomic-chat
+- qwen
+- qwen3
+- qwen3-coder
+- gguf
+- imatrix
+- quantized
+- llama.cpp
+---
+<center>
+<div style="display:flex; justify-content:center; align-items:center; gap:10px; flex-wrap:wrap;">
+<a href="https://atomic.chat"><img src="https://huggingface.co/AlexAtomic/qwen3-coder-30b-a3b-GGUF/resolve/main/pill_atomic_v3.png" alt="Atomic Chat" width="186"></a>
+<a href="https://discord.gg/8wGSsvmg4V"><img src="https://huggingface.co/AlexAtomic/qwen3-coder-30b-a3b-GGUF/resolve/main/pill_discord_v3.png" alt="Join Discord" width="184"></a>
+<a href="https://github.com/AtomicBot-ai/Atomic-Chat"><img src="https://huggingface.co/AlexAtomic/qwen3-coder-30b-a3b-GGUF/resolve/main/pill_github_v3.png" alt="GitHub" width="141"></a>
+</div>
+<br/>
+<img src="https://huggingface.co/AlexAtomic/qwen3-coder-30b-a3b-GGUF/resolve/main/hero.png" alt="Qwen3 Coder 30B A3B" style="width:100%; max-width:100%; height:auto; margin-bottom:0.6em;"/>
+<div style="display:flex; justify-content:center; gap:0.5em;">
+<a href="https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct"><strong>Base model: Qwen/Qwen3-Coder-30B-A3B-Instruct</strong></a>
+</div>
+</center>
+**Qwen3 Coder 30B A3B**, self-quantized to GGUF by [Atomic Chat](https://atomic.chat). Built straight from Qwen's original weights with a per-tensor importance matrix. Runs fully offline.
+## Highlights
+- **Agentic coding specialist** with significant performance among open models on agentic coding, agentic browser-use, and other foundational coding tasks.
+- **Efficient MoE**: 30.5B total parameters, only 3.3B activated per token (128 experts, 8 activated).
+- **256K native context** (262,144 tokens), extendable up to ~1M tokens with Yarn, optimized for repository-scale understanding.
+- **Tool calling built in** with a specially designed function-call format, supporting platforms such as Qwen Code and CLINE.
+- **Non-thinking mode only** — does not emit `<think></think>` blocks; no `enable_thinking` flag required.
+- **Full quant ladder** with an importance matrix on every quant over [`calibration_datav3`](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8).
+> [!NOTE]
+> These GGUFs are **self-quantized from the original weights**, not a repack. The importance matrix keeps low-bit quants closer to the full-precision model.
+> [!IMPORTANT]
+> Always pass `--jinja` so the **Qwen3 Coder 30B A3B chat template** is applied. Without it the model can emit malformed turns.
+## Model Overview
+| Property | Value |
+|---|---|
+| Base model | `Qwen/Qwen3-Coder-30B-A3B-Instruct` |
+| Total / active parameters | 30.5B total, 3.3B activated (128 experts, 8 activated) |
+| Layers | 48 |
+| Context length | 262,144 native (extendable to ~1M with Yarn) |
+| Architecture | Causal LM, Mixture-of-Experts; GQA (32 Q heads, 4 KV heads) |
+| This repo | GGUF quants (imatrix) |
+See the [official model card](https://huggingface.co/Qwen/Qwen3-Coder-30B-A3B-Instruct) for Qwen's published benchmark results.
+## Choosing a quant
+| Quant | Size | Notes |
+|---|---|---|
+| `Q2_K` | 11.3 GB | Smallest. Minimal RAM, clear quality drop. |
+| `IQ3_M` | 13.5 GB | Beats Q3 at similar size thanks to imatrix. Best low-RAM pick. |
+| `Q3_K_M` | 14.7 GB | Low quality but usable. |
+| `Q3_K_L` | 15.9 GB | A step above Q3_K_M. |
+| `IQ4_XS` | 16.4 GB | Excellent quality for size. Recommended low-bit. |
+| `Q4_K_S` | 17.5 GB | Compact Q4, fast. |
+| **`Q4_K_M`** | 18.6 GB | **Recommended default. Best balance of size, speed and quality.** |
+| **`UD-Q4_K_XL`** | 18.8 GB | **Dynamic. Embeddings and output kept at Q8_0 for higher quality at a Q4 footprint.** |
+| `Q5_K_S` | 19.7 GB | Higher quality. |
+| `Q5_K_M` | 12.1 GB | Higher quality, low loss. |
+| `Q6_K` | 17.4 GB | Near lossless. |
+| `Q8_0` | 20.3 GB | Effectively lossless, reference quality. |
+> [!TIP]
+> Pick the largest file that fits your (V)RAM with room for context. `Q4_K_M` or `UD-Q4_K_XL` is the sweet spot for most setups; `Q6_K` or `Q8_0` for maximum fidelity.
+## Get started
+Run Qwen3 Coder 30B A3B locally with:
+- **[Atomic Chat](https://atomic.chat):** the easiest path. Open the app, search `AlexAtomic/qwen3-coder-30b-a3b-GGUF`, pick a quant, hit **Use this model**.
+- **llama.cpp:** `llama-server -hf AlexAtomic/qwen3-coder-30b-a3b-GGUF:Q4_K_M --jinja -c 8192`
+- **Ollama:** `ollama run hf.co/AlexAtomic/qwen3-coder-30b-a3b-GGUF:Q4_K_M`
+- **LM Studio / Jan:** search the repo id, download any quant.
+## Best practices
+| Parameter | Value |
+|---|---|
+| temperature | 0.7 |
+| top_p | 0.8 |
+| top_k | 20 |
+| repetition_penalty | 1.05 |
+Qwen's recommended settings for this model (non-thinking); recommended output length 65,536 tokens.
+## Run in llama.cpp
+```bash
+git clone https://github.com/ggerganov/llama.cpp
+cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
+cmake --build llama.cpp/build --config Release -j --target llama-cli llama-server
+```
+```bash
+./llama.cpp/build/bin/llama-server \
+    -hf AlexAtomic/qwen3-coder-30b-a3b-GGUF:UD-Q4_K_XL \
+    --jinja -ngl 99 -c 8192 -fa on
+```
+## How these were made
+1. Download `Qwen/Qwen3-Coder-30B-A3B-Instruct` (original weights).
+2. Convert to f16 GGUF with [llama.cpp](https://github.com/ggerganov/llama.cpp).
+3. Build an importance matrix over `calibration_datav3` (100 chunks).
+4. Quantize the full ladder with `--imatrix`.
+5. `UD-Q4_K_XL` additionally pins the token-embedding and output tensors to `Q8_0`.
+## License
+Released by Qwen under the Apache 2.0 license. Quantized by Atomic Chat.

hero.png ADDED Viewed

Git LFS Details

SHA256: c6e6df069ee6e4f711d712073f403a557f6602c0d7f553072504338082cb4161
Pointer size: 131 Bytes
Size of remote file: 259 kB

pill_atomic_v3.png ADDED Viewed

pill_discord_v3.png ADDED Viewed

pill_github_v3.png ADDED Viewed

qwen3-coder-30b-a3b-IQ3_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:01f44ed4567a9b129398fb8111d53f622e30e2ddf139789cac63d3e68af4ca68
+size 13513064480

qwen3-coder-30b-a3b-IQ4_XS.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6f7ce69b74ede41880492b2765c0fdbf746e55da71b127bdb1cb1dc53086b565
+size 16368351264

qwen3-coder-30b-a3b-Q2_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a4e64886408921200d7ae5caf89d6a8e8cb8c695e248a425fc619b314e5e9a04
+size 11258611744

qwen3-coder-30b-a3b-Q3_K_L.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8756a5593ad92532443cdc1e065e5b405f47a47eb22fa15916264784c7345509
+size 15900672032

qwen3-coder-30b-a3b-Q3_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:10396f4d203e3507d0e8e45a6557f7813f5e2d9ac275ba44b579663acd11a60d
+size 14711848992

qwen3-coder-30b-a3b-Q4_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:22e70e6305c8fa5ea0bbbdf11fa6f1779b5546b33dd38bda2c2274b5e5d574b5
+size 18556688416

qwen3-coder-30b-a3b-Q4_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:218027f6bd58460fd3b948090bc67a7c14b755e5f2fbcddd76fd921cc893dfcc
+size 17456011296

qwen3-coder-30b-a3b-Q5_K_M.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8bcd3c38c250898bba2cc39916efbe04334244abce1a888b79fb5bf2d9e85b66
+size 12137205760

qwen3-coder-30b-a3b-Q5_K_S.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:93eabfe1dfefd0764d3fb91c342a1caf95e5ccd46b172ff69ba0b64590d012b1
+size 19654266880

qwen3-coder-30b-a3b-Q6_K.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:10c33659726fcc6119f5e4c9c3034ce6bbb61e44db2b6fa538ba7e0353487c22
+size 17428013056

qwen3-coder-30b-a3b-Q8_0.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cf1c958e47881496ea5516694cfb12720fed55180b696941f16f32900fa6fd4e
+size 20333920256

qwen3-coder-30b-a3b-UD-Q4_K_XL.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e4ed3bcbe5e7a1aeac2b912f62adaff964ea0f678bae0e3f7e5df266b4d07f8a
+size 18787631136