antonyMox/Qwen3.6-27B-abliterated-AutoRound-INT4-MTP

// ANTONYMOX · UNCENSORED BUILD

Qwen3.6-27B abliterated — INT4 · MTP

Decensored without losing brains. Keeps its MTP head (2× decode), runs up to 256k context on one card, reads images, and ships the benchmark JSON — not just a claim.

KL 0.0089 MTP preserved 256k ctx vision 19 GB Apache-2.0

70

LCB-HARD

87

TOOL-USE

0.0089

KL DRIFT

115

TOK/S · MTP4

19GB

ON DISK

📊 All benchmarks & throughput measured on a single RTX 4090 (48 GB) · vLLM 0.19 · MTP n=4 · seed 42

A gentle Heretic abliteration of the pristine Qwen/Qwen3.6-27B → AutoRound INT4 → the 15-tensor MTP head grafted back in BF16 so speculative decoding works out of the box in vLLM. Removing refusals cost zero hard-coding points (70 = 70 vs the clean base) and tool-use went up (87 vs 83). Pick a preset, run one script, done.

---

Highlights

🔓 Uncensored, sharp

refusals gone, hard score unchanged (70 = 70)

🛠️ Better tool-use

87 vs 83 — beats the clean base

⚡ ~2× decode

MTP head kept → spec-decode works

📏 256k context (native)

supports 256k; up to 256k fits on a 48 GB card

🖼️ Multimodal

images & video · vision tower full BF16

🧠 Reads it all

7/8 exact recall @ 150k tokens

🎯 One-click presets

24 / 32 / 48 GB — measured, not guessed

🧾 Proof in-repo

the per-task benchmark JSON ships with it

---

Benchmarks — decensoring cost nothing, and where it sits

Same base Qwen/Qwen3.6-27B, one harness. Decensoring left hard unchanged (70 = 70 vs the reference narrow quant) and tool-use went up (83 → 87):

Model (our harness)	Recipe	hard	tools	Size
Lorbus Qwen3.6-27B narrow INT4 (reference)	narrow	70	83	18.5 GB
➡️ This — abliterated	narrow + Heretic	70	87	19 GB
wide sister	wide	78	83	26 GB
antonyMox 35B-A3B (coming soon)	MoE narrow	48	83	22 GB
Qwen3-Coder-Next 80B (UD-IQ4_XS, reference)	80B MoE	62	80	~38 GB

Numbers are % of tasks passed (40 hard coding + 30 tool tasks; raw pass/total in benchmark-results/). Conditions: temp 1.0 / top_p 0.95 / top_k 20 / min_p 0 · seed 42 · non-thinking · vLLM 0.19 · RTX 4090 (48 GB) · MTP n=4. Solutions run in a restricted sandbox (no network) — safe, identical scoring. The Lorbus row is our own measurement of their public quant on the same tasks, not their claim. The 80B Coder-Next row is likewise our own measurement of the UD-IQ4_XS GGUF (via LM Studio) on the same task set — a bigger, different model shown for scale. Reproduce it, don't trust it.

---

Surgically gentle abliteration

Abliteration strips the refusal direction — done carelessly it also scrambles the model's brains. Heretic co-minimizes refusals and KL divergence from the original. Lower KL = closer to the untouched model = intelligence kept. We tuned for minimum drift:

Approach	KL divergence ↓ (gentler)	Refusals
Typical manual abliteration	0.45 – 1.04	low
Heretic — flagship showcase (Gemma-3-12B)	0.16	3/100
➡️ This model (Qwen3.6-27B)	0.0089	11/100

~18× below Heretic's own showcase, 50–100× below manual. The model barely moved from the base, so <think> reasoning and coding stayed intact. (KL is model/harness-dependent, so cross-model figures aren't a strict race — the order-of-magnitude gap is the point.) We kept 11/100 refusals as the price of rock-bottom drift — brains first.

Kept in BF16 (not quantized) — surgical precision

Component	Tensors	Why
🧬 MTP head	15× mtp.*	makes speculative decoding work
👁️ Vision tower	333× visual.*	full image/video, untouched by quant
🌊 Mamba/GDN control	A_log, conv1d, dt_bias, in_proj_a/b, norm	quantizing these silently kills long-context logic

Everything heavy is 4-bit; only the fragile bits that carry reasoning, speed and vision stay full-precision.

---

Speed — MTP speculative decoding

Honest 768-token × 3-run measurement on a single RTX 4090 (48 GB), vLLM 0.19:

MTP n	short tok/s	28k-ctx tok/s	acceptance @28k
1	72	39	78%
2	91	46	66%
4 ⭐	115	49	46%
5	119	45	37%

Use num_speculative_tokens: 4. Speculative decoding is lossless — n changes speed only, never output. All presets use n=4.

---

Multimodal — sees images & video

The full vision-language model, not a text-only trim. The entire vision tower (333 tensors) is kept in full BF16 — quantization never touched it — so image/video understanding is identical to the pristine base. Serve with --limit-mm-per-prompt '{"image":4,"video":1}'. Published benchmarks cover text/coding/tool-use; vision quality is inherited, not separately re-scored.

---

Quickstart — pick your GPU

GPU	VRAM	Script	max-model-len	gpu-mem-util
RTX 3090 / 4090	24 GB	run_24gb.sh	40 960	0.95
RTX 5090	32 GB	run_32gb.sh	200 000	0.92
RTX 4090 48G / A6000 / L40S	48 GB	run_48gb.sh	200 000	0.60

vllm serve antonyMox/Qwen3.6-27B-abliterated-AutoRound-INT4-MTP \
  --quantization auto-round \
  --max-model-len 40960 \
  --gpu-memory-utilization 0.95 \
  --max-num-seqs 1 \
  --kv-cache-dtype fp8_e4m3 \
  --enable-prefix-caching \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --reasoning-parser qwen3 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":4}' \
  --trust-remote-code

Long context is cheap here: Qwen3.6 is a hybrid (Mamba/GDN + attention), so KV costs only ~37 MB/1k tokens at fp8. The model's native maximum is 256k (262,144) tokens — how much of it fits depends on how much VRAM you give vLLM:

Context vs mem-util (48 GB card, measured)

gpu-mem-util	context that fits	~free VRAM	notes
0.48	~48k	~25 GB	leaves lots of room for a 2nd model
0.60 ⭐ (default preset)	~200k	~19 GB	balanced — room for another model on the same card
0.65	~256k	~17 GB	full native context
0.68	256k · conc 1.30x	~15 GB	comfortable headroom

Our default preset is a deliberate 0.60 / 200k — it keeps ~19 GB free so you can load a second model on the same card. Want the full 256k? Just raise --gpu-memory-utilization to ~0.65–0.68.

---

⚠️ Pitfall — never set --max-model-len at exact KV capacity

On vLLM 0.19 hybrid models: if max-model-len equals KV capacity (startup log shows Maximum concurrency ... : 1.00x), requests in the last ~3 % of the window pass validation then hang forever — no response, no error, nothing logged, GPU idle. The scheduler can't allocate the final KV block (draft tokens need headroom the validator ignores). Above the window → instant HTTP 400 (fine); below the danger zone → fine. Only the top sliver dead-locks.

Keep --max-model-len ≥ 2 attention blocks (2 × 1632) below capacity — our presets already do. Check the log: Maximum concurrency for N tokens: 1.13x must be ≥ ~1.10x, not 1.00x. Also: the log line GPU KV cache size: X tokens understates real capacity ~3× on hybrids (it divides by KV-cache groups) — trust only the Maximum concurrency line.

---

Long-context recall (measured)

Needle test on a 150k-token prompt (48 GB preset), exact-quote recall by depth:

Depth	1%	25%	40%	46%	50%	56%	75%	100%
Recall	✅	✅	✅	✅	⚠️*	✅	✅	✅

7/8 exact. The miss (*) at 50% returned the neighbouring line — mild "lost-in-the-middle", not a hallucination. Prefix caching works: a repeat 150k query answered in 5.5 s vs 131 s cold.

---

🧬 The family

Clean quants of the same base — pick by hardware:

• wide · 26 GB · hard 78 🏆 — maximum reasoning, for 48 GB+ cards.
• abliterated · 19 GB · hard 70 (this one) — uncensored, fits 24 GB cards.
• 35B-A3B · 22 GB · hard 48 (coming soon) — fast MoE (~188 tok/s).

All ours: AutoRound INT4, MTP preserved, multimodal, published benchmarks. Measured against the community reference Lorbus narrow INT4 (hard 70) — see the comparison table above.

---

Recipe

1. Base — pristine Qwen/Qwen3.6-27B BF16.
2. Abliteration — Heretic, gentle: KL 0.0089, refusals 11/100, <think> intact.
3. Quant — AutoRound W4A16, group 128, 200 iters, dense. BF16-kept: mtp.*, visual.*, Mamba control.
4. MTP — 15 mtp.* tensors grafted from the pristine BF16.
5. Sampler — ships temp 1.0 / top_p 0.95 / top_k 20 (generation_config.json).

---

Why HuggingFace shows "~6.7B params"

Not a small model — it's how the widget reads a packed INT4 quant. The 800 quantized tensors pack 8× int4 into each int32, so HF counts 3.06 B int32 slots, not the ~24 B real int4 weights. With the BF16 parts, the true model is Qwen3.6-27B, 19 GB. Every AutoRound/GPTQ INT4 looks like this. Verified: language_model 4.55 B · other 1.27 B · vision 0.46 B · mtp 0.42 B.

---

⚠️ Usage & responsibility

This is an uncensored research artifact — read before use.

• Reduced refusals by design — can produce content a safety-aligned model would decline.
• No safety guarantees — outputs are not filtered; review before use, especially in production.
• You are responsible — for lawful, ethical use in your jurisdiction; all consequences rest with you.
• No liability — provided "as is" as a community artifact, no warranty of any kind.
• Not for harm — for research, evaluation, and lawful applications only.

License

Apache-2.0, same as the base. Quantized & abliterated from Qwen/Qwen3.6-27B. Community build; benchmark numbers from our own harness — JSON included, methodology open.