Qwen3.8-27B MQ V2 ladder for hipfire

Text-only Qwen3.8-27B artifacts for hipfire, the Rust-native LLM inference engine for AMD GPUs.

This repository contains the admitted MQ3V2 through MQ6V2 product ladder and matching DFlash drafters. MQ4V2 supersedes MQ4V1 as hipfire's canonical mq4 product. Artifacts remain self-describing by HFQM quant-type ID; old MQ4V1 files are still readable but are not the current product.

Product filenames

Each bit-width has exactly three tiers:

• mqN-xt: embeddings protected; lm_head and recurrent outputs remain at the base codec.
• mqN: embeddings and lm_head protected.
• mqN-pro: recurrent/SSM outputs also lifted. MQ3 Pro uses ssm_out:mq6v2 to stay inside the 3.x model-bpw class.

Product	File	Model bpw	Size (GB)	WT2 KLD
mq3-xt	qwen3.8-27b.mq3-xt	3.503	11.778	0.248348
mq3	qwen3.8-27b.mq3	3.753	12.619	0.153658
mq3-pro	qwen3.8-27b.mq3-pro	3.922	13.184	0.130314
mq4-xt	qwen3.8-27b.mq4-xt	4.456	14.980	0.057449
mq4	qwen3.8-27b.mq4	4.659	15.663	0.039033
mq4-pro	qwen3.8-27b.mq4-pro	4.897	16.464	0.032495
mq5-xt	qwen3.8-27b.mq5-xt	5.408	18.183	0.015028
mq5	qwen3.8-27b.mq5	5.564	18.706	0.010255
mq5-pro	qwen3.8-27b.mq5-pro	5.746	19.319	0.009006
mq6-xt	qwen3.8-27b.mq6-xt	6.361	21.386	0.004389
mq6	qwen3.8-27b.mq6	6.469	21.750	0.002771
mq6-pro	qwen3.8-27b.mq6-pro	6.596	22.174	0.002208

MQ2V2 was fully measured but is deliberately not published as a product: all three tiers produced catastrophic WT2 KLD around 12–13 and degraded/repetitive text.

DFlash drafters

Draft	File	Size (GB)	Role
MQ3V2	qwen38-27b-dflash-mq3.hfq	0.985	Optional same-bit controller
MQ4V2	qwen38-27b-dflash-mq4.hfq	1.210	Recommended controller
MQ5V2	qwen38-27b-dflash-mq5.hfq	1.434	Optional same-bit controller
MQ6V2	qwen38-27b-dflash-mq6.hfq	1.659	Optional same-bit controller

Sizes are decimal GB, matching Hugging Face.

The MQ4V2 control drafter won the matched decode screen for every admitted MQ3/MQ5/MQ6 target tier and is therefore the recommended default controller. The same-bit drafts remain available for explicit size/codec experiments.

DFlash is opt-in:

hipfire pull qwen3.8:27b
hipfire pull qwen3.8:27b-draft-mq4
hipfire run --spec dflash \
  --model-draft ~/.hipfire/models/qwen38-27b-dflash-mq4.hfq \
  qwen3.8:27b "Design a streaming Rust file deduplicator."

Usage

# Canonical MQ4V2 base tier
hipfire pull qwen3.8:27b
hipfire run qwen3.8:27b "Design a streaming Rust file deduplicator."

# Explicit ladder cells
hipfire pull qwen3.8:27b-mq3-xt
hipfire pull qwen3.8:27b-mq4-pro
hipfire pull qwen3.8:27b-mq6

Runtime defaults

Every non-draft Qwen3.8 ladder tag receives the same registry tag policy:

Setting	Value
KV cache	Q8
KV backend	VMM
Native context	262,144 tokens
Maximum output	81,920 tokens
Temperature	1.0
Top-p	0.95
Top-k	20
Min-p	0.0
Presence penalty	0.0
Repeat penalty	1.0
Reasoning effort	xhigh
Thinking budget	uncapped

Quality and performance evidence

Teacher KLD

The WT2 KLD values in the product table use 24 scored chunks against the pinned WT2 teacher with Q8 KV and prefill scoring. They measure quantization quality rather than runtime speed. A second 24-chunk validation-selection fixture (v6sel) independently preserved the same tier ordering. MQ2V2 was rejected before publication because its WT2 KLD was 12.44–13.24.

HumanEval pass@1

All 36 product × decode-mode arms ran the full 164-task HumanEval suite on gfx1201: one stateless native-serve request per task, greedy sampling, thinking off, Q8/VMM KV, and an 81,920-token output ceiling. DFlash rows use the recommended MQ4V2 controller; MTP rows use the matching product sidecar. HE/2 + HE/3 is the explicit smoke pair used throughout ladder bring-up.

Product	AR pass@1	MTP pass@1	DFlash pass@1	HE/2 + HE/3
mq3-xt	94.51%	95.12%	93.90%	pass
mq3	96.95%	96.95%	95.73%	pass
mq3-pro	96.95%	97.56%	96.95%	pass
mq4-xt	95.12%	95.73%	95.73%	pass
mq4	97.56%	97.56%	96.95%	pass
mq4-pro	97.56%	97.56%	97.56%	pass
mq5-xt	98.78%	98.17%	98.78%	pass
mq5	97.56%	97.56%	98.78%	pass
mq5-pro	98.17%	98.17%	98.17%	pass
mq6-xt	98.78%	98.78%	98.78%	pass
mq6	98.17%	98.78%	98.78%	pass
mq6-pro	98.17%	98.78%	98.17%	pass

DFlash controller screen

This fresh-process gfx1201 screen compares each target with its same-bit draft and with the MQ4V2 control draft. Decode is tok/s; $\tau$ is emitted tokens per verification cycle. MQ4V2 won or tied every admitted target, so it is the registry default while all four drafters remain downloadable.

Target	Same-bit decode	MQ4 control decode	Same-bit $\tau$	MQ4 control $\tau$
mq3-xt	237.4	264.8	11.70	13.11
mq3	225.5	250.8	11.70	13.11
mq3-pro	221.2	248.1	11.70	13.11
mq4-xt	251.6	251.6	11.70	11.70
mq4	263.3	263.3	13.11	13.11
mq4-pro	258.2	258.2	13.11	13.11
mq5-xt	221.3	225.2	13.11	13.11
mq5	214.5	217.4	13.11	13.11
mq5-pro	213.2	217.0	13.11	13.11
mq6-xt	207.8	211.5	13.11	13.11
mq6	202.8	206.6	13.11	13.11
mq6-pro	202.1	205.9	13.11	13.11

Native throughput

The full promoted product matrix on Radeon RX 7900 XTX (gfx1100) uses fixed-shape hipfire bench --matrix with Q8/VMM KV, --spec off, --backend noslots, --workload stateless, 5 measured runs, and 3 warmups. TG128@128 generates 128 tokens from an initial context of 128 tokens. The MQ4 rows are the 2026-08-24 post-multi-wave screen; MQ5 and MQ6 were refreshed on 2026-08-26 at runtime commit 363e93d77.

Product	PP512 tok/s	TG128@128 tok/s
mq4-xt	883.00	47.74
mq4	873.10	45.13
mq4-pro	831.80	43.58
mq5-xt	814.90	40.32
mq5	803.30	38.95
mq5-pro	767.50	37.92
mq6-xt	803.90	27.34
mq6	797.00	26.88
mq6-pro	759.60	27.75

These are fixture-bound medians, not performance floors. The complete five-sample arrays, artifact hashes, and binary hashes are preserved in the dated historical checkpoint.

An earlier base-tier screen at runtime commit 9b6a83ae8 used 3 measured runs, 5 warmups, and a different PP2048/single-token decode fixture:

GPU	Product	PP512 tok/s	PP2048 tok/s	TG1@128 tok/s
Radeon RX 7900 XTX (gfx1100)	mq4	995.31	972.61	45.18
Radeon RX 7900 XTX (gfx1100)	mq5	807.00	787.49	38.01
Radeon RX 7900 XTX (gfx1100)	mq6	800.04	781.35	25.93
Radeon 8060S (gfx1151)	mq3	218.61	213.02	14.86
Radeon 8060S (gfx1151)	mq4	376.01	365.17	13.46
Radeon 8060S (gfx1151)	mq5	164.83	160.80	11.60
Radeon 8060S (gfx1151)	mq6	193.18	186.80	9.52

The gfx1100 MQ3V2 batched WMMA route remains deliberately unpromoted after a device-loss incident in DFlash verification; AR uses the proven row-wise path. On gfx1201, the full HumanEval AR runs above measured median prefill/decode throughput of 514.35/37.3 tok/s (mq3), 623.5/33.1 (mq4), 470.2/28.1 (mq5), and 582.0/25.1 (mq6). HumanEval prompt lengths vary, so those gfx1201 medians are not fixed-shape PP512 measurements and are reported separately.

Wire format

MQ V2 uses an 8-byte dual-FP16 affine header per 256-weight group: one scale/zero pair for weights 0–127 and one for 128–255. Wire IDs are MQ4V2 qt44, MQ6V2 qt47, MQ5V2 qt48, and MQ3V2 qt49.

License and attribution

The upstream Qwen3.8 weights are Apache-2.0 licensed. These quantized artifacts were produced for hipfire using its MagnumQuant FWHT-rotated quantization family. See the upstream model card and hipfire provenance.