Qwen3.5-4B-DFlash-MLX-6bit

MLX 6-bit quantized version of Qwen3.5-4B-DFlash for Apple Silicon.

This release provides a compact MLX conversion of the DFlash drafter model, reducing memory usage while preserving compatibility with speculative decoding workflows on Apple Silicon devices.

About DFlash

DFlash is a speculative decoding framework that uses a lightweight block diffusion model to generate draft tokens in parallel, allowing a larger target model to verify multiple tokens at once. Compared to traditional autoregressive drafting methods, DFlash can substantially improve inference throughput while maintaining output quality.

This model is the --drafter component-- and must be used together with a compatible target model:

Qwen3.5-4B
Qwen3.5-4B-Instruct
Other compatible Qwen3.5 4B variants

It is --not a standalone chat model--.

Quantization

This release uses:

Format: MLX
Quantization: Uniform 6-bit
Group Size: 64
Target Platform: Apple Silicon

Unlike oQ releases, this model does --not-- use sensitivity-aware mixed precision quantization. All supported weights are quantized uniformly to 6 bits.

Recommended Usage

DFlash works by pairing a draft model with a target model.

Example MLX workflow:

from dflash.model_mlx import load, load_draft, stream_generate

model, tokenizer = load("Qwen/Qwen3.5-4B")
draft = load_draft("Qwen3.5-4B-DFlash-MLX-6bit")

messages = [
    {
        "role": "user",
        "content": "Explain speculative decoding."
    }
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=True,
)

for r in stream_generate(
    model,
    draft,
    tokenizer,
    prompt,
    block_size=16,
    max_tokens=2048,
    temperature=0.6,
):
    print(r.text, end="", flush=True)

Recommended Settings

For best results:

temp: 1.0
top_p: 0.95
top_k: 20
min_p: 0
rep_penalty: 1.05
presence_penalty: 1.5
enable_thinking: true

These settings provide a good balance between exploration, acceptance rate, and generation quality when paired with a Qwen3.5 target model.

Intended Use

This model is intended for:

Speculative decoding
Low-latency inference
Apple Silicon deployments
MLX inference pipelines
Research into block diffusion drafting

This model is not intended to be used as a standalone assistant.

Hardware Recommendations

Recommended:

Apple M1 Pro / Max
Apple M2 Pro / Max / Ultra
Apple M3 Series
Apple M4 Series

Works well with:

MLX
DFlash MLX
Custom inference frameworks
Apple Silicon local deployments

Credits

z-lab DFlash Team
Qwen Team
Apple MLX

References

DFlash introduces block diffusion speculative decoding, enabling parallel draft generation and significant inference acceleration compared with traditional autoregressive drafting approaches.

Citation

@article{chen2026dflash,
  title={DFlash: Block Diffusion for Flash Speculative Decoding},
  author={Chen, Jian and Liang, Yesheng and Liu, Zhijian},
  year={2026},
  eprint={2602.06036},
  archivePrefix={arXiv},
  primaryClass={cs.LG}
}