m51Lab-MiniMax-M2.7-REAP-139B-A10B-NVFP4-GGUF

NVFP4 mixed-precision GGUF quantization of the REAP-40% pruned MiniMax M2.7 model, optimised for NVIDIA Blackwell GPUs. The NVFP4 tensor types use native Blackwell FP4 hardware acceleration for maximum inference speed, while smaller tensors (norms, biases, router, embeddings) are preserved in Q4_K for accuracy.

About the Source Model

m51Lab-MiniMax-M2.7-REAP-139B-A10B is a REAP-40% pruned variant of MiniMax M2.7, reducing total parameters from 229B to 139B while preserving the 10B active parameters per token. REAP (Router-weighted Expert Activation Pruning) prunes 40% of experts per MoE block (256 → 154) based on router-gated activation patterns.

• Base model: MiniMaxAI/MiniMax-M2.7 (229B MoE, 62 layers)
• Pruning method: REAP (Lasby et al., 2025, arXiv:2510.13999)
• Pruning rate: 40% of experts per MoE block
• Active parameters: ~10B per token
• Architecture: Sparse MoE decoder-only Transformer
  • Hidden dim: 3072
  • Attention heads: 48 query / 8 KV (GQA)
  • Experts: 154 per block (top-8 activated)
  • Expert routing: Sigmoid gating with learnable bias
  • Context length: 196,608 tokens
  • Vocab: 200,064 (GPT-2 tokenizer)

Evaluation (from source)

HumanEval pass@1 (on completed): 83.3% (90/108)

All evaluations conducted on the Q4_K_M base variant. NVFP4 mixed quantization targets the same tensor patterns used by other Q4_K_M quality benchmarks, so quality should be equivalent or better (NVFP4 at 4.52 BPW vs Q4_K_M at 4.84 BPW on the output, but with hardware-accelerated inference on Blackwell).

Original Creator: m51Lab / dervig | License: Modified MIT (inherited from MiniMaxAI/MiniMax-M2.7)

Quantization Details

This is a mixed NVFP4/Q4_K quantization created using llama-quantize with --tensor-type-file on a Q4_K_M imatrix-quantized base.

Tensor Mapping

Tensor Pattern	Quantization	Rationale
attn_q	NVFP4	Large attention projection — benefits from FP4 hardware acceleration
attn_k	NVFP4	Attention key projection
attn_v	NVFP4	Attention value projection
attn_output	NVFP4	Attention output projection — large weight matrix
ffn_gate_exps	NVFP4	MoE gate expert weights — massive (1536 × 3072 × 154)
ffn_up_exps	NVFP4	MoE up expert weights — massive
ffn_down_exps	NVFP4	MoE down expert weights — massive (was Q6_K in base, significant size reduction)
All norms, biases, router, embeddings	Q4_K (fallthrough)	Small tensors, kept in Q4_K for accuracy

Size Comparison

Quantization	Size	BPW	Notes
Source (Q4_K_M imatrix)	79 GB	4.84	From mradermacher
This (NVFP4 mixed)	74 GB	4.52	~6.7% smaller, hardware-accelerated

The size reduction comes primarily from ffn_down_exps tensors which were Q6_K in the base (568 MB each) and are now NVFP4 (390 MB each) — a 31% reduction on those 62 tensors alone.

Performance

Benchmarked on single NVIDIA RTX PRO 6000 Blackwell Workstation Edition (97 GB VRAM) with llama.cpp build 9253 (commit 29f148222), CUDA sm_120, native Blackwell FP4 acceleration. Strict GPU 0 isolation (CUDA_VISIBLE_DEVICES=0).

Recommended Configuration

Setting	Value	Notes
Context size	80,000 tokens	Sweet spot; max f16 KV is 88K (88K barely fits, 90K+ OOMs)
Batch size	512	Decode is bandwidth-bound; b=512 == b=4096
KV cache type	f16 / f16	~12% faster than q8_0 on this MoE; f16 satisfies the no-below-q8 quality floor
Flash attention	on	Required for Blackwell
Speculative decoding	ngram-mod (n_match=24, n_min=4, n_max=48)	MTP heads stripped by REAP — n-gram is the only option. ~25% accept rate
Split mode	none (single GPU)	All on RTX 6000 Pro
Main GPU	0

Measured Throughput (canonical, "Give me a history of dotnet" prompt, 300 generated tokens)

Config	VRAM (MiB)	Prefill (tok/s)	Generation (tok/s)
20K ctx, f16 KV, ngram	80,577	443.0	132.9
20K ctx, f16 KV, no spec	80,973	437.1	133.6
20K ctx, q8_0 KV, no spec	78,287	404.7	120.0
80K ctx, f16 KV, ngram	95,091	—	132.7
80K ctx, q8_0 KV, no spec	86,389	404.3	120.2
131K ctx, q8_0 KV, no spec	92,967	404.1	120.1

Context Ceiling Analysis (f16 KV)

Context	Result	VRAM (MiB)	Notes
80,000	OK	95,091	Sweet spot — full perf, 2.8 GB headroom
85,000	OK	96,331	1.5 GB headroom; full perf
86,000	OK	96,511	Full perf
87,000	OK	96,759	Full perf, 1.1 GB headroom (recommended max)
88,000	OK	97,007	880 MiB headroom; prefill drops 23%, gen drops 14% under memory pressure
90,000	OOM	—	cudaMalloc failed: out of memory at load (allocating 396 MiB)
131,000 (q8_0 KV)	OK	92,967	Need q8_0 KV to fit; f16 KV needs ~31 GiB for KV cache alone

Key Findings

• Decode is purely bandwidth-bound: batch size 512–4096 all yield ~120 tok/s with q8_0 KV and ~133 tok/s with f16 KV. Larger batch sizes give no throughput benefit.
• f16 KV is faster than q8_0 on this MoE on Blackwell: ~11–13% decode speedup, with zero quality concern (f16 > q8_0 by definition, so it satisfies the no-below-q8 constraint).
• Context scaling doesn't affect decode speed: 20K and 131K contexts both yield ~120 tok/s (q8_0 KV) — context only affects VRAM and prefill.
• f16 KV costs exactly 2× q8_0 VRAM: 128 MiB/1K tokens for q8_0, 256 MiB/1K tokens for f16.
• Bandwidth ceiling is ~135 tok/s on this single 6000 Pro — no spec, batch, or context tuning exceeds this.

NVFP4 on Blackwell uses native FP4 tensor core acceleration, providing significantly higher throughput than equivalent bit-width integer quants on the same hardware. The expert weights (ffn_*_exps) — which are the largest tensors in the model — benefit most from this acceleration.

Usage

llama.cpp Server (Recommended)

llama-server \
  -m m51Lab-MiniMax-M2.7-REAP-139B-A10B.NVFP4.gguf \
  --host 127.0.0.1 --port 8080 \
  -ngl 99 \
  -c 80000 \
  -b 512 -ub 512 \
  --threads 64 \
  --flash-attn on \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --split-mode none \
  --main-gpu 0 \
  --cont-batching \
  --no-warmup \
  --spec-type ngram-mod \
  --spec-ngram-mod-n-match 24 \
  --spec-ngram-mod-n-min 4 \
  --spec-ngram-mod-n-max 48

For maximum context (use q8_0 KV to fit 131K):

  -c 131000 \
  --cache-type-k q8_0 \
  --cache-type-v q8_0
  # (omit the --spec-type args for max context + max speed; f16 KV hits OOM at 90K+)

Recommended Sampling Parameters

• Temperature: 0.6–1.0
• top_p: 0.95
• top_k: 20–40
• min_p: 0.01
• repeat_penalty: 1.1

Speculative Decoding

This model does not include MTP (Multi-Token Prediction) heads — the REAP pruning process does not preserve the MTP module from the original MiniMax M2.7. N-gram speculative decoding is supported and yields ~25% accept rate, contributing ~3-5 tok/s extra:

llama-server \
  -m m51Lab-MiniMax-M2.7-REAP-139B-A10B.NVFP4.gguf \
  -ngl 99 -c 20000 \
  --spec-type ngram-mod \
  --spec-ngram-mod-n-match 24 \
  --spec-ngram-mod-n-min 4 \
  --spec-ngram-mod-n-max 48

Hardware Requirements

• Required: NVIDIA Blackwell GPU (RTX 5090, RTX PRO 6000, B200, etc.) for NVFP4 hardware acceleration
• VRAM: ~62 GB minimum for model weights (single GPU), additional VRAM for KV cache
  • q8_0 KV: 128 MiB per 1K context tokens
  • f16 KV: 256 MiB per 1K context tokens
• Tested: Single RTX PRO 6000 (97 GB) — full benchmark sweep at contexts 20K–131K
• System RAM: 128 GB+ recommended
• CPU: 32+ cores recommended (Threadripper-class) for prompt cache management

Reproduction

# 1. Download the Q4_K_M imatrix source and imatrix file
hf download mradermacher/m51Lab-MiniMax-M2.7-REAP-139B-A10B-i1-GGUF \
  m51Lab-MiniMax-M2.7-REAP-139B-A10B.i1-Q4_K_M.gguf \
  m51Lab-MiniMax-M2.7-REAP-139B-A10B.imatrix.gguf \
  --local-dir ./source

# 2. Create tensor-type file
cat > nvfp4-tensor-types.txt << 'EOF'
attn_q=nvfp4
attn_k=nvfp4
attn_v=nvfp4
attn_output=nvfp4
ffn_gate_exps=nvfp4
ffn_up_exps=nvfp4
ffn_down_exps=nvfp4
EOF

# 3. Quantize
llama-quantize \
  --allow-requantize \
  --imatrix ./source/m51Lab-MiniMax-M2.7-REAP-139B-A10B.imatrix.gguf \
  --tensor-type-file nvfp4-tensor-types.txt \
  ./source/m51Lab-MiniMax-M2.7-REAP-139B-A10B.i1-Q4_K_M.gguf \
  m51Lab-MiniMax-M2.7-REAP-139B-A10B.NVFP4.gguf \
  Q4_K

File Information

• Model file: m51Lab-MiniMax-M2.7-REAP-139B-A10B.NVFP4.gguf (~74 GB)
• Format: GGUF v3
• Architecture: minimax-m2
• Quantization: Mixed NVFP4 (large weights) / Q4_K (small weights)
• No mmproj required — text-only model

Acknowledgments

• m51Lab — REAP-40% pruning of MiniMax M2.7 and base GGUF conversion
• dervig — Publishing the pruned model on HuggingFace
• MiniMax AI — base MiniMax M2.7 model
• mradermacher — Q4_K_M imatrix quantization and i1-GGUF variant
• ggml-org/llama.cpp — NVFP4 quantization support and inference engine (build 9253+)
• Cerebras Research — REAP method (Lasby et al., 2025)

License

Modified MIT License — inherited from MiniMaxAI/MiniMax-M2.7. See the base model repository for full license terms.

Citation

@article{lasby2025reap,
  title   = {REAP the Experts: Why Pruning Prevails for One-Shot MoE compression},
  author  = {Lasby, Mike and Hussein, Ahmed and Spyra, Jayden and Chkroun, Ivan
             and Suleiman, Oriol Sans and Ioannou, Nikoli and Hyder, Ammar Ali
             and Jacobs, Sam and Chaturvedi, Sachin and Mishra, Shreyanshu
             and Aboutalebi, Hossei and Rugol, Vasileios},
  journal = {arXiv preprint arXiv:2510.13999},
  year    = {2025}
}

@misc{minimax_m2_7,
  title  = {MiniMax-M2.7},
  author = {MiniMax AI},
  year   = {2026},
  url    = {https://huggingface.co/MiniMaxAI/MiniMax-M2.7}
}