gpt-oss-120b-reap-48

A surgically expert-pruned openai/gpt-oss-120b: 48 of 128 experts kept per layer (top-48 by REAP saliency), reducing the checkpoint from 65.2 GB to 26.4 GB while matching the full model on the target task. Native MXFP4 expert weights are preserved unmodified; serves in stock vLLM (CUDA and Intel XPU) and converts to GGUF.

• 36 layers x 48 experts, top-4 routing, ~45B total params, ~5.1B active
• Fits a single 32 GB GPU with 128k context (fp8 KV cache)
• Router softmax renormalizes natively over surviving experts; no retraining

Method

Experts ranked per layer by REAP saliency (Cerebras, arXiv:2510.13999): the mean over routed tokens of (renormalized router gate x L2 norm of the expert output), computed layerwise on a single RTX 4080 (16 GB) with MXFP4 dequantized on the fly. Calibration: 60 production prompts from a resume/job scoring workload plus the full model's own generation traces (harmony analysis channel + final JSON). Calibrating on prompt text alone destroys output-format compliance; including generation traces is what makes deep pruning survivable. Saliency reproduces across independent hosts/GPUs at rank correlation 0.997 (top-48 set overlap 99.2%).

Evaluation (frozen 311-prompt production replay, decision kappa vs stored outcomes)

model	kappa vs prod	agreement vs full-120b	parse
gpt-oss-120b (full, self-consistency)	0.485-0.527	0.890	100%
this model	0.508-0.538	0.823-0.835	100%
gpt-oss-20b	0.456	0.781	100%
frequency-pruned 48/128 (ablation)	0.149	0.641	87%

Kappa vs production decisions falls inside the full model's own self-consistency range, i.e. task-equal within measurement noise, while beating gpt-oss-20b on every metric. Numbers replicate across CUDA (RTX 5090) and Intel XPU (Arc Pro B-series) backends. EAGLE3 speculative decoding with RedHatAI/gpt-oss-120b-speculator.eagle3 works as-is (~40% draft acceptance, mean acceptance length 2.1).

Scope: pruning was calibrated for a narrow scoring task. General-domain capability outside that distribution is reduced and unmeasured here. For general use, prune with your own calibration traffic (tooling in the repo below).

Tooling

Pruning, saliency, and eval scripts: https://github.com/bryanvine/gpt-oss-subset (Apache 2.0). Write-up: https://bryanvine.github.io/gpt-oss-pruning/ (calibration workload: the resume/job scoring service behind https://reach.jobs). Derived from openai/gpt-oss-120b (Apache 2.0); usage subject to the gpt-oss usage policy.