Update README.md

README.md

@@ -15,11 +15,23 @@ base_model_relation: quantized
 # DeepSeek-R1-0528-GPTQ-Int4-Int8Mix-Compact
 Base mode [deepseek-ai/DeepSeek-R1-0528](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528)
 
-This repository contains a mixed-precision (Int4 + selective Int8) GPTQ version of DeepSeek-R1-0528 for vLLM. We began with a standard 4-bit (AWQ/GPTQ) conversion that follows vLLM’s default quantization layout, but early tests showed that a fully-Int4 model could not meet the compute demands of this checkpoint and may produce unstable outputs.
+This repository delivers an Int4 + selectively-Int8 GPTQ `DeepSeek-R1-0528` model: only layers that are highly sensitive to quantization remain in Int8, while the rest stay Int4—preserving generation quality with minimal file-size overhead.
 
-Guided by this preliminary analysis, we introduced targeted, per-layer Int8 refinement: only the layers most sensitive to quantization are stored in lnt8 (the compadct version has more int8 layers), while the rest remain Int4. This keeps the file-size increase minimal compared with the pure 4-bit baseline while restoring response quality.
+Preliminary trials show that converting the entire model to pure Int4 (AWQ/GPTQ) under the quantization layout used in vLLM’s current DeepSeek-R1 implementation degrades inference accuracy and can produce faulty outputs. Layer-wise fine-grained quantization substantially mitigates this issue.
 
-Currently, vllm==0.9.0 does not support per-layer quantization settings for the moe module. I've provided a basic implementation by adding the get_moe_quant_method function within the gptq_marlin.py file. Before the PR is merged, please replace the corresponding file with the attached one.
+    Temporary patch:
+    vLLM == 0.9.0 does not yet natively support per-layer quantization for MoE modules.
+    We added get_moe_quant_method to gptq_marlin.py as an interim fix.
+    Until the upstream PR is merged, please replace the original file with the one provided in this repo.
+
+Variant Overview
+
+| Variant     | Characteristics                                                         | File Size | Recommended Scenario                                     |
+|-------------|-------------------------------------------------------------------------|-----------|----------------------------------------------------------|
+| **Compact** | More Int8 layers, higher fidelity                                       | 414 GB    | Ample GPU memory & strict quality needs (e.g., 8 × A100) |
+| **Lite**    | Only the most critical layers upgraded to Int8; size close to pure Int4 | 355 GB    | Resource-constrained, lightweight server deployments     |
+
+Choose the variant that best matches your hardware and quality requirements.
 
 ### 【Model Update Date】
 ```