Update README.md
Browse files
README.md
CHANGED
@@ -19,12 +19,11 @@ base_model:
|
|
19 |
- **KV cache quantization:** OCP FP8
|
20 |
- **Calibration Dataset:** [Pile](https://huggingface.co/datasets/mit-han-lab/pile-val-backup)
|
21 |
|
22 |
-
|
23 |
-
|
24 |
|
25 |
# Model Quantization
|
26 |
|
27 |
-
|
28 |
|
29 |
**Quantization scripts:**
|
30 |
```
|
|
|
19 |
- **KV cache quantization:** OCP FP8
|
20 |
- **Calibration Dataset:** [Pile](https://huggingface.co/datasets/mit-han-lab/pile-val-backup)
|
21 |
|
22 |
+
This model is a quantized version of [meta-llama/Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct),optimized using [AMD-Quark](https://quark.docs.amd.com/latest/index.html) framework with MXFP4 quantization.
|
|
|
23 |
|
24 |
# Model Quantization
|
25 |
|
26 |
+
The model was quantized from [meta-llama/Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct) using [AMD-Quark](https://quark.docs.amd.com/latest/index.html). Weights and activations were quantized to MXFP4, and KV caches were quantized to FP8. The AutoSmoothQuant algorithm was applied to enhance accuracy during quantization.
|
27 |
|
28 |
**Quantization scripts:**
|
29 |
```
|