linzhao-amd commited on
Commit
b7a882b
·
verified ·
1 Parent(s): 7b7373c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -3
README.md CHANGED
@@ -19,12 +19,11 @@ base_model:
19
  - **KV cache quantization:** OCP FP8
20
  - **Calibration Dataset:** [Pile](https://huggingface.co/datasets/mit-han-lab/pile-val-backup)
21
 
22
- The model is the quantized version of the [Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct) model, which is an auto-regressive language model that uses an optimized transformer architecture. For more information, please check [here](https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct). The MXFP4 model is quantized with [AMD-Quark](https://quark.docs.amd.com/latest/index.html).
23
-
24
 
25
  # Model Quantization
26
 
27
- This model was obtained by quantizing [Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct)'s weights and activations to MXFP4 and KV caches to FP8, using AutoSmoothQuant algorithm in [AMD-Quark](https://quark.docs.amd.com/latest/index.html).
28
 
29
  **Quantization scripts:**
30
  ```
 
19
  - **KV cache quantization:** OCP FP8
20
  - **Calibration Dataset:** [Pile](https://huggingface.co/datasets/mit-han-lab/pile-val-backup)
21
 
22
+ This model is a quantized version of [meta-llama/Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct),optimized using [AMD-Quark](https://quark.docs.amd.com/latest/index.html) framework with MXFP4 quantization.
 
23
 
24
  # Model Quantization
25
 
26
+ The model was quantized from [meta-llama/Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-405B-Instruct) using [AMD-Quark](https://quark.docs.amd.com/latest/index.html). Weights and activations were quantized to MXFP4, and KV caches were quantized to FP8. The AutoSmoothQuant algorithm was applied to enhance accuracy during quantization.
27
 
28
  **Quantization scripts:**
29
  ```