amd
/

Llama-3.1-405B-Instruct-MXFP4-Preview

8-bit precision

Model card Files Files and versions Community

linzhao-amd commited on 24 days ago

Commit

ed6154a

·

verified ·

1 Parent(s): 00fb3f3

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -36,7 +36,7 @@ python3 quantize_quark.py --model_dir "meta-llama/Llama-3.1-405B-Instruct" \
                           --quant_algo autosmoothquant \
                           --min_kv_scale 1.0 \
                           --model_export hf_format \
-                          --output_dir $output_path \
                           --multi_gpu
 ```

                           --quant_algo autosmoothquant \
                           --min_kv_scale 1.0 \
                           --model_export hf_format \
+                          --output_dir amd/Llama-3.1-405B-Instruct-MXFP4 \
                           --multi_gpu
 ```