machinez
/

Codestral-22B-v0.1-exl2

text-generation-inference

Model card Files Files and versions

machinez commited on May 29, 2024

Commit

12fd1bd

·

verified ·

1 Parent(s): e96f3e8

Update README.md

Model branches and sizes

Files changed (1) hide show

README.md +7 -1

README.md CHANGED Viewed

@@ -26,7 +26,13 @@ Finally quantized (eg. 4.0bpw)
 python3 convert.py -i ~/models/Codestral-22B-v0.1-hf/ -o /tmp/exl2/ -nr -m ~/models/Machinez_Codestral-22B-v0.1-exl2/measurement.json -cf ~/models/Machinez_Codestral-22B-v0.1-exl2_4.0bpw/ -b 4.0
 ```
 # Model Card for Codestral-22B-v0.1

 python3 convert.py -i ~/models/Codestral-22B-v0.1-hf/ -o /tmp/exl2/ -nr -m ~/models/Machinez_Codestral-22B-v0.1-exl2/measurement.json -cf ~/models/Machinez_Codestral-22B-v0.1-exl2_4.0bpw/ -b 4.0
 ```
+## Quantization
+- [4.0bpw](https://huggingface.co/machinez/Codestral-22B-v0.1-exl2/tree/4_0) 11.5gb
+- [5.0bpw](https://huggingface.co/machinez/Codestral-22B-v0.1-exl2/tree/5_0) 14.0gb
+- [5.5bpw](https://huggingface.co/machinez/Codestral-22B-v0.1-exl2/tree/5_5)
+- [6.0bpw](https://huggingface.co/machinez/Codestral-22B-v0.1-exl2/tree/6_0) 17.0gb
+- [7.0bpw](https://huggingface.co/machinez/Codestral-22B-v0.1-exl2/tree/7_0)
+- [8.0bpw](https://huggingface.co/machinez/Codestral-22B-v0.1-exl2/tree/8_0) 21.0gb
 # Model Card for Codestral-22B-v0.1