Hugging Face's logo

RedHatAI
/
granite-3.1-8b-instruct-quantized.w4a16

Text Generation
Safetensors
English
granite
w4a16
int4
vllm
conversational
compressed-tensors
Model card Files
xet
Community
1
nm-research commited on
Commit
b35f64e
·
verified ·
1 Parent(s): d9ba0ad

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -205,12 +205,12 @@ evalplus.evaluate \
205
 
206
  | Metric | ibm-granite/granite-3.1-8b-instruct | neuralmagic-ent/granite-3.1-8b-instruct-quantized.w4a16 |
207
  |-----------------------------------------|:---------------------------------:|:-------------------------------------------:|
208
- | IFEval (Inst Level Strict Acc, 0-shot)| | 73.14 |
209
  | BBH (Acc-Norm, 3-shot) | 53.19 | 51.52 |
210
  | Math-Hard (Exact-Match, 4-shot) | 14.77 | 16.66 |
211
  | GPQA (Acc-Norm, 0-shot) | 31.76 | 29.91 |
212
  | MUSR (Acc-Norm, 0-shot) | 46.01 | 45.75 |
213
- | MMLU-Pro (Acc, 5-shot) | 74.01 | 0.3423 |
214
  | **Average Score** | **42.61** | **41.87** |
215
  | **Recovery** | **100.00** | **98.26** |
216
 
 
205
 
206
  | Metric | ibm-granite/granite-3.1-8b-instruct | neuralmagic-ent/granite-3.1-8b-instruct-quantized.w4a16 |
207
  |-----------------------------------------|:---------------------------------:|:-------------------------------------------:|
208
+ | IFEval (Inst Level Strict Acc, 0-shot)| 74.01 | 73.14 |
209
  | BBH (Acc-Norm, 3-shot) | 53.19 | 51.52 |
210
  | Math-Hard (Exact-Match, 4-shot) | 14.77 | 16.66 |
211
  | GPQA (Acc-Norm, 0-shot) | 31.76 | 29.91 |
212
  | MUSR (Acc-Norm, 0-shot) | 46.01 | 45.75 |
213
+ | MMLU-Pro (Acc, 5-shot) | 35.81 | 34.23 |
214
  | **Average Score** | **42.61** | **41.87** |
215
  | **Recovery** | **100.00** | **98.26** |
216