More quality results
Browse files
README.md
CHANGED
|
@@ -185,6 +185,7 @@ Okay, the user is asking if I can talk to them. First, I need to clarify that I
|
|
| 185 |
| mmlu_pro | - | - |
|
| 186 |
| bbh | - | - |
|
| 187 |
| **Reasoning** | | |
|
|
|
|
| 188 |
| gpqa_main_zeroshot | - | 27.46 |
|
| 189 |
| **Multilingual** | | |
|
| 190 |
| m_mmlu | - | - |
|
|
@@ -203,7 +204,7 @@ Need to install lm-eval from source: https://github.com/EleutherAI/lm-evaluation
|
|
| 203 |
|
| 204 |
## baseline
|
| 205 |
```Shell
|
| 206 |
-
lm_eval --model hf --model_args pretrained=
|
| 207 |
```
|
| 208 |
|
| 209 |
## int8 dynamic activation and int4 weight quantization (8da4w)
|
|
|
|
| 185 |
| mmlu_pro | - | - |
|
| 186 |
| bbh | - | - |
|
| 187 |
| **Reasoning** | | |
|
| 188 |
+
| hellaswag | - | 54.39 |
|
| 189 |
| gpqa_main_zeroshot | - | 27.46 |
|
| 190 |
| **Multilingual** | | |
|
| 191 |
| m_mmlu | - | - |
|
|
|
|
| 204 |
|
| 205 |
## baseline
|
| 206 |
```Shell
|
| 207 |
+
lm_eval --model hf --model_args pretrained=HuggingFaceTB/SmolLM3-3B --tasks mmlu --device cuda:0 --batch_size auto
|
| 208 |
```
|
| 209 |
|
| 210 |
## int8 dynamic activation and int4 weight quantization (8da4w)
|