Update README.md
Browse files
README.md
CHANGED
|
@@ -169,18 +169,6 @@ Hello! As an AI, I don't have consciousness in the way humans do, but I am fully
|
|
| 169 |
|
| 170 |
We rely on [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) to evaluate the quality of the quantized model.
|
| 171 |
|
| 172 |
-
Need to install lm-eval from source: https://github.com/EleutherAI/lm-evaluation-harness#install
|
| 173 |
-
|
| 174 |
-
## baseline
|
| 175 |
-
```Shell
|
| 176 |
-
lm_eval --model hf --model_args pretrained=microsoft/Phi-4-mini-instruct --tasks hellaswag --device cuda:0 --batch_size 8
|
| 177 |
-
```
|
| 178 |
-
|
| 179 |
-
## int8 dynamic activation and int4 weight quantization (8da4w)
|
| 180 |
-
```Shell
|
| 181 |
-
lm_eval --model hf --model_args pretrained=pytorch/Phi-4-mini-instruct-8da4w --tasks hellaswag --device cuda:0 --batch_size 8
|
| 182 |
-
```
|
| 183 |
-
|
| 184 |
| Benchmark | | |
|
| 185 |
|----------------------------------|----------------|---------------------------|
|
| 186 |
| | Phi-4-mini-ins | Phi-4-mini-instruct-8da4w |
|
|
@@ -203,6 +191,21 @@ lm_eval --model hf --model_args pretrained=pytorch/Phi-4-mini-instruct-8da4w --t
|
|
| 203 |
| Mathqa (0-shot) | 42.31 | 36.95 |
|
| 204 |
| **Overall** | 55.35 | 48.45 |
|
| 205 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 206 |
|
| 207 |
# Exporting to ExecuTorch
|
| 208 |
|
|
|
|
| 169 |
|
| 170 |
We rely on [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) to evaluate the quality of the quantized model.
|
| 171 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 172 |
| Benchmark | | |
|
| 173 |
|----------------------------------|----------------|---------------------------|
|
| 174 |
| | Phi-4-mini-ins | Phi-4-mini-instruct-8da4w |
|
|
|
|
| 191 |
| Mathqa (0-shot) | 42.31 | 36.95 |
|
| 192 |
| **Overall** | 55.35 | 48.45 |
|
| 193 |
|
| 194 |
+
<details>
|
| 195 |
+
<summary> Reproduce Model Quality Results </summary>
|
| 196 |
+
|
| 197 |
+
Need to install lm-eval from source: https://github.com/EleutherAI/lm-evaluation-harness#install
|
| 198 |
+
|
| 199 |
+
## baseline
|
| 200 |
+
```Shell
|
| 201 |
+
lm_eval --model hf --model_args pretrained=microsoft/Phi-4-mini-instruct --tasks hellaswag --device cuda:0 --batch_size 8
|
| 202 |
+
```
|
| 203 |
+
|
| 204 |
+
## int8 dynamic activation and int4 weight quantization (8da4w)
|
| 205 |
+
```Shell
|
| 206 |
+
lm_eval --model hf --model_args pretrained=pytorch/Phi-4-mini-instruct-8da4w --tasks hellaswag --device cuda:0 --batch_size 8
|
| 207 |
+
```
|
| 208 |
+
</details>
|
| 209 |
|
| 210 |
# Exporting to ExecuTorch
|
| 211 |
|