Update README.md
Browse files
README.md
CHANGED
|
@@ -85,6 +85,9 @@ vllm serve pytorch/Phi-4-mini-instruct-float8dq --tokenizer microsoft/Phi-4-mini
|
|
| 85 |
|
| 86 |
# Model Quality
|
| 87 |
We rely on [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) to evaluate the quality of the quantized model.
|
|
|
|
|
|
|
|
|
|
| 88 |
|
| 89 |
## baseline
|
| 90 |
```
|
|
@@ -120,6 +123,11 @@ lm_eval --model hf --model_args pretrained=pytorch/Phi-4-mini-instruct-float8dq
|
|
| 120 |
|
| 121 |
# Model Performance
|
| 122 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 123 |
## Results (H100 machine)
|
| 124 |
| Benchmark | | |
|
| 125 |
|----------------------------------|----------------|--------------------------|
|
|
|
|
| 85 |
|
| 86 |
# Model Quality
|
| 87 |
We rely on [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) to evaluate the quality of the quantized model.
|
| 88 |
+
Need to install lm-eval from source:
|
| 89 |
+
https://github.com/EleutherAI/lm-evaluation-harness#install
|
| 90 |
+
|
| 91 |
|
| 92 |
## baseline
|
| 93 |
```
|
|
|
|
| 123 |
|
| 124 |
# Model Performance
|
| 125 |
|
| 126 |
+
Need to install vllm nightly to get some recent changes
|
| 127 |
+
```
|
| 128 |
+
pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
|
| 129 |
+
```
|
| 130 |
+
|
| 131 |
## Results (H100 machine)
|
| 132 |
| Benchmark | | |
|
| 133 |
|----------------------------------|----------------|--------------------------|
|