Update README.md
Browse files
README.md
CHANGED
|
@@ -268,6 +268,15 @@ context size to be available with llamafile for any given model, you can
|
|
| 268 |
pass the `-c 0` flag. The default temperature for these llamafiles is 0.
|
| 269 |
It can be changed, e.g. `--temp 0.8`.
|
| 270 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 271 |
## About Quantization
|
| 272 |
|
| 273 |
Our own evaluation of this model leads us to believe that it works best
|
|
|
|
| 268 |
pass the `-c 0` flag. The default temperature for these llamafiles is 0.
|
| 269 |
It can be changed, e.g. `--temp 0.8`.
|
| 270 |
|
| 271 |
+
## Benchmarks
|
| 272 |
+
|
| 273 |
+
| cpu\_info | model\_filename | size | test | t/s |
|
| 274 |
+
| -----------------------------------------: | ---------------------------------------: | ---------: | ------------: | --------------: |
|
| 275 |
+
| AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q8\_0 | 33.82 GiB | pp512 | 94.34 |
|
| 276 |
+
| AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q8\_0 | 33.82 GiB | tg16 | 5.61 |
|
| 277 |
+
| AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q5\_0 | 22.03 GiB | pp512 | 95.08 |
|
| 278 |
+
| AMD Ryzen Threadripper PRO 7995WX (znver4) | granite-34b-code-instruct.Q5\_0 | 22.03 GiB | tg16 | 7.78 |
|
| 279 |
+
|
| 280 |
## About Quantization
|
| 281 |
|
| 282 |
Our own evaluation of this model leads us to believe that it works best
|