Update README.md
Browse files
README.md
CHANGED
|
@@ -39,12 +39,14 @@ python builder.py \
|
|
| 39 |
-p int4 \
|
| 40 |
-e dml
|
| 41 |
```
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
128
|
| 46 |
-
256
|
| 47 |
-
512
|
| 48 |
-
1024
|
| 49 |
-
2048
|
|
|
|
|
|
|
| 50 |
|
|
|
|
| 39 |
-p int4 \
|
| 40 |
-e dml
|
| 41 |
```
|
| 42 |
+
## Performance Measure
|
| 43 |
+
| Prompt_len | TTFS | TPS |
|
| 44 |
+
|------------|------|------|
|
| 45 |
+
| 128 | 0.733 | 20.411 |
|
| 46 |
+
| 256 | 0.873 | 20.487 |
|
| 47 |
+
| 512 | 1.386 | 19.789 |
|
| 48 |
+
| 1024 | 2.526 | 18.627 |
|
| 49 |
+
| 2048 | 5.088 | 16.662 |
|
| 50 |
+
|
| 51 |
+
Perplexity of hardware run: **7.261**
|
| 52 |
|