ynhe commited on
Commit
2d2f9c8
·
verified ·
1 Parent(s): 887d1ac

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -95,9 +95,9 @@ model-index:
95
 
96
  - Inference Speed
97
 
98
- We measured the average inference speed (tokens/s) of generating 1024 new tokens and 5198 (8192-2998) tokens with the context of an video (which takes 2998 tokens) under BF16 precision.
99
 
100
- |Quantization | Speed (3022 tokens) | Speed (8192 tokens) w/o vision| Speed(8192 tokens) w/ vision|
101
  |--- |--- |---| ---|
102
  |BF16 | 33.40 | 31.91 | 21.33|
103
  |INT4 | - | 31.95 | - |
 
95
 
96
  - Inference Speed
97
 
98
+ We measured the average inference speed (tokens/s) of generating 1024 new tokens and 5198 (8192-2998) tokens with the context of an video (which takes 2998 tokens) under BF16 precision. w/ encoder indicates that the inference includes the time for video encoder.
99
 
100
+ |Quantization | Speed (3022 tokens) | Speed (8192 tokens) w/o encoder| Speed(8192 tokens) w/ encoder|
101
  |--- |--- |---| ---|
102
  |BF16 | 33.40 | 31.91 | 21.33|
103
  |INT4 | - | 31.95 | - |