ibm-ai-platform
/

granite-7b-lab-accelerator

Model card Files Files and versions

JRosenkranz commited on May 1, 2024

Commit

f76a90c

·

verified ·

1 Parent(s): 226bd2a

Update README.md

Files changed (1) hide show

README.md +2 -2

README.md CHANGED Viewed

@@ -145,7 +145,7 @@ python fms-extras/scripts/paged_speculative_inference.py \
     --speculator_source=hf \
     --speculator_variant=1_4b \
     --top_k_tokens_per_head=4,3,2,2,2 \
-    --compile \
 ```
 ##### batch_size=4 (compile)
@@ -162,5 +162,5 @@ python fms-extras/scripts/paged_speculative_inference.py \
     --speculator_variant=1_4b \
     --top_k_tokens_per_head=4,3,2,2,2 \
     --batch_input \
-    --compile \
 ```

     --speculator_source=hf \
     --speculator_variant=1_4b \
     --top_k_tokens_per_head=4,3,2,2,2 \
+    --compile
 ```
 ##### batch_size=4 (compile)
     --speculator_variant=1_4b \
     --top_k_tokens_per_head=4,3,2,2,2 \
     --batch_input \
+    --compile
 ```