ibm-ai-platform
/

codellama-13b-accelerator

Model card Files Files and versions

JRosenkranz commited on Apr 26, 2024

Commit

ab784c3

·

verified ·

1 Parent(s): f392e11

Update README.md

Files changed (1) hide show

README.md +3 -0

README.md CHANGED Viewed

@@ -144,6 +144,7 @@ python fms-extras/scripts/paged_speculative_inference.py \
     --tokenizer=$MODEL_PATH \
     --speculator_path=ibm-fms/codellama-13b-accelerator \
     --speculator_source=hf \
     --top_k_tokens_per_head=4,3,2,2,2,2,2 \
     --prompt_type=code \
     --compile \
@@ -161,6 +162,7 @@ python fms-extras/scripts/paged_speculative_inference.py \
     --tokenizer=$MODEL_PATH \
     --speculator_path=ibm-fms/codellama-13b-accelerator \
     --speculator_source=hf \
     --top_k_tokens_per_head=4,3,2,2,2,2,2 \
     --prompt_type=code \
     --compile \
@@ -177,6 +179,7 @@ python fms-extras/scripts/paged_speculative_inference.py \
     --tokenizer=$MODEL_PATH \
     --speculator_path=ibm-fms/codellama-13b-accelerator \
     --speculator_source=hf \
     --batch_input \
     --top_k_tokens_per_head=4,3,2,2,2,2,2 \
     --prompt_type=code \

     --tokenizer=$MODEL_PATH \
     --speculator_path=ibm-fms/codellama-13b-accelerator \
     --speculator_source=hf \
+    --speculator_variant=2b \
     --top_k_tokens_per_head=4,3,2,2,2,2,2 \
     --prompt_type=code \
     --compile \
     --tokenizer=$MODEL_PATH \
     --speculator_path=ibm-fms/codellama-13b-accelerator \
     --speculator_source=hf \
+    --speculator_variant=2b \
     --top_k_tokens_per_head=4,3,2,2,2,2,2 \
     --prompt_type=code \
     --compile \
     --tokenizer=$MODEL_PATH \
     --speculator_path=ibm-fms/codellama-13b-accelerator \
     --speculator_source=hf \
+    --speculator_variant=2b \
     --batch_input \
     --top_k_tokens_per_head=4,3,2,2,2,2,2 \
     --prompt_type=code \