juliendenize commited on
Commit
cb17b97
·
verified ·
1 Parent(s): 0c8c06c

Update README.md (#31)

Browse files

- Update README.md (c3e22d3f9f15649c42fb3d61ed29e609d85d0339)

Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -126,7 +126,11 @@ We recommend that you use Mistral-Small-3.2-24B-Instruct-2506 in a server/client
126
  1. Spin up a server:
127
 
128
  ```
129
- vllm serve mistralai/Mistral-Small-3.2-24B-Instruct-2506 --tokenizer_mode mistral --config_format mistral --load_format mistral --tool-call-parser mistral --enable-auto-tool-choice --limit_mm_per_prompt 'image=10' --tensor-parallel-size 2
 
 
 
 
130
  ```
131
 
132
  **Note:** Running Mistral-Small-3.2-24B-Instruct-2506 on GPU requires ~55 GB of GPU RAM in bf16 or fp16.
 
126
  1. Spin up a server:
127
 
128
  ```
129
+ vllm serve mistralai/Mistral-Small-3.2-24B-Instruct-2506 \
130
+ --tokenizer_mode mistral --config_format mistral \
131
+ --load_format mistral --tool-call-parser mistral \
132
+ --enable-auto-tool-choice --limit-mm-per-prompt '{"image":10}' \
133
+ --tensor-parallel-size 2
134
  ```
135
 
136
  **Note:** Running Mistral-Small-3.2-24B-Instruct-2506 on GPU requires ~55 GB of GPU RAM in bf16 or fp16.