| ## Usage: Serving with SGLang | |
| Download the weights. You can replace `/local/grok-2` with any other folder name you prefer. | |
| ``` | |
| hf download xai-org/grok-2 --local-dir /local/grok-2 | |
| ``` | |
| Launch a server. | |
| ``` | |
| python3 -m sglang.launch_server --model /local/grok-2 --tokenizer-path /local/grok-2/tokenizer.tok.json --tp 8 --quantization fp8 --attention-backend triton | |
| ``` | |
| Send a request. | |
| ``` | |
| python3 -m sglang.test.send_one --prompt "Human: What is your name? <|separator|>\n\nAssistant:" | |
| ``` |