Update README.md
Browse files
README.md
CHANGED
@@ -37,6 +37,9 @@ python3 quantize_quark.py --model_dir "meta-llama/Meta-Llama-3.1-405B-Instruct"
|
|
37 |
```
|
38 |
|
39 |
# Deployment
|
|
|
|
|
|
|
40 |
|
41 |
## Evaluation
|
42 |
|
|
|
37 |
```
|
38 |
|
39 |
# Deployment
|
40 |
+
### Use with vLLM
|
41 |
+
|
42 |
+
This model can be deployed efficiently using the [vLLM](https://docs.vllm.ai/en/latest/) backend.
|
43 |
|
44 |
## Evaluation
|
45 |
|