How to use it? Can I have a conversation?
#10
by
viaho
- opened
General conversation or specific questions？？？
You can use vLLM or SGLang to serve it and then use the OpenAI chat completions API to poll the model!