Update README.md
Browse files
README.md
CHANGED
|
@@ -9,6 +9,7 @@ license: apache-2.0
|
|
| 9 |
</div>
|
| 10 |
|
| 11 |
<p align="center">
|
|
|
|
| 12 |
<a href="https://openchat.team">Online Demo</a> •
|
| 13 |
<a href="https://discord.gg/pQjnXvNKHY">Discord</a> •
|
| 14 |
<a href="https://huggingface.co/openchat">Huggingface</a> •
|
|
@@ -29,9 +30,9 @@ OpenChat is an innovative library of open-source language models, fine-tuned wit
|
|
| 29 |
|
| 30 |
## Usage
|
| 31 |
|
| 32 |
-
To use this model, we highly recommend installing the OpenChat package by following the [installation guide](#installation) and using the OpenChat OpenAI-compatible API server by running the serving command from the table below. The server is optimized for high-throughput deployment using [vLLM](https://github.com/vllm-project/vllm) and can run on a consumer GPU with 24GB RAM. To enable tensor parallelism, append `--tensor-parallel-size N` to the serving command.
|
| 33 |
|
| 34 |
-
Once started, the server listens at `localhost:18888` for requests and is compatible with the [OpenAI ChatCompletion API specifications](https://platform.openai.com/docs/api-reference/chat). Please refer to the example request below for reference. Additionally, you can use the [OpenChat Web UI](#web-ui) for a user-friendly experience.
|
| 35 |
|
| 36 |
If you want to deploy the server as an online service, you can use `--api-keys sk-KEY1 sk-KEY2 ...` to specify allowed API keys and `--disable-log-requests --disable-log-stats --log-file openchat.log` for logging only to a file. For security purposes, we recommend using an [HTTPS gateway](https://fastapi.tiangolo.com/es/deployment/concepts/#security-https) in front of the server.
|
| 37 |
|
|
|
|
| 9 |
</div>
|
| 10 |
|
| 11 |
<p align="center">
|
| 12 |
+
<a href="https://github.com/imoneoi/openchat">GitHub</a> •
|
| 13 |
<a href="https://openchat.team">Online Demo</a> •
|
| 14 |
<a href="https://discord.gg/pQjnXvNKHY">Discord</a> •
|
| 15 |
<a href="https://huggingface.co/openchat">Huggingface</a> •
|
|
|
|
| 30 |
|
| 31 |
## Usage
|
| 32 |
|
| 33 |
+
To use this model, we highly recommend installing the OpenChat package by following the [installation guide](https://github.com/imoneoi/openchat#installation) in our repository and using the OpenChat OpenAI-compatible API server by running the serving command from the table below. The server is optimized for high-throughput deployment using [vLLM](https://github.com/vllm-project/vllm) and can run on a consumer GPU with 24GB RAM. To enable tensor parallelism, append `--tensor-parallel-size N` to the serving command.
|
| 34 |
|
| 35 |
+
Once started, the server listens at `localhost:18888` for requests and is compatible with the [OpenAI ChatCompletion API specifications](https://platform.openai.com/docs/api-reference/chat). Please refer to the example request below for reference. Additionally, you can use the [OpenChat Web UI](https://github.com/imoneoi/openchat#web-ui) for a user-friendly experience.
|
| 36 |
|
| 37 |
If you want to deploy the server as an online service, you can use `--api-keys sk-KEY1 sk-KEY2 ...` to specify allowed API keys and `--disable-log-requests --disable-log-stats --log-file openchat.log` for logging only to a file. For security purposes, we recommend using an [HTTPS gateway](https://fastapi.tiangolo.com/es/deployment/concepts/#security-https) in front of the server.
|
| 38 |
|