HaoxingChen commited on
Commit
99de339
·
verified ·
1 Parent(s): c4c69e5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +49 -1
README.md CHANGED
@@ -39,7 +39,9 @@ We introduce **GroveMoE**, a new sparse architecture using **adjugate experts**
39
 
40
  We bold the top1 scores separately for all models. More details are reported in our [technical report](https://arxiv.org/abs/2508.07785).
41
 
42
- ## Usage
 
 
43
  Below, there are some code snippets on how to get quickly started with running the model. First, install the Transformers library.
44
 
45
  ```sh
@@ -79,6 +81,52 @@ content = tokenizer.decode(output_ids, skip_special_tokens=True)
79
  print("content:", content)
80
  ```
81
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
82
  ## Citation
83
  ```bibtex
84
  @article{GroveMoE,
 
39
 
40
  We bold the top1 scores separately for all models. More details are reported in our [technical report](https://arxiv.org/abs/2508.07785).
41
 
42
+ ## Run GroveMoE
43
+
44
+ ### 🤗 Transformers Quick Start
45
  Below, there are some code snippets on how to get quickly started with running the model. First, install the Transformers library.
46
 
47
  ```sh
 
81
  print("content:", content)
82
  ```
83
 
84
+ ### 🚀 SGLang Quick Start
85
+ For SGLang, you can follow the steps below to deploy:
86
+
87
+ 1️⃣ Install Dependencies
88
+
89
+ First, clone the repository:
90
+ ```shell
91
+ git clone https://github.com/inclusionAI/GroveMoE.git
92
+ ```
93
+ Then, install Transformers:
94
+ ```shell
95
+ cd src/transformers-4.51.3
96
+ pip install .
97
+ ```
98
+ Next, install SGLang:
99
+ ```shell
100
+ cd src/sglang-0.4.6.post5
101
+ pip install .
102
+
103
+ ```
104
+
105
+ 2️⃣ Launch the Server
106
+
107
+ Run the following command to start SGLang:
108
+ ```shell
109
+ python -m sglang.launch_server \
110
+ --model-path inclusionAI/GroveMoE-Inst \
111
+ --port 30000 \
112
+ --context-length 32768
113
+ ```
114
+
115
+ 3️⃣ Access the API
116
+
117
+ Once started, the OpenAI-compatible API will be available at `http://localhost:30000/v1`.
118
+
119
+ Test it with curl:
120
+ ```shell
121
+ curl http://localhost:30000/v1/chat/completions \
122
+ -H "Content-Type: application/json" \
123
+ -d '{
124
+ "model": "inclusionAI/GroveMoE-Inst",
125
+ "messages": [{"role": "user", "content": "Hello, SGLang!"}]
126
+ }'
127
+
128
+ ```
129
+
130
  ## Citation
131
  ```bibtex
132
  @article{GroveMoE,