GroveMoE-Base / README.md
HaoxingChen's picture
Update README.md
9f063b3 verified
metadata
library_name: transformers
license: apache-2.0
pipeline_tag: text-generation

GroveMoE-Base

🤗 Models   |    📑 Paper    |    🔗 Github  

Highlights

We introduce GroveMoE, a new sparse architecture using adjugate experts for dynamic computation allocation, featuring the following key highlights:

  • Architecture: Novel adjugate experts grouped with ordinary experts; shared computation is executed once, then reused, cutting FLOPs.
  • Sparse Activation: 33 B params total, only 3.14–3.28 B active per token.
  • Traning: Mid-training + SFT, up-cycled from Qwen3-30B-A3B-Base; preserves prior knowledge while adding new capabilities.

Model Downloads

Model #Total Params #Activated Params Download
GroveMoE-Base 33B 3.14~3.28B 🤗 HuggingFace
GroveMoE-Inst 33B 3.14~3.28B 🤗 HuggingFace

Citation

@article{GroveMoE,
title = {GroveMoE: Towards Efficient and Superior MoE LLMs with Adjugate Experts},
author = {Wu, Haoyuan and Chen, Haoxing and Chen, Xiaodong and Zhou, Zhanchao and Chen, Tieyuan and Zhuang, Yihong and Lu, Guoshan and Zhao, Junbo and Liu, Lin and Huang, Zenan and Lan, Zhenzhong and Yu, Bei and Li, Jianguo},
journal = {arXiv preprint arXiv:2508.07785},
year = {2025}
}