Research paper
SmallThinker: A Family of Efficient Large Language Models Natively Trained for Local Deployment
Research paper by Yixin Song and collaborators. Published 2025-07-28. Topics: Mixture-of-Experts (MoE), sparse feed-forward networks, pre-attention router, NoPE-RoPE hybrid sparse attention mechanism, Q4_0 quantization.
Yixin Song, Zhenliang Xue, Dongliang Wei, Feiyang Chen, Jianxiang Gao, Junchen Liu, Hangyu Liang, Guangshuo Qin, Chengrong Tian, Bo Wen, Longyu Zhao, Xinrui Zheng, Zeyu Mi, Haibo Chen
CC BY 4.0View resource