BlackMamba: Mixture of Experts for State-Space Models
Research paper by Quentin Anthony and collaborators. Published 2024-02-01. Topics: state-space models, SSM, Mamba, mixture-of-experts, MoE.
Resource information
- Authors
- Quentin Anthony, Yury Tokpanov, Paolo Glorioso, Beren Millidge
- Research topics
- state-space models, SSM, Mamba, mixture-of-experts, MoE, BlackMamba, linear time and memory complexity, language modeling, long sequence processing, inference FLOPs, training FLOPs, custom dataset
- Publication date
- 2024-02-01
- Publication
- arXiv
- Last record update
- Nov 3, 2026