Falcon Mamba: The First Competitive Attention-free 7B Language Model
Research paper by Jingwei Zuo and collaborators. Published 2024-10-07. Topics: Mamba architecture, large language model, Transformers, Mistral 7B, Llama3.1 8B.
Resource information
- Authors
- Jingwei Zuo, Maksim Velikanov, Dhia Eddine Rhaiem, Ilyas Chahed, Younes Belkada, Guillaume Kunsch, Hakim Hacid
- Research topics
- Mamba architecture, large language model, Transformers, Mistral 7B, Llama3.1 8B, Falcon2 11B, Gemma 7B, RecurrentGemma 9B, RWKV-v6 Finch 7B/14B, Open LLM Leaderboard
- Publication date
- 2024-10-07
- Publication
- arXiv
- Last record update
- Nov 3, 2026
