5,538,209 AI resources indexedWorking toward the world’s largest AI resource index.

PDF

Falcon Mamba: The First Competitive Attention-free 7B Language Model

Research paper by Jingwei Zuo and collaborators. Published 2024-10-07. Topics: Mamba architecture, large language model, Transformers, Mistral 7B, Llama3.1 8B.

Resource information

Authors
Jingwei Zuo, Maksim Velikanov, Dhia Eddine Rhaiem, Ilyas Chahed, Younes Belkada, Guillaume Kunsch, Hakim Hacid
Research topics
Mamba architecture, large language model, Transformers, Mistral 7B, Llama3.1 8B, Falcon2 11B, Gemma 7B, RecurrentGemma 9B, RWKV-v6 Finch 7B/14B, Open LLM Leaderboard
Publication date
2024-10-07
Publication
arXiv
Last record update
Nov 3, 2026