JambaTalk: Speech-Driven 3D Talking Head Generation Based on Hybrid Transformer-Mamba Model
Research paper by Farzaneh Jafari and collaborators. arXiv record dated 2025-12-06. Research area: Computer vision.
Resource information
- Authors
- Farzaneh Jafari, Stefano Berretti, Anup Basu
- Research area
- Computer vision
- Application domain
- General AI research
- AI methods
- Transformers
- Research topics
- cs.CV
- arXiv record date
- 2025-12-06
- Publication
- arXiv
- Upstream last modified
- Dec 9, 2025
- Last record update
- Nov 4, 2026
Abstract
In recent years, the talking head generation has become a focal point for researchers. Considerable effort is being made to refine lip-sync motion, capture expressive facial expressions, generate natural head poses, and achieve high-quality video. However, no single model has yet achieved equivalence across all quantitative and qualitative metrics. We introduce Jamba, a hybrid Transformer-Mamba model, to animate a 3D face. Mamba, a pioneering Structured State Space Model (SSM) architecture, was developed to overcome the limitations of conventional Transformer architectures, particularly in handling long sequences. This challenge has constrained traditional models. Jamba combines the advantages of both the Transformer and Mamba approaches, offering a comprehensive solution. Based on the foundational Jamba block, we present JambaTalk to enhance motion variety and lip sync through multimodal integration. Extensive experiments reveal that our method achieves performance comparable or superior to state-of-the-art models.