Cambrian-P: Pose-Grounded Video Understanding
Research paper by Jihan Yang and collaborators. arXiv record dated 2026-07-23. Research area: Computer vision.
Resource information
- Authors
- Jihan Yang, Zifan Zhao, Xichen Pan, Shusheng Yang, Junyi Zhang, Bingyi Kang, Hu Xu, Shang-Wen Li, Saining Xie
- Research area
- Computer vision
- Application domain
- General AI research
- AI methods
- Large language models
- Research topics
- cs.CV
- arXiv record date
- 2026-07-23
- Publication
- arXiv
- Upstream last modified
- Jul 24, 2026
- License
- CC BY 4.0
- Last record update
- Nov 4, 2026
Abstract
Camera pose matters. The position and orientation of each viewpoint define a shared spatial coordinate frame that relates observations across video frames. Yet this signal is largely absent from multimodal LLMs (MLLMs) for video understanding, which process frames as isolated 2D snapshots, instead of the persistent scene humans perceive. We revisit pose as a lightweight supervisory signal and introduce Cambrian-P, a video MLLM augmented with per-frame learnable camera tokens and a pose regression head. With a carefully designed sampling scheme, the model achieves substantial gains of 4.5-6.5% on spatial reasoning benchmarks such as VSI-Bench, generalizes across eight additional spatial and general video QA benchmarks, and, as a byproduct, achieves state of the art streaming pose estimation on ScanNet. Surprisingly, training on pseudo-annotated poses from in-the-wild video further improves general video QA benchmarks, showing pose helps beyond spatial reasoning. Together, these results position camera pose as a fundamental signal for video models that reason about the physical world.