Seeing Beyond Frames: Zero-Shot Pedestrian Intention Prediction with Raw Temporal Video and Multimodal Cues
Research paper by Pallavi Zambare and collaborators. arXiv record dated 2025-07-25. Research area: Computer vision.
Resource information
- Authors
- Pallavi Zambare, Venkata Nikhil Thanikella, Ying Liu
- Research area
- Computer vision
- Application domain
- Transportation
- AI methods
- Large language models
- Research topics
- cs.CV, cs.AI, cs.LG
- arXiv record date
- 2025-07-25
- Publication
- arXiv
- Upstream last modified
- Jul 30, 2025
- License
- CC BY 4.0
- Last record update
- Nov 3, 2026
Abstract
Pedestrian intention prediction is essential for autonomous driving in complex urban environments. Conventional approaches depend on supervised learning over frame sequences and require extensive retraining to adapt to new scenarios. Here, we introduce BF-PIP (Beyond Frames Pedestrian Intention Prediction), a zero-shot approach built upon Gemini 2.5 Pro. It infers crossing intentions directly from short, continuous video clips enriched with structured JAAD metadata. In contrast to GPT-4V based methods that operate on discrete frames, BF-PIP processes uninterrupted temporal clips. It also incorporates bounding-box annotations and ego-vehicle speed via specialized multimodal prompts. Without any additional training, BF-PIP achieves 73% prediction accuracy, outperforming a GPT-4V baseline by 18 %. These findings illustrate that combining temporal video inputs with contextual cues enhances spatiotemporal perception and improves intent inference under ambiguous conditions. This approach paves the way for agile, retraining-free perception module in intelligent transportation system.