Train visual math reasoning with GRPO
A TRL example for improving image-based math answers with automatically checked rewards.
Published by Hugging Face TRL
How to use it
The script reads the multimodal Open-R1 8K verified math dataset and includes launch examples for Qwen2.5-VL-3B-Instruct and SmolVLM2-2.2B-Instruct.
Follow the published Accelerate command and dependency instructions. Check whether the reward function recognizes valid equivalent answers before interpreting improvement as better reasoning.
- Framework
- Hugging Face TRL
- Dataset
- lmms-lab/multimodal-open-r1-8k-verified
- Hardware
- Minimum VRAM unspecified.
- Access
- Public publisher script and launch examples.