Teach Qwen3 to play Sudoku with GRPO and OpenEnv
A reinforcement-learning recipe for training a model to make valid moves over multiple interactions with Sudoku.
Published by Hugging Face TRL
How to use it
The script starts from a Qwen3-1.7B model already warmed up on Sudoku games. It connects to an OpenEnv Sudoku environment and scores the games produced by tool-calling rollouts.
Set up the environment endpoint and choose the documented colocated or separate vLLM option. Compare the starting and trained models on the same held-out puzzles and record both completion rates and invalid moves.
- Framework
- Hugging Face TRL
- Dataset
- Interactive Sudoku games from the OpenEnv TextArena environment.
- Hardware
- One GPU with colocated vLLM or two GPUs for the separate-server example. Publisher tested on one A100 80 GB.
- Access
- Public publisher script and notebook; a configured OpenEnv endpoint is required.