Distill a chat model with TRL's GOLD trainer
An on-policy distillation example that trains a smaller chat model using a separate teacher.
Published by Hugging Face TRL
How to use it
The source shows a Qwen3-4B teacher with smaller student configurations. It uses the TRL GOLD trainer and a chat dataset selected from the NVIDIA Nemotron specialized collection.
Use the example commands to set the teacher, student, and data configuration. Compare the trained student and its baseline on held-out tasks before drawing conclusions from the distillation loss.
- Framework
- Hugging Face TRL
- Dataset
- nvidia/Nemotron-Pretraining-Specialized-v1.1
- Hardware
- Minimum VRAM unspecified.
- Access
- Public publisher script and launch examples.