Train a Qwen3 reward model on preference pairs
A runnable example that trains a scalar reward scorer using the Skywork Qwen3 checkpoint.
Published by Axolotl
How to use it
The configuration loads Skywork-Reward-V2-Qwen3-8B as a sequence classifier with one score. It trains on the Argilla Intel Orca preference pairs and includes a reward-centering coefficient.
Evaluate on held-out comparisons that reflect the intended application. Useful checks include pairwise accuracy and whether scores change for superficial differences such as answer length.
- Framework
- Axolotl
- Dataset
- argilla/distilabel-intel-orca-dpo-pairs
- Hardware
- Minimum VRAM unspecified.
- Access
- Public publisher configuration; model and dataset terms apply.