SmolTulu: Higher Learning Rate to Batch Size Ratios Can Lead to Better Reasoning in SLMs
Research paper by Sultan Alrashed. Published 2024-12-11. Topics: instruction-tuned, SmolTulu-DPO-1130, SmolLM2-1.7B, learning rate, batch size.
Resource information
- Authors
- Sultan Alrashed
- Research topics
- instruction-tuned, SmolTulu-DPO-1130, SmolLM2-1.7B, learning rate, batch size, ARC, GSM8K, HellaSwag, IFEval, performance, efficient model alignment
- Publication date
- 2024-12-11
- Publication
- arXiv
- Last record update
- Nov 3, 2026