Apertus LLM Family Expansion via Distillation and Quantization
Research paper by Andrei Panferov and collaborators. arXiv record dated 2026-06-10. Research area: Machine learning.
Resource information
- Authors
- Andrei Panferov, Davit Melikidze, Martin Jaggi, Dan Alistarh
- Research area
- Machine learning
- Application domain
- General AI research
- AI methods
- Large language models
- Research topics
- cs.LG
- arXiv record date
- 2026-06-10
- Publication
- arXiv
- Upstream last modified
- Jun 11, 2026
- Last record update
- Nov 4, 2026
Abstract
The wide adoption of LLMs has led to their use in great variety of applications and scenarios, such as chatbot assistants and data annotation, creating the need for the models to satisfy certain budget and hardware constraints. This has led to the trend of LLMs being released in batches consisting of similar models of various sizes for the family of models to adhere to as wide of a range of constraints as possible. In this paper, we validate distillation and quantization as a cost-effective way to expand model families to new sizes and hardware formats. Based on the open-recipe Apertus 8B LLM, we produce Apertus-v1.1 - a distilled family of models with up to 4B parameters trained on 1.7T permissive license tokens. We demonstrate cost-efficiency and strong accuracy performance of our approach for covering large ranges of hardware and systems requirements.