Memory-based Language Models: An Efficient, Explainable, and Eco-friendly Approach to Large Language Modeling
Research paper by Antal van den Bosch and collaborators. Published 2025-10-25. Topics: k-nearest neighbor classification, memory-based language modeling, next-token prediction, ecological footprint, CPUs.
Resource information
- Authors
- Antal van den Bosch, Ainhoa Risco Patón, Teun Buijse, Peter Berck, Maarten van Gompel
- Research area
- Language technology
- Application domain
- General AI research
- AI methods
- Large language models, Neural networks
- Research topics
- cs.CL
- arXiv record date
- 2025-10-25
- Publication date
- 2025-10-25
- Publication
- arXiv
- Upstream last modified
- Oct 28, 2025
- Last record update
- Nov 3, 2026
Abstract
We present memory-based language modeling as an efficient, eco-friendly alternative to deep neural network-based language modeling. It offers log-linearly scalable next-token prediction performance and strong memorization capabilities. Implementing fast approximations of k-nearest neighbor classification, memory-based language modeling leaves a relatively small ecological footprint both in training and in inference mode, as it relies fully on CPUs and attains low token latencies. Its internal workings are simple and fully transparent. We compare our implementation of memory-based language modeling, OLIFANT, with GPT-2 and GPT-Neo on next-token prediction accuracy, estimated emissions and speeds, and offer some deeper analyses of the model.