Memory-based Language Models: An Efficient, Explainable, and
  Eco-friendly Approach to Large Language Modeling
We present memory-based language modeling as an efficient, eco-friendly
alternative to deep neural network-based language modeling. It offers
log-linearly scalable next-token prediction performance and strong memorization
capabilities. Implementing fast approximations of k-nearest neighbor
classification, memory-based language modeling leaves a relatively small
ecological footprint both in training and in inference mode, as it relies fully
on CPUs and attains low token latencies. Its internal workings are simple and
fully transparent. We compare our implementation of memory-based language
modeling, OLIFANT, with GPT-2 and GPT-Neo on next-token prediction accuracy,
estimated emissions and speeds, and offer some deeper analyses of the model.
69006c0222d452aac6dd4475
Antal van den Bosch
claimed_verified
2025-11-14T14:22:47.795Z
634026af07e80abe233ad75f
https://cdn-avatars.huggingface.co/v1/production/uploads/634026af07e80abe233ad75f/1W0i48gT0HgHfOueJtmi5.jpeg
Antal
antalvdb
user
antalvdb
69006c0222d452aac6dd4476
Ainhoa Risco Patón
69006c0222d452aac6dd4477
Teun Buijse
69006c0222d452aac6dd4478
Peter Berck
69006c0222d452aac6dd4479
Maarten van Gompel
Memory-based Language Models: An Efficient, Explainable, and
  Eco-friendly Approach to Large Language Modeling
We present memory-based language modeling as an efficient, eco-friendly
alternative to deep neural network-based language modeling. It offers
log-linearly scalable next-token prediction performance and strong memorization
capabilities. Implementing fast approximations of k-nearest neighbor
classification, memory-based language modeling leaves a relatively small
ecological footprint both in training and in inference mode, as it relies fully
on CPUs and attains low token latencies. Its internal workings are simple and
fully transparent. We compare our implementation of memory-based language
modeling, OLIFANT, with GPT-2 and GPT-Neo on next-token prediction accuracy,
estimated emissions and speeds, and offer some deeper analyses of the model.
Memory-based language modeling provides an efficient, eco-friendly alternative to deep neural networks, offering scalable performance and strong memorization with low ecological impact.
k-nearest neighbor classification memory-based language modeling next-token prediction ecological footprint CPUs token latencies OLIFANT GPT-2 GPT-Neo