🏆 Nacrith: a 135M model that out-compresses everything on natural language
text
🏆 Nacrith: a 135M model that out-compresses everything on natural language
new_line
new_line
What if a tiny LM could compress english text better than _every_ compressor out there — classical or neural, small or large?
text
What if a tiny LM could compress english text better than _every_ compressor out there — classical or neural, small or large?
new_line
new_line
Nacrith pairs SmolLM2-135M with an ensemble of online predictors and high-precision arithmetic coding. 
text
Nacrith pairs SmolLM2-135M with an ensemble of online predictors and high-precision arithmetic coding. 
new_line
new_line
What's inside
text
What's inside
new_line
new_line
The standard LLM+arithmetic coding approach wastes ~75% of CDF precision on large vocabularies. Our CDF-24 fix alone recovers 0.5 bpb. On top: a token N-gram that skips the GPU on predictable tokens, an adaptive bias head, llama.cpp backend (7× faster than PyTorch), multi-GPU parallel compression, and a binary file format (NC06) — the first LLM-based binary compressor we know of.
text
The standard LLM+arithmetic coding approach wastes ~75% of CDF precision on large vocabularies. Our CDF-24 fix alone recovers 0.5 bpb. On top: a token N-gram that skips the GPU on predictable tokens, an adaptive bias head, llama.cpp backend (7× faster than PyTorch), multi-GPU parallel compression, and a binary file format (NC06) — the first LLM-based binary compressor we know of.
new_line
new_line
Runs on a GTX 1050 Ti. ~500 MB weights, ~1.2 GB VRAM per worker.
text
Runs on a GTX 1050 Ti. ~500 MB weights, ~1.2 GB VRAM per worker.
new_line
new_line
💻 Code: 
text
💻 Code: 
https://github.com/robtacconelli/Nacrith-GPU
https://github.com/robtacconelli/Nacrith-GPU
link
new_line
⭐ Space: 
text
⭐ Space: 
https://huggingface.co/spaces/robtacconelli/Nacrith-GPU
robtacconelli/Nacrith-GPU
space
resource
https://huggingface.co/spaces/robtacconelli/Nacrith-GPU
new_line
📄 Paper: 
text
📄 Paper: 
Nacrith: Neural Lossless Compression via Ensemble Context Modeling and High-Precision CDF Coding (2602.19626)
https://huggingface.co/papers/2602.19626
2602.19626
paper
resource
https://huggingface.co/papers/2602.19626
new_line
new_line
Try it, break it, share your results — all feedback welcome. ⭐ on the repo appreciated!
text
Try it, break it, share your results — all feedback welcome. ⭐ on the repo appreciated!
new_line
new_line
Results across all systems we tested:
text
Results across all systems we tested:
new_line
-   alice29.txt → 0.918 bpb (−44% vs CMIX, −20% vs ts_zip) — below the 2nd-order Shannon entropy bound
text
-   alice29.txt → 0.918 bpb (−44% vs CMIX, −20% vs ts_zip) — below the 2nd-order Shannon entropy bound
new_line
-   enwik8 (100 MB) → 0.9389 bpb (−8% vs FineZip/LLMZip's 8B model, −15% vs ts_zip)
text
-   enwik8 (100 MB) → 0.9389 bpb (−8% vs FineZip/LLMZip's 8B model, −15% vs ts_zip)
new_line
-   Unseen text → 0.723 bpb on a doc published after training cutoff — no memorization, 26% better than FineZip/LLMZip on the same model
text
-   Unseen text → 0.723 bpb on a doc published after training cutoff — no memorization, 26% better than FineZip/LLMZip on the same model
new_line
new_line
SmolLM2-135M by 
text
SmolLM2-135M by 
https://cdn-avatars.huggingface.co/v1/production/uploads/651e96991b97c9f33d26bde6/e4VK7uW5sTeCYupD0s_ob.png
https://huggingface.co/HuggingFaceTB
HuggingFaceTB
org
resource
https://huggingface.co/HuggingFaceTB
🏆 Nacrith: a 135M model that out-compresses everything on natural language

What if a tiny LM could compress english text better than _every_ compressor out there — classical or neural, small or large?

Nacrith pairs SmolLM2-135M with an ensemble of online predictors and high-precision arithmetic coding. 

What's inside

The standard LLM+arithmetic coding approach wastes ~75% of CDF precision on large vocabularies. Our CDF-24 fix alone recovers 0.5 bpb. On top: a token N-gram that skips the GPU on predictable tokens, an adaptive bias head, llama.cpp backend (7× faster than PyTorch), multi-GPU parallel compression, and a binary file format (NC06) — the first LLM-based binary compressor we know of.

Runs on a GTX 1050 Ti. ~500 MB weights, ~1.2 GB VRAM per worker.

💻 Code: https://github.com/robtacconelli/Nacrith-GPU
⭐ Space: https://huggingface.co/spaces/robtacconelli/Nacrith-GPU
📄 Paper: https://huggingface.co/papers/2602.19626

Try it, break it, share your results — all feedback welcome. ⭐ on the repo appreciated!

Results across all systems we tested:
-   alice29.txt → 0.918 bpb (−44% vs CMIX, −20% vs ts_zip) — below the 2nd-order Shannon entropy bound
-   enwik8 (100 MB) → 0.9389 bpb (−8% vs FineZip/LLMZip's 8B model, −15% vs ts_zip)
-   Unseen text → 0.723 bpb on a doc published after training cutoff — no memorization, 26% better than FineZip/LLMZip on the same model

SmolLM2-135M by https://huggingface.co/HuggingFaceTB
🚀
giux78 robtacconelli John6666 Inflammable1230 umeshkumarasamy KrisKale45 kaupane unalkalkan imnotkitty alspinu alexsobolev OzTianlu
🔥
cnmoro Inflammable1230 MAHDI1235 uzvisa robtacconelli unalkalkan alspinu alexsobolev
6797938a9c245187cc9efc30
https://cdn-avatars.huggingface.co/v1/production/uploads/6797938a9c245187cc9efc30/njq11GKUf8ub1EmkUa3zW.jpeg
Zixi "Oz" Li
OzTianlu
user