Experimental 
text
Experimental 
https://huggingface.co/eaddario/DeepSeek-R1-Distill-Llama-8B-GGUF
eaddario/DeepSeek-R1-Distill-Llama-8B-GGUF
model
resource
https://huggingface.co/eaddario/DeepSeek-R1-Distill-Llama-8B-GGUF
 is now available.
text
 is now available.
new_line
new_line
Got close to achieve a 10% reduction in size but the quality started to deteriorate so this version has been pruned more conservatively. Sizes, on average, are about 8% smaller with only a very small penalty (< 1%) in quality.
text
Got close to achieve a 10% reduction in size but the quality started to deteriorate so this version has been pruned more conservatively. Sizes, on average, are about 8% smaller with only a very small penalty (< 1%) in quality.
new_line
new_line
After trying with different models and parameter count, I suspect the best I'll be able to do with the current process is  between 6 and 8% reduction so have decided to try a different approach. I'll publish process and findings next.
text
After trying with different models and parameter count, I suspect the best I'll be able to do with the current process is  between 6 and 8% reduction so have decided to try a different approach. I'll publish process and findings next.
new_line
new_line
For background: 
text
For background: 
https://huggingface.co/posts/eaddario/832567461491467
https://huggingface.co/posts/eaddario/832567461491467
link
Experimental https://huggingface.co/eaddario/DeepSeek-R1-Distill-Llama-8B-GGUF is now available.

Got close to achieve a 10% reduction in size but the quality started to deteriorate so this version has been pruned more conservatively. Sizes, on average, are about 8% smaller with only a very small penalty (< 1%) in quality.

After trying with different models and parameter count, I suspect the best I'll be able to do with the current process is  between 6 and 8% reduction so have decided to try a different approach. I'll publish process and findings next.

For background: https://huggingface.co/posts/eaddario/832567461491467
👍
John6666 JohnRoger dantezxcd