llama.cpp is 26.8% faster than ollama. 
text
llama.cpp is 26.8% faster than ollama. 
new_line
I have upgraded both, and using the same settings, I am running the same DeepSeek R1 Distill 1.5B on the same hardware. It's an Apples to Apples comparison. 
text
I have upgraded both, and using the same settings, I am running the same DeepSeek R1 Distill 1.5B on the same hardware. It's an Apples to Apples comparison. 
new_line
new_line
Total duration: 
text
Total duration: 
new_line
llama.cpp 6.85 sec <- 26.8% faster
text
llama.cpp 6.85 sec <- 26.8% faster
new_line
ollama 8.69 sec
text
ollama 8.69 sec
new_line
new_line
Breakdown by phase:
text
Breakdown by phase:
new_line
Model loading
text
Model loading
new_line
llama.cpp 241 ms <- 2x faster
text
llama.cpp 241 ms <- 2x faster
new_line
ollama 553 ms
text
ollama 553 ms
new_line
new_line
Prompt processing
text
Prompt processing
new_line
llama.cpp 416.04 tokens/s with an eval time 45.67 ms <- 10x faster
text
llama.cpp 416.04 tokens/s with an eval time 45.67 ms <- 10x faster
new_line
ollama 42.17 tokens/s with an eval time of 498 ms
text
ollama 42.17 tokens/s with an eval time of 498 ms
new_line
new_line
Token generation
text
Token generation
new_line
llama.cpp 137.79 tokens/s with an eval time 6.62 sec <- 13% faster
text
llama.cpp 137.79 tokens/s with an eval time 6.62 sec <- 13% faster
new_line
ollama 122.07 tokens/s with an eval time 7.64 sec
text
ollama 122.07 tokens/s with an eval time 7.64 sec
new_line
new_line
llama.cpp is LLM inference in C/C++; ollama adds abstraction layers and marketing. 
text
llama.cpp is LLM inference in C/C++; ollama adds abstraction layers and marketing. 
new_line
new_line
Make sure you own your AI. AI in the cloud is not aligned with you; it's aligned with the company that owns it.
text
Make sure you own your AI. AI in the cloud is not aligned with you; it's aligned with the company that owns it.
llama.cpp is 26.8% faster than ollama. 
I have upgraded both, and using the same settings, I am running the same DeepSeek R1 Distill 1.5B on the same hardware. It's an Apples to Apples comparison. 

Total duration: 
llama.cpp 6.85 sec <- 26.8% faster
ollama 8.69 sec

Breakdown by phase:
Model loading
llama.cpp 241 ms <- 2x faster
ollama 553 ms

Prompt processing
llama.cpp 416.04 tokens/s with an eval time 45.67 ms <- 10x faster
ollama 42.17 tokens/s with an eval time of 498 ms

Token generation
llama.cpp 137.79 tokens/s with an eval time 6.62 sec <- 13% faster
ollama 122.07 tokens/s with an eval time 7.64 sec

llama.cpp is LLM inference in C/C++; ollama adds abstraction layers and marketing. 

Make sure you own your AI. AI in the cloud is not aligned with you; it's aligned with the company that owns it.
👍
ali6parmak benhaotang klevr Rsln abolfazlsj ngxson akahana yasserrmd googlefan t1u1 clem thomwolf nirmitk pulkitmehtawork BjornMelin samcognymai
🚀
John6666 pabloce ngxson ArthurZ clem philipp-zettl thomwolf
64429aaf7feb866811b12f73
https://cdn-avatars.huggingface.co/v1/production/uploads/64429aaf7feb866811b12f73/LxUVODgksCAlncbvbRGHu.png
Benjamin Paine
benjamin-paine
user
66c26b6fb01b19d8c3c2467b
https://cdn-avatars.huggingface.co/v1/production/uploads/66c26b6fb01b19d8c3c2467b/yAYym6-t_Q7iCSBFgoUoo.jpeg
DV
Delta-Vector
user
630f3e4002ce39336c411048
https://cdn-avatars.huggingface.co/v1/production/uploads/630f3e4002ce39336c411048/Dbb855C55XdPd3rRH9auc.jpeg
alkinun
AtAndDev
user
6506f9cfd55dd4e15cb31988
https://cdn-avatars.huggingface.co/v1/production/uploads/6506f9cfd55dd4e15cb31988/EIUO_i2jrpdauHTYCE38p.png
Tech Meld
Tech-Meld
user
6640bbd0220cfa8cbfdce080
https://cdn-avatars.huggingface.co/v1/production/uploads/6640bbd0220cfa8cbfdce080/wiAHUu5ewawyipNs0YFBR.png
John Smith
John6666
user