What an eventful day in Open Source LLMs today:
text
What an eventful day in Open Source LLMs today:
new_line
new_line
Mistral released Codestral Mamba 🐍
text
Mistral released Codestral Mamba 🐍
new_line
> Beats DeepSeek QwenCode, best model < 10B, competitive with Codestral 22B
text
> Beats DeepSeek QwenCode, best model < 10B, competitive with Codestral 22B
new_line
> Mamba 2 architecture - supports up to 256K context
text
> Mamba 2 architecture - supports up to 256K context
new_line
> Apache 2.0 licensed, perfect for local code assistant
text
> Apache 2.0 licensed, perfect for local code assistant
new_line
> Transformers & llama.cpp integration upcoming!
text
> Transformers & llama.cpp integration upcoming!
new_line
new_line
Model checkpoint: 
text
Model checkpoint: 
https://huggingface.co/mistralai/mamba-codestral-7B-v0.1
https://huggingface.co/mistralai/mamba-codestral-7B-v0.1
link
new_line
new_line
Hugging Face dropped SmolLM 🤏
text
Hugging Face dropped SmolLM 🤏
new_line
> Beats MobileLLM, Qwen 0.5B, Phi 1.5B and more!
text
> Beats MobileLLM, Qwen 0.5B, Phi 1.5B and more!
new_line
> 135M, 360M, and 1.7B param model checkpoints
text
> 135M, 360M, and 1.7B param model checkpoints
new_line
> Trained on 600B high-quality synthetic + FineWeb Edu tokens
text
> Trained on 600B high-quality synthetic + FineWeb Edu tokens
new_line
> Architecture: Llama + GQA + 2048 ctx length 
text
> Architecture: Llama + GQA + 2048 ctx length 
new_line
> Ripe for fine-tuning and on-device deployments.
text
> Ripe for fine-tuning and on-device deployments.
new_line
> Works out of the box with Transformers!
text
> Works out of the box with Transformers!
new_line
new_line
Model checkpoints: 
text
Model checkpoints: 
https://huggingface.co/collections/HuggingFaceTB/smollm-6695016cad7167254ce15966
HuggingFaceTB/smollm-6695016cad7167254ce15966
collection
resource
https://huggingface.co/collections/HuggingFaceTB/smollm-6695016cad7167254ce15966
new_line
new_line
Mistral released Mathstral 7B ∑
text
Mistral released Mathstral 7B ∑
new_line
> 56.6% on MATH and 63.47% on MMLU
text
> 56.6% on MATH and 63.47% on MMLU
new_line
> Same architecture as Mistral 7B
text
> Same architecture as Mistral 7B
new_line
> Works out of the box with Transformers & llama.cpp 
text
> Works out of the box with Transformers & llama.cpp 
new_line
> Released under Apache 2.0 license
text
> Released under Apache 2.0 license
new_line
new_line
Model checkpoint: 
text
Model checkpoint: 
https://huggingface.co/mistralai/mathstral-7B-v0.1
https://huggingface.co/mistralai/mathstral-7B-v0.1
link
new_line
new_line
Pretty dope day for open source ML. Can't wait to see what the community builds with it and to support them further! 🤗
text
Pretty dope day for open source ML. Can't wait to see what the community builds with it and to support them further! 🤗
new_line
new_line
What's your favourite from the release today?
text
What's your favourite from the release today?
What an eventful day in Open Source LLMs today:

Mistral released Codestral Mamba 🐍
> Beats DeepSeek QwenCode, best model < 10B, competitive with Codestral 22B
> Mamba 2 architecture - supports up to 256K context
> Apache 2.0 licensed, perfect for local code assistant
> Transformers & llama.cpp integration upcoming!

Model checkpoint: https://huggingface.co/mistralai/mamba-codestral-7B-v0.1

Hugging Face dropped SmolLM 🤏
> Beats MobileLLM, Qwen 0.5B, Phi 1.5B and more!
> 135M, 360M, and 1.7B param model checkpoints
> Trained on 600B high-quality synthetic + FineWeb Edu tokens
> Architecture: Llama + GQA + 2048 ctx length 
> Ripe for fine-tuning and on-device deployments.
> Works out of the box with Transformers!

Model checkpoints: https://huggingface.co/collections/HuggingFaceTB/smollm-6695016cad7167254ce15966

Mistral released Mathstral 7B ∑
> 56.6% on MATH and 63.47% on MMLU
> Same architecture as Mistral 7B
> Works out of the box with Transformers & llama.cpp 
> Released under Apache 2.0 license

Model checkpoint: https://huggingface.co/mistralai/mathstral-7B-v0.1

Pretty dope day for open source ML. Can't wait to see what the community builds with it and to support them further! 🤗

What's your favourite from the release today?
🔥
reach-vb Presidentlin osanseviero dvilasuero louisbrulenaudet nawed ajibawa-2023 Alexandro14 tharrmeehan Mdubbya lucianosb WOOSAH nbroad
🤝
reach-vb dvilasuero louisbrulenaudet
🚀
reach-vb dvilasuero jeffboudier
👍
dashfunnydashdash
64aea8ff67511bd3d965697b
https://cdn-avatars.huggingface.co/v1/production/uploads/64aea8ff67511bd3d965697b/Jxn52EmDF5RApJh8antxn.jpeg
Feynman Innovations
ajibawa-2023
user