There are two very interesting reasoning models from 
text
There are two very interesting reasoning models from 
https://cdn-avatars.huggingface.co/v1/production/uploads/63d3095c2727d7888cbb54e2/Uv-Lx8PVGviqokfOyYlCN.png
https://huggingface.co/ServiceNow-AI
ServiceNow-AI
org
resource
https://huggingface.co/ServiceNow-AI
 that I think are flying under everyone's radar - lets take a closer look at 
text
 that I think are flying under everyone's radar - lets take a closer look at 
https://huggingface.co/ServiceNow-AI/Apriel-1.5-15b-Thinker
ServiceNow-AI/Apriel-1.5-15b-Thinker
model
resource
https://huggingface.co/ServiceNow-AI/Apriel-1.5-15b-Thinker
 (#10 on the ReasonScape rankings) and 
text
 (#10 on the ReasonScape rankings) and 
https://huggingface.co/ServiceNow-AI/Apriel-Nemotron-15b-Thinker
ServiceNow-AI/Apriel-Nemotron-15b-Thinker
model
resource
https://huggingface.co/ServiceNow-AI/Apriel-Nemotron-15b-Thinker
 (landing just below its brother at #12).
text
 (landing just below its brother at #12).
new_line
new_line
A rather interesting attribute of these models is I have absolutely no idea what they are fine-tuned from, other then some kind of pre-small Mistrals!  The non-nemo 15b looks like Mistral Pixtral 12B, but with 8 more layers while the nemo 15b analogously looks like Mistral NeMo 12B but with 10 more layers and a smaller max context length.
text
A rather interesting attribute of these models is I have absolutely no idea what they are fine-tuned from, other then some kind of pre-small Mistrals!  The non-nemo 15b looks like Mistral Pixtral 12B, but with 8 more layers while the nemo 15b analogously looks like Mistral NeMo 12B but with 10 more layers and a smaller max context length.
new_line
new_line
The performance trade-offs between these two models are quite clear: the Nemotron provides ~30% shorter answers but at the expense of totally collapsing under difficulty on 4 of the 12 tasks ... which all just happen to have "Math" in common, so it's pretty easy to point the finger at exactly what the price for the lower reasoning token usage is here.
text
The performance trade-offs between these two models are quite clear: the Nemotron provides ~30% shorter answers but at the expense of totally collapsing under difficulty on 4 of the 12 tasks ... which all just happen to have "Math" in common, so it's pretty easy to point the finger at exactly what the price for the lower reasoning token usage is here.
new_line
new_line
In principle 
text
In principle 
https://huggingface.co/ServiceNow-AI/Apriel-1.5-15b-Thinker
ServiceNow-AI/Apriel-1.5-15b-Thinker
model
resource
https://huggingface.co/ServiceNow-AI/Apriel-1.5-15b-Thinker
 is multimodal and should be able to reason about image queries but this is not something I have tried as ReasonScape is not currently able to evaluate VLMs - perhaps a future improvement.
text
 is multimodal and should be able to reason about image queries but this is not something I have tried as ReasonScape is not currently able to evaluate VLMs - perhaps a future improvement.
There are two very interesting reasoning models from https://huggingface.co/ServiceNow-AI that I think are flying under everyone's radar - lets take a closer look at https://huggingface.co/ServiceNow-AI/Apriel-1.5-15b-Thinker (#10 on the ReasonScape rankings) and https://huggingface.co/ServiceNow-AI/Apriel-Nemotron-15b-Thinker (landing just below its brother at #12).

A rather interesting attribute of these models is I have absolutely no idea what they are fine-tuned from, other then some kind of pre-small Mistrals!  The non-nemo 15b looks like Mistral Pixtral 12B, but with 8 more layers while the nemo 15b analogously looks like Mistral NeMo 12B but with 10 more layers and a smaller max context length.

The performance trade-offs between these two models are quite clear: the Nemotron provides ~30% shorter answers but at the expense of totally collapsing under difficulty on 4 of the 12 tasks ... which all just happen to have "Math" in common, so it's pretty easy to point the finger at exactly what the price for the lower reasoning token usage is here.

In principle https://huggingface.co/ServiceNow-AI/Apriel-1.5-15b-Thinker is multimodal and should be able to reason about image queries but this is not something I have tried as ReasonScape is not currently able to evaluate VLMs - perhaps a future improvement.
🔥
unmodeled-tyler John6666 aquiffoo