3C3H AraGen Leaderboard welcomes today 
text
3C3H AraGen Leaderboard welcomes today 
https://huggingface.co/deepseek-ai/DeepSeek-V3
deepseek-ai/DeepSeek-V3
model
resource
https://huggingface.co/deepseek-ai/DeepSeek-V3
 and 12 other models (including the late gpt-3.5 💀) to the ranking of best LLMs in Arabic !
text
 and 12 other models (including the late gpt-3.5 💀) to the ranking of best LLMs in Arabic !
new_line
new_line
new_line
Observations:
text
Observations:
new_line
- DeepSeek-v3 ranked 3rd and only Open model among the top 5 !
text
- DeepSeek-v3 ranked 3rd and only Open model among the top 5 !
new_line
new_line
- A 14B open model (
text
- A 14B open model (
https://huggingface.co/Qwen/Qwen2.5-14B-Instruct
Qwen/Qwen2.5-14B-Instruct
model
resource
https://huggingface.co/Qwen/Qwen2.5-14B-Instruct
) outperforms gpt-3.5-turbo-0125 (from last year). This shows how much we came in advancing and supporting Arabic presence within the LLM ecosystem ! 
text
) outperforms gpt-3.5-turbo-0125 (from last year). This shows how much we came in advancing and supporting Arabic presence within the LLM ecosystem ! 
new_line
new_line
- Contrary to what observed in likelihood-acc leaderboards (like 
text
- Contrary to what observed in likelihood-acc leaderboards (like 
https://huggingface.co/spaces/OALL/Open-Arabic-LLM-Leaderboard
OALL/Open-Arabic-LLM-Leaderboard
space
resource
https://huggingface.co/spaces/OALL/Open-Arabic-LLM-Leaderboard
) further finetuned models like 
text
) further finetuned models like 
https://huggingface.co/maldv/Qwentile2.5-32B-Instruct
maldv/Qwentile2.5-32B-Instruct
model
resource
https://huggingface.co/maldv/Qwentile2.5-32B-Instruct
 actually decreased the performance compared to the original model 
text
 actually decreased the performance compared to the original model 
https://huggingface.co/Qwen/Qwen2.5-32B-Instruct
Qwen/Qwen2.5-32B-Instruct
model
resource
https://huggingface.co/Qwen/Qwen2.5-32B-Instruct
.
text
.
new_line
It's worth to note that the decrease is statiscally insignificant which imply that at best, the out-domain finetuning do not really hurts the model original capabilities acquired during pretraining. 
text
It's worth to note that the decrease is statiscally insignificant which imply that at best, the out-domain finetuning do not really hurts the model original capabilities acquired during pretraining. 
new_line
Previous work addressed this (finetuning VS pretraining) but more investigation in this regard is required (any PhDs here ? This could be your question ...)
text
Previous work addressed this (finetuning VS pretraining) but more investigation in this regard is required (any PhDs here ? This could be your question ...)
new_line
new_line
new_line
Check out the latest rankings: 
text
Check out the latest rankings: 
https://huggingface.co/spaces/inceptionai/AraGen-Leaderboard
https://huggingface.co/spaces/inceptionai/AraGen-Leaderboard
link
3C3H AraGen Leaderboard welcomes today https://huggingface.co/deepseek-ai/DeepSeek-V3 and 12 other models (including the late gpt-3.5 💀) to the ranking of best LLMs in Arabic !


Observations:
- DeepSeek-v3 ranked 3rd and only Open model among the top 5 !

- A 14B open model (https://huggingface.co/Qwen/Qwen2.5-14B-Instruct) outperforms gpt-3.5-turbo-0125 (from last year). This shows how much we came in advancing and supporting Arabic presence within the LLM ecosystem ! 

- Contrary to what observed in likelihood-acc leaderboards (like https://huggingface.co/spaces/OALL/Open-Arabic-LLM-Leaderboard) further finetuned models like https://huggingface.co/maldv/Qwentile2.5-32B-Instruct actually decreased the performance compared to the original model https://huggingface.co/Qwen/Qwen2.5-32B-Instruct.
It's worth to note that the decrease is statiscally insignificant which imply that at best, the out-domain finetuning do not really hurts the model original capabilities acquired during pretraining. 
Previous work addressed this (finetuning VS pretraining) but more investigation in this regard is required (any PhDs here ? This could be your question ...)


Check out the latest rankings: https://huggingface.co/spaces/inceptionai/AraGen-Leaderboard
👍
prithivMLmods John6666 adamm-hf MohamedRashad Lyte abdeljalilELmajjodi