Adapting Language-Specific LLMs to a Reasoning Model in One Day via Model Merging -- An Open Recipe
This paper investigates data selection and model merging methodologies aimed at incorporating advanced reasoning capabilities such as those of DeepSeek R1 into language-specific large language models (LLMs), with a particular focus on the Thai LLM. Our goal is to enhance the reasoning capabilities of language-specific LLMs while maintaining their target language abilities. DeepSeek R1 excels in reasoning but primarily benefits high-resource languages such as English and Chinese. However, low-resource languages remain underserved due to the dominance of English-centric training data and model optimizations, which limit performance in these languages. This limitation results in unreliable code-switching and diminished effectiveness on tasks in low-resource languages. Meanwhile, local and regional LLM initiatives have attempted to bridge this gap by developing language-specific LLMs that focus on improving local linguistic fidelity. We demonstrate that, with only publicly available datasets and a computational budget of $120, it is possible to enhance the reasoning capabilities of language-specific LLMs to match the level of DeepSeek R1, without compromising their performance on target language tasks.
67aea8d7926b659c7e959bbc
Kunat Pipatanakul
admin_assigned
2025-02-14T12:04:29.437Z
62d192c2d50433c35eb1b48e
https://cdn-avatars.huggingface.co/v1/production/uploads/62d192c2d50433c35eb1b48e/VjmDu8GOIuLuQNBQdQLLS.png
Kunat Pipatanakul
kunato
user
kunato
67aea8d7926b659c7e959bbd
Pittawat Taveekitworachai
claimed_verified
2025-02-14T08:01:21.838Z
615313b0793ef66b3324da1f
https://cdn-avatars.huggingface.co/v1/production/uploads/615313b0793ef66b3324da1f/VyJniD3dxbV5a2CMgVVQ2.jpeg
Pittawat Taveekitworachai
pittawat
user
pittawat
67aea8d7926b659c7e959bbe
Potsawee Manakul
admin_assigned
2025-02-14T12:04:35.434Z
63f6a050b4c9a104f4b95755
https://cdn-avatars.huggingface.co/v1/production/uploads/63f6a050b4c9a104f4b95755/eJQyJkenSz536j-EGcpkH.jpeg
Potsawee Manakul
potsawee
user
potsawee
67aea8d7926b659c7e959bbf
Kasima Tharnpipitchai
Adapting Language-Specific LLMs to a Reasoning Model in One Day via Model Merging -- An Open Recipe
This paper investigates data selection and model merging methodologies aimed at incorporating advanced reasoning capabilities such as those of DeepSeek R1 into language-specific large language models (LLMs), with a particular focus on the Thai LLM. Our goal is to enhance the reasoning capabilities of language-specific LLMs while maintaining their target language abilities. DeepSeek R1 excels in reasoning but primarily benefits high-resource languages such as English and Chinese. However, low-resource languages remain underserved due to the dominance of English-centric training data and model optimizations, which limit performance in these languages. This limitation results in unreliable code-switching and diminished effectiveness on tasks in low-resource languages. Meanwhile, local and regional LLM initiatives have attempted to bridge this gap by developing language-specific LLMs that focus on improving local linguistic fidelity. We demonstrate that, with only publicly available datasets and a computational budget of $120, it is possible to enhance the reasoning capabilities of language-specific LLMs to match the level of DeepSeek R1, without compromising their performance on target language tasks.
Research demonstrates that language-specific LLMs can achieve advanced reasoning capabilities comparable to DeepSeek R1 using limited computational resources while preserving target language performance.
data selection model merging large language models DeepSeek R1 language-specific LLMs reasoning capabilities code-switching computational budget public datasets