Xmodel-2.5: 1.3B Data-Efficient Reasoning SLM
Large language models deliver strong reasoning and tool-use skills, yet their computational demands make them impractical for edge or cost-sensitive deployments. We present Xmodel-2.5, a 1.3-billion-parameter small language model designed as a drop-in agent core. Training with maximal-update parameterization (μP) allows hyper-parameters tuned on a 20M-parameter proxy to transfer directly to the full model, even under the parameter-tied tie-word-embedding architecture. A 1.4T-token Warmup--Stable--Decay curriculum is used, and we further show that switching from AdamW to Muon during the decay phase improves the 13-task reasoning average by 4.58\,\% while keeping every other hyper-parameter fixed, verifying that early AdamW stability can be paired with late Muon sharpening for better downstream performance. FP8-mixed-precision training balances accuracy and throughput. All checkpoints, recipes, and evaluation code are released under the Apache-2.0 license.https://huggingface.co/XiaoduoAILab/Xmodel-2.5 and https://huggingface.co/XiaoduoAILab/Xmodel-2.5-history (training checkpoints). Training code and evaluation harness: https://github.com/XiaoduoAILab/Xmodel-2.5.
692b8c644397b1ec214f65c1
Yang Liu
claimed_verified
2025-11-30T20:02:39.565Z
62bc03c2c5b1c4341ea2cc81
https://cdn-avatars.huggingface.co/v1/production/uploads/62bc03c2c5b1c4341ea2cc81/bsnW3P-sfTSw8K-lOdmrV.jpeg
Yang Liu
foamliu
user
foamliu
692b8c644397b1ec214f65c2
Xiaolong Zhong
692b8c644397b1ec214f65c3
Ling Jiang
Xmodel-2.5: 1.3B Data-Efficient Reasoning SLM
Large language models deliver strong reasoning and tool-use skills, yet their computational demands make them impractical for edge or cost-sensitive deployments. We present Xmodel-2.5, a 1.3-billion-parameter small language model designed as a drop-in agent core. Training with maximal-update parameterization (μP) allows hyper-parameters tuned on a 20M-parameter proxy to transfer directly to the full model, even under the parameter-tied tie-word-embedding architecture. A 1.4T-token Warmup--Stable--Decay curriculum is used, and we further show that switching from AdamW to Muon during the decay phase improves the 13-task reasoning average by 4.58\,\% while keeping every other hyper-parameter fixed, verifying that early AdamW stability can be paired with late Muon sharpening for better downstream performance. FP8-mixed-precision training balances accuracy and throughput. All checkpoints, recipes, and evaluation code are released under the Apache-2.0 license.https://huggingface.co/XiaoduoAILab/Xmodel-2.5 and https://huggingface.co/XiaoduoAILab/Xmodel-2.5-history (training checkpoints). Training code and evaluation harness: https://github.com/XiaoduoAILab/Xmodel-2.5.
Xmodel-2.5, a 1.3-billion-parameter language model, uses maximal-update parameterization and a modified training curriculum to improve performance and efficiency, making it suitable for edge deployments.
maximal-update parameterization tie-word-embedding Warmup-Stable-Decay curriculum AdamW Muon FP8-mixed-precision training