Decouple to Generalize: Context-First Self-Evolving Learning for Data-Scarce Vision-Language Reasoning
Recent vision-language models (VLMs) achieve remarkable reasoning through reinforcement learning (RL), which provides a feasible solution for realizing continuous self-evolving large vision-language models (LVLMs) in the era of experience. However, RL for VLMs requires abundant high-quality multimodal data, especially challenging in specialized domains like chemistry, earth sciences, and multimodal mathematics. Existing strategies such as synthetic data and self-rewarding mechanisms suffer from limited distributions and alignment difficulties, ultimately causing reward hacking: models exploit high-reward patterns, collapsing policy entropy and destabilizing training. We propose DoGe (Decouple to Generalize), a dual-decoupling framework that guides models to first learn from context rather than problem solving by refocusing on the problem context scenarios overlooked by synthetic data methods. By decoupling learning process into dual components (Thinker and Solver), we reasonably quantify the reward signals of this process and propose a two-stage RL post-training approach from freely exploring context to practically solving tasks. Second, to increase the diversity of training data, DoGe constructs an evolving curriculum learning pipeline: an expanded native domain knowledge corpus and an iteratively evolving seed problems pool. Experiments show that our method consistently outperforms the baseline across various benchmarks, providing a scalable pathway for realizing self-evolving LVLMs.
6937c87819d912300c34a455
Tingyu Li
claimed_verified
2025-12-09T09:21:24.369Z
66b72e76d3e1ba8885ce8aed
/avatars/12d9c6a952e6c1a35a5bfd233c552364.svg
TINGYU LI
Saito-Karuha
user
Saito-Karuha
6937c87819d912300c34a456
Zheng Sun
6937c87819d912300c34a457
Jingxuan Wei
admin_assigned
2025-12-09T11:51:49.650Z
6446445aab86b035add78ae9
/avatars/de85d372224b91cd04fc541a384cfc98.svg
Jingxuan Wei
weijingxuan
user
weijingxuan
6937c87819d912300c34a458
Siyuan Li
6937c87819d912300c34a459
Conghui He
admin_assigned
2025-12-09T11:51:58.977Z
63f9fca8d4349b157a109eec
/avatars/fa1f2ae7972d7cde99dab178136ccbb0.svg
Conghui He
conghui
user
conghui
6937c87819d912300c34a45a
Lijun Wu
6937c87819d912300c34a45b
Cheng Tan
claimed_verified
2025-12-10T13:07:57.971Z
64be296a46cc3cdfbb057f7e
https://cdn-avatars.huggingface.co/v1/production/uploads/64be296a46cc3cdfbb057f7e/jSHeNY2AcPifCZzJyFhr4.jpeg
Cheng Tan
chengtan9907
user
chengtan9907
Decouple to Generalize: Context-First Self-Evolving Learning for Data-Scarce Vision-Language Reasoning
Recent vision-language models (VLMs) achieve remarkable reasoning through reinforcement learning (RL), which provides a feasible solution for realizing continuous self-evolving large vision-language models (LVLMs) in the era of experience. However, RL for VLMs requires abundant high-quality multimodal data, especially challenging in specialized domains like chemistry, earth sciences, and multimodal mathematics. Existing strategies such as synthetic data and self-rewarding mechanisms suffer from limited distributions and alignment difficulties, ultimately causing reward hacking: models exploit high-reward patterns, collapsing policy entropy and destabilizing training. We propose DoGe (Decouple to Generalize), a dual-decoupling framework that guides models to first learn from context rather than problem solving by refocusing on the problem context scenarios overlooked by synthetic data methods. By decoupling learning process into dual components (Thinker and Solver), we reasonably quantify the reward signals of this process and propose a two-stage RL post-training approach from freely exploring context to practically solving tasks. Second, to increase the diversity of training data, DoGe constructs an evolving curriculum learning pipeline: an expanded native domain knowledge corpus and an iteratively evolving seed problems pool. Experiments show that our method consistently outperforms the baseline across various benchmarks, providing a scalable pathway for realizing self-evolving LVLMs.
DoGe, a dual-decoupling framework, enhances vision-language models by separating context learning from problem solving, using a curriculum learning pipeline to improve reward signals and data diversity.
vision-language models reinforcement learning large vision-language models synthetic data self-rewarding mechanisms reward hacking dual-decoupling framework Thinker Solver two-stage RL post-training curriculum learning native domain knowledge corpus seed problems pool