DARC: Decoupled Asymmetric Reasoning Curriculum for LLM Evolution
Self-play with large language models has emerged as a promising paradigm for achieving self-improving artificial intelligence. However, existing self-play frameworks often suffer from optimization instability, due to (i) non-stationary objectives induced by solver-dependent reward feedback for the Questioner, and (ii) bootstrapping errors from self-generated pseudo-labels used to supervise the Solver. To mitigate these challenges, we introduce DARC (Decoupled Asymmetric Reasoning Curriculum), a two-stage framework that stabilizes the self-evolution process. First, we train the Questioner to synthesize difficulty-calibrated questions, conditioned on explicit difficulty levels and external corpora. Second, we train the Solver with an asymmetric self-distillation mechanism, where a document-augmented teacher generates high-quality pseudo-labels to supervise the student Solver that lacks document access. Empirical results demonstrate that DARC is model-agnostic, yielding an average improvement of 10.9 points across nine reasoning benchmarks and three backbone models. Moreover, DARC consistently outperforms all baselines and approaches the performance of fully supervised models without relying on human annotations.The code is available at https://github.com/RUCBM/DARC.
697058efa8be625b19c2af2d
Shengda Fan
697058efa8be625b19c2af2e
Xuyan Ye
claimed_verified
2026-01-21T09:19:52.790Z
68905a353cf91a8e828fd8a1
https://cdn-avatars.huggingface.co/v1/production/uploads/68905a353cf91a8e828fd8a1/lyXMELqDGOHNlXeYlTL5X.jpeg
Xuyan Ye
LulaCola
user
LulaCola
697058efa8be625b19c2af2f
Yankai Lin
DARC: Decoupled Asymmetric Reasoning Curriculum for LLM Evolution
Self-play with large language models has emerged as a promising paradigm for achieving self-improving artificial intelligence. However, existing self-play frameworks often suffer from optimization instability, due to (i) non-stationary objectives induced by solver-dependent reward feedback for the Questioner, and (ii) bootstrapping errors from self-generated pseudo-labels used to supervise the Solver. To mitigate these challenges, we introduce DARC (Decoupled Asymmetric Reasoning Curriculum), a two-stage framework that stabilizes the self-evolution process. First, we train the Questioner to synthesize difficulty-calibrated questions, conditioned on explicit difficulty levels and external corpora. Second, we train the Solver with an asymmetric self-distillation mechanism, where a document-augmented teacher generates high-quality pseudo-labels to supervise the student Solver that lacks document access. Empirical results demonstrate that DARC is model-agnostic, yielding an average improvement of 10.9 points across nine reasoning benchmarks and three backbone models. Moreover, DARC consistently outperforms all baselines and approaches the performance of fully supervised models without relying on human annotations.The code is available at https://github.com/RUCBM/DARC.
A two-stage framework called DARC stabilizes self-play with large language models by decoupling question generation and using asymmetric self-distillation with document-augmented teachers to improve reasoning performance.
self-play large language models optimization instability non-stationary objectives bootstrapping errors self-generated pseudo-labels Questioner Solver decoupled asymmetric reasoning curriculum self-distillation document-augmented teacher self-evolution process reasoning benchmarks backbone models fully supervised models