Guided Self-Evolving LLMs with Minimal Human Supervision
AI self-evolution has long been envisioned as a path toward superintelligence, where models autonomously acquire, refine, and internalize knowledge from their own learning experiences. Yet in practice, unguided self-evolving systems often plateau quickly or even degrade as training progresses. These failures arise from issues such as concept drift, diversity collapse, and mis-evolution, as models reinforce their own biases and converge toward low-entropy behaviors. To enable models to self-evolve in a stable and controllable manner while minimizing reliance on human supervision, we introduce R-Few, a guided Self-Play Challenger-Solver framework that incorporates lightweight human oversight through in-context grounding and mixed training. At each iteration, the Challenger samples a small set of human-labeled examples to guide synthetic question generation, while the Solver jointly trains on human and synthetic examples under an online, difficulty-based curriculum. Across math and general reasoning benchmarks, R-Few achieves consistent and iterative improvements. For example, Qwen3-8B-Base improves by +3.0 points over R-Zero on math tasks and achieves performance on par with General-Reasoner, despite the latter being trained on 20 times more human data. Ablation studies confirm the complementary contributions of grounded challenger training and curriculum-based solver training, and further analysis shows that R-Few mitigates drift, yielding more stable and controllable co-evolutionary dynamics.
692fb2c026742347f61dac94
Wenhao Yu
claimed_verified
2025-12-03T09:15:34.540Z
5feab3a28a3201f8e554c969
https://cdn-avatars.huggingface.co/v1/production/uploads/1660795228685-5feab3a28a3201f8e554c969.png
Wenhao Yu
wyu1
user
wyu1
692fb2c026742347f61dac95
Zhenwen Liang
692fb2c026742347f61dac96
Chengsong Huang
claimed_verified
2025-12-22T11:01:19.785Z
62ea79dd01ed9b0e8f61ccd3
/avatars/70af83e0e267be39fcd5f23b85e2dafa.svg
Chengsong Huang
ChengsongHuang
user
ChengsongHuang
692fb2c026742347f61dac97
Kishan Panaganti
claimed_verified
2025-12-22T11:01:21.684Z
68266b5261ed4d89177c3612
/avatars/e41ef277047334174eca408aba2a63db.svg
Kishan Panaganti
kishanpb
user
kishanpb
692fb2c026742347f61dac98
Tianqing Fang
claimed_verified
2026-01-26T08:34:30.253Z
641129818573c51c0458b793
/avatars/d4bc67c160a07146cf41c614678aa36b.svg
Tianqing Fang
tqfang229
user
tqfang229
692fb2c026742347f61dac99
Haitao Mi
692fb2c026742347f61dac9a
Dong Yu
Guided Self-Evolving LLMs with Minimal Human Supervision
AI self-evolution has long been envisioned as a path toward superintelligence, where models autonomously acquire, refine, and internalize knowledge from their own learning experiences. Yet in practice, unguided self-evolving systems often plateau quickly or even degrade as training progresses. These failures arise from issues such as concept drift, diversity collapse, and mis-evolution, as models reinforce their own biases and converge toward low-entropy behaviors. To enable models to self-evolve in a stable and controllable manner while minimizing reliance on human supervision, we introduce R-Few, a guided Self-Play Challenger-Solver framework that incorporates lightweight human oversight through in-context grounding and mixed training. At each iteration, the Challenger samples a small set of human-labeled examples to guide synthetic question generation, while the Solver jointly trains on human and synthetic examples under an online, difficulty-based curriculum. Across math and general reasoning benchmarks, R-Few achieves consistent and iterative improvements. For example, Qwen3-8B-Base improves by +3.0 points over R-Zero on math tasks and achieves performance on par with General-Reasoner, despite the latter being trained on 20 times more human data. Ablation studies confirm the complementary contributions of grounded challenger training and curriculum-based solver training, and further analysis shows that R-Few mitigates drift, yielding more stable and controllable co-evolutionary dynamics.
R-Few, a guided Self-Play Challenger-Solver framework, enables stable and controllable model self-evolution with minimal human supervision, achieving performance improvements on math and reasoning benchmarks.
self-evolution superintelligence concept drift diversity collapse mis-evolution R-Few Self-Play Challenger-Solver in-context grounding mixed training synthetic question generation online curriculum Qwen3-8B-Base R-Zero General-Reasoner ablation studies co-evolutionary dynamics