SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning
Large Language Model (LLM) agents have shown stunning results in complex tasks, yet they often operate in isolation, failing to learn from past experiences. Existing memory-based methods primarily store raw trajectories, which are often redundant and noise-heavy. This prevents agents from extracting high-level, reusable behavioral patterns that are essential for generalization. In this paper, we propose SkillRL, a framework that bridges the gap between raw experience and policy improvement through automatic skill discovery and recursive evolution. Our approach introduces an experience-based distillation mechanism to build a hierarchical skill library SkillBank, an adaptive retrieval strategy for general and task-specific heuristics, and a recursive evolution mechanism that allows the skill library to co-evolve with the agent's policy during reinforcement learning. These innovations significantly reduce the token footprint while enhancing reasoning utility. Experimental results on ALFWorld, WebShop and seven search-augmented tasks demonstrate that SkillRL achieves state-of-the-art performance, outperforming strong baselines over 15.3% and maintaining robustness as task complexity increases. Code is available at this https://github.com/aiming-lab/SkillRL.
698aba731b2dc6b37d61b0e4
Peng Xia
claimed_verified
2026-02-11T11:18:58.896Z
643e9ee6f6bb3c31a26e7bc4
/avatars/acfaa7d6a23dada24c86b954c3be116a.svg
Peng Xia
richardxp888
user
richardxp888
698aba731b2dc6b37d61b0e5
Jianwen Chen
698aba731b2dc6b37d61b0e6
Hanyang Wang
698aba731b2dc6b37d61b0e7
Jiaqi Liu
698aba731b2dc6b37d61b0e8
Kaide Zeng
698aba731b2dc6b37d61b0e9
Yu Wang
claimed_verified
2026-02-11T11:18:56.703Z
63234809155b0e2c44f354d6
/avatars/60d38f8f0e12363f3f5e0388e635d7b6.svg
Yu Wang
YuWangX
user
YuWangX
698aba731b2dc6b37d61b0ea
Siwei Han
698aba731b2dc6b37d61b0eb
Yiyang Zhou
698aba731b2dc6b37d61b0ec
Xujiang Zhao
698aba731b2dc6b37d61b0ed
Haifeng Chen
698aba731b2dc6b37d61b0ee
Zeyu Zheng
698aba731b2dc6b37d61b0ef
Cihang Xie
698aba731b2dc6b37d61b0f0
Huaxiu Yao
claimed_verified
2026-03-19T13:05:07.727Z
65f667a455009c4ad9e6ac4c
/avatars/ed10c2cf2ba3fde6d7da93f076961607.svg
Yao
Huaxiu
user
Huaxiu
SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning
Large Language Model (LLM) agents have shown stunning results in complex tasks, yet they often operate in isolation, failing to learn from past experiences. Existing memory-based methods primarily store raw trajectories, which are often redundant and noise-heavy. This prevents agents from extracting high-level, reusable behavioral patterns that are essential for generalization. In this paper, we propose SkillRL, a framework that bridges the gap between raw experience and policy improvement through automatic skill discovery and recursive evolution. Our approach introduces an experience-based distillation mechanism to build a hierarchical skill library SkillBank, an adaptive retrieval strategy for general and task-specific heuristics, and a recursive evolution mechanism that allows the skill library to co-evolve with the agent's policy during reinforcement learning. These innovations significantly reduce the token footprint while enhancing reasoning utility. Experimental results on ALFWorld, WebShop and seven search-augmented tasks demonstrate that SkillRL achieves state-of-the-art performance, outperforming strong baselines over 15.3% and maintaining robustness as task complexity increases. Code is available at this https://github.com/aiming-lab/SkillRL.
SkillRL enables LLM agents to improve through hierarchical skill discovery and recursive policy evolution, achieving superior performance on complex tasks while reducing computational overhead.
large language model agents reinforcement learning skill discovery recursive evolution skill library SkillBank experience-based distillation adaptive retrieval strategy policy improvement token footprint