Experiential Reinforcement Learning
Reinforcement learning has become the central approach for language models (LMs) to learn from environmental reward or feedback. In practice, the environmental feedback is usually sparse and delayed. Learning from such signals is challenging, as LMs must implicitly infer how observed failures should translate into behavioral changes for future iterations. We introduce Experiential Reinforcement Learning (ERL), a training paradigm that embeds an explicit experience-reflection-consolidation loop into the reinforcement learning process. Given a task, the model generates an initial attempt, receives environmental feedback, and produces a reflection that guides a refined second attempt, whose success is reinforced and internalized into the base policy. This process converts feedback into structured behavioral revision, improving exploration and stabilizing optimization while preserving gains at deployment without additional inference cost. Across sparse-reward control environments and agentic reasoning benchmarks, ERL consistently improves learning efficiency and final performance over strong reinforcement learning baselines, achieving gains of up to +81% in complex multi-step environments and up to +11% in tool-using reasoning tasks. These results suggest that integrating explicit self-reflection into policy training provides a practical mechanism for transforming feedback into durable behavioral improvement.
69941b5e50fb2c0be4783de6
Taiwei Shi
claimed_verified
2026-02-17T15:08:55.504Z
62e1b3cb3eb0730f621a83f6
https://cdn-avatars.huggingface.co/v1/production/uploads/1658958764563-noauth.jpeg
Taiwei Shi
MaksimSTW
user
MaksimSTW
69941b5e50fb2c0be4783de7
Sihao Chen
claimed_verified
2026-02-18T09:07:12.393Z
607d11d688160e14e4e2e4cf
/avatars/d9dff689bb705bcd40b76da1326560eb.svg
Sihao Chen
sihaochen
user
sihaochen
69941b5e50fb2c0be4783de8
Bowen Jiang
69941b5e50fb2c0be4783de9
Linxin Song
claimed_verified
2026-02-17T15:08:53.295Z
64d660308ebc40443813f014
/avatars/516bb2d2383be99794e366dfb41636b6.svg
Linxin Song
linxinso
user
linxinso
69941b5e50fb2c0be4783dea
Longqi Yang
69941b5e50fb2c0be4783deb
Jieyu Zhao
Experiential Reinforcement Learning
Reinforcement learning has become the central approach for language models (LMs) to learn from environmental reward or feedback. In practice, the environmental feedback is usually sparse and delayed. Learning from such signals is challenging, as LMs must implicitly infer how observed failures should translate into behavioral changes for future iterations. We introduce Experiential Reinforcement Learning (ERL), a training paradigm that embeds an explicit experience-reflection-consolidation loop into the reinforcement learning process. Given a task, the model generates an initial attempt, receives environmental feedback, and produces a reflection that guides a refined second attempt, whose success is reinforced and internalized into the base policy. This process converts feedback into structured behavioral revision, improving exploration and stabilizing optimization while preserving gains at deployment without additional inference cost. Across sparse-reward control environments and agentic reasoning benchmarks, ERL consistently improves learning efficiency and final performance over strong reinforcement learning baselines, achieving gains of up to +81% in complex multi-step environments and up to +11% in tool-using reasoning tasks. These results suggest that integrating explicit self-reflection into policy training provides a practical mechanism for transforming feedback into durable behavioral improvement.
Experiential Reinforcement Learning introduces an explicit experience-reflection-consolidation loop that improves learning efficiency and performance in sparse-reward environments by enabling structured behavioral revision without additional inference costs.
reinforcement learning environmental feedback policy training self-reflection behavioral revision exploration optimization