Learning from Trials and Errors: Reflective Test-Time Planning for Embodied LLMs
Embodied LLMs endow robots with high-level task reasoning, but they cannot reflect on what went wrong or why, turning deployment into a sequence of independent trials where mistakes repeat rather than accumulate into experience. Drawing upon human reflective practitioners, we introduce Reflective Test-Time Planning, which integrates two modes of reflection: reflection-in-action, where the agent uses test-time scaling to generate and score multiple candidate actions using internal reflections before execution; and reflection-on-action, which uses test-time training to update both its internal reflection model and its action policy based on external reflections after execution. We also include retrospective reflection, allowing the agent to re-evaluate earlier decisions and perform model updates with hindsight for proper long-horizon credit assignment. Experiments on our newly-designed Long-Horizon Household benchmark and MuJoCo Cupboard Fitting benchmark show significant gains over baseline models, with ablative studies validating the complementary roles of reflection-in-action and reflection-on-action. Qualitative analyses, including real-robot trials, highlight behavioral correction through reflection.
699e7520dfbcf0b800aecb7c
Yining Hong
699e7520dfbcf0b800aecb7d
Huang Huang
699e7520dfbcf0b800aecb7e
Manling Li
claimed_verified
2026-04-13T07:38:40.045Z
6746a140f2ca2162e3bcfe2b
/avatars/d9d8cfb5f112e6ed7f6152fc230135d3.svg
Manling Li
ManlingLi
user
ManlingLi
699e7520dfbcf0b800aecb7f
Li Fei-Fei
699e7520dfbcf0b800aecb80
Jiajun Wu
699e7520dfbcf0b800aecb81
Yejin Choi
Learning from Trials and Errors: Reflective Test-Time Planning for Embodied LLMs
Embodied LLMs endow robots with high-level task reasoning, but they cannot reflect on what went wrong or why, turning deployment into a sequence of independent trials where mistakes repeat rather than accumulate into experience. Drawing upon human reflective practitioners, we introduce Reflective Test-Time Planning, which integrates two modes of reflection: reflection-in-action, where the agent uses test-time scaling to generate and score multiple candidate actions using internal reflections before execution; and reflection-on-action, which uses test-time training to update both its internal reflection model and its action policy based on external reflections after execution. We also include retrospective reflection, allowing the agent to re-evaluate earlier decisions and perform model updates with hindsight for proper long-horizon credit assignment. Experiments on our newly-designed Long-Horizon Household benchmark and MuJoCo Cupboard Fitting benchmark show significant gains over baseline models, with ablative studies validating the complementary roles of reflection-in-action and reflection-on-action. Qualitative analyses, including real-robot trials, highlight behavioral correction through reflection.
Reflective Test-Time Planning enhances robot decision-making by integrating multiple reflection mechanisms that enable learning from experience and improving long-horizon task performance.
embodied LLMs test-time planning reflection-in-action reflection-on-action retrospective reflection test-time scaling test-time training long-horizon task reasoning behavioral correction