Imagine-then-Plan: Agent Learning from Adaptive Lookahead with World Models
Recent advances in world models have shown promise for modeling future dynamics of environmental states, enabling agents to reason and act without accessing real environments. Current methods mainly perform single-step or fixed-horizon rollouts, leaving their potential for complex task planning under-exploited. We propose Imagine-then-Plan (ITP), a unified framework for agent learning via lookahead imagination, where an agent's policy model interacts with the learned world model, yielding multi-step ``imagined'' trajectories. Since the imagination horizon may vary by tasks and stages, we introduce a novel adaptive lookahead mechanism by trading off the ultimate goal and task progress. The resulting imagined trajectories provide rich signals about future consequences, such as achieved progress and potential conflicts, which are fused with current observations, formulating a partially observable and imaginable Markov decision process to guide policy learning. We instantiate ITP with both training-free and reinforcement-trained variants. Extensive experiments across representative agent benchmarks demonstrate that ITP significantly outperforms competitive baselines. Further analyses validate that our adaptive lookahead largely enhances agents' reasoning capability, providing valuable insights into addressing broader, complex tasks.
696877d40ac10a06522f6abf
Youwei Liu
claimed_verified
2026-01-16T15:28:48.730Z
673ed93eb616e6ea9b23af22
/avatars/1bc13cd1a6d9bb6df252132d52741efa.svg
Liu Youwei
loyiv
user
loyiv
696877d40ac10a06522f6ac0
Jian Wang
claimed_verified
2026-01-15T09:33:36.254Z
63a4117984a6a25c65bc2fff
https://cdn-avatars.huggingface.co/v1/production/uploads/63a4117984a6a25c65bc2fff/PxgTP_U5a5egCC8xvSh0s.png
Jian Wang
jwanglvy
user
jwanglvy
696877d40ac10a06522f6ac1
Hanlin Wang
696877d40ac10a06522f6ac2
Beichen Guo
696877d40ac10a06522f6ac3
Wenjie Li
Imagine-then-Plan: Agent Learning from Adaptive Lookahead with World Models
Recent advances in world models have shown promise for modeling future dynamics of environmental states, enabling agents to reason and act without accessing real environments. Current methods mainly perform single-step or fixed-horizon rollouts, leaving their potential for complex task planning under-exploited. We propose Imagine-then-Plan (ITP), a unified framework for agent learning via lookahead imagination, where an agent's policy model interacts with the learned world model, yielding multi-step ``imagined'' trajectories. Since the imagination horizon may vary by tasks and stages, we introduce a novel adaptive lookahead mechanism by trading off the ultimate goal and task progress. The resulting imagined trajectories provide rich signals about future consequences, such as achieved progress and potential conflicts, which are fused with current observations, formulating a partially observable and imaginable Markov decision process to guide policy learning. We instantiate ITP with both training-free and reinforcement-trained variants. Extensive experiments across representative agent benchmarks demonstrate that ITP significantly outperforms competitive baselines. Further analyses validate that our adaptive lookahead largely enhances agents' reasoning capability, providing valuable insights into addressing broader, complex tasks.
Imagine-then-Plan framework enables agent learning through adaptive lookahead imagination, combining imagined trajectories with current observations to guide policy learning in complex task scenarios.
world models lookahead imagination multi-step trajectories adaptive lookahead mechanism partially observable Markov decision process policy learning reinforcement training agent benchmarks