Planning from Observation and Interaction
Observational learning requires an agent to learn to perform a task by referencing only observations of the performed task. This work investigates the equivalent setting in real-world robot learning where access to hand-designed rewards and demonstrator actions are not assumed. To address this data-constrained setting, this work presents a planning-based Inverse Reinforcement Learning (IRL) algorithm for world modeling from observation and interaction alone. Experiments conducted entirely in the real-world demonstrate that this paradigm is effective for learning image-based manipulation tasks from scratch in under an hour, without assuming prior knowledge, pre-training, or data of any kind beyond task observations. Moreover, this work demonstrates that the learned world model representation is capable of online transfer learning in the real-world from scratch. In comparison to existing approaches, including IRL, RL, and Behavior Cloning (BC), which have more restrictive assumptions, the proposed approach demonstrates significantly greater sample efficiency and success rates, enabling a practical path forward for online world modeling and planning from observation and interaction. Videos and more at: https://uwrobotlearning.github.io/mpail2/.
69a63772663a122a7ce66673
Tyler Han
claimed_verified
2026-03-03T10:39:53.344Z
67e8f74469f530a28c90fad1
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/kgmG-qBzPst52ZxuMov6Q.png
Tyler Han
than-uw
user
than-uw
69a63772663a122a7ce66674
Siyang Shen
69a63772663a122a7ce66675
Rohan Baijal
69a63772663a122a7ce66676
Harine Ravichandiran
69a63772663a122a7ce66677
Bat Nemekhbold
claimed_verified
2026-03-23T11:00:35.626Z
6695f781659a52fa2750a83d
/avatars/be66afa64934f0d2b3a7d4ee56520403.svg
Bat Nemekhbold
bxtbold
user
bxtbold
69a63772663a122a7ce66678
Kevin Huang
69a63772663a122a7ce66679
Sanghun Jung
69a63772663a122a7ce6667a
Byron Boots
Planning from Observation and Interaction
Observational learning requires an agent to learn to perform a task by referencing only observations of the performed task. This work investigates the equivalent setting in real-world robot learning where access to hand-designed rewards and demonstrator actions are not assumed. To address this data-constrained setting, this work presents a planning-based Inverse Reinforcement Learning (IRL) algorithm for world modeling from observation and interaction alone. Experiments conducted entirely in the real-world demonstrate that this paradigm is effective for learning image-based manipulation tasks from scratch in under an hour, without assuming prior knowledge, pre-training, or data of any kind beyond task observations. Moreover, this work demonstrates that the learned world model representation is capable of online transfer learning in the real-world from scratch. In comparison to existing approaches, including IRL, RL, and Behavior Cloning (BC), which have more restrictive assumptions, the proposed approach demonstrates significantly greater sample efficiency and success rates, enabling a practical path forward for online world modeling and planning from observation and interaction. Videos and more at: https://uwrobotlearning.github.io/mpail2/.
A planning-based inverse reinforcement learning algorithm enables real-world robot manipulation learning from observations alone, achieving efficient online transfer learning without prior knowledge or pre-training.
inverse reinforcement learning world modeling observational learning real-world robot learning planning-based IRL online transfer learning sample efficiency