RoboPocket: Improve Robot Policies Instantly with Your Phone
Scaling imitation learning is fundamentally constrained by the efficiency of data collection. While handheld interfaces have emerged as a scalable solution for in-the-wild data acquisition, they predominantly operate in an open-loop manner: operators blindly collect demonstrations without knowing the underlying policy's weaknesses, leading to inefficient coverage of critical state distributions. Conversely, interactive methods like DAgger effectively address covariate shift but rely on physical robot execution, which is costly and difficult to scale. To reconcile this trade-off, we introduce RoboPocket, a portable system that enables Robot-Free Instant Policy Iteration using single consumer smartphones. Its core innovation is a Remote Inference framework that visualizes the policy's predicted trajectory via Augmented Reality (AR) Visual Foresight. This immersive feedback allows collectors to proactively identify potential failures and focus data collection on the policy's weak regions without requiring a physical robot. Furthermore, we implement an asynchronous Online Finetuning pipeline that continuously updates the policy with incoming data, effectively closing the learning loop in minutes. Extensive experiments demonstrate that RoboPocket adheres to data scaling laws and doubles the data efficiency compared to offline scaling strategies, overcoming their long-standing efficiency bottleneck. Moreover, our instant iteration loop also boosts sample efficiency by up to 2times in distributed environments a small number of interactive corrections per person. Project page and videos: https://robo-pocket.github.io.
69aa50d5c4f6edddcc6c86dc
Junjie Fang
69aa50d5c4f6edddcc6c86dd
Wendi Chen
claimed_verified
2026-03-06T13:55:02.439Z
670fad9288056fa3e0183c47
/avatars/d76c1de37dcf4053e83abfd87fb16e0f.svg
Wendi Chen (SII)
WendiChen
user
WendiChen
69aa50d5c4f6edddcc6c86de
Han Xue
69aa50d5c4f6edddcc6c86df
Fangyuan Zhou
69aa50d5c4f6edddcc6c86e0
Tian Le
69aa50d5c4f6edddcc6c86e1
Yi Wang
69aa50d5c4f6edddcc6c86e2
Yuting Zhang
69aa50d5c4f6edddcc6c86e3
Jun Lv
69aa50d5c4f6edddcc6c86e4
Chuan Wen
69aa50d5c4f6edddcc6c86e5
Cewu Lu
RoboPocket: Improve Robot Policies Instantly with Your Phone
Scaling imitation learning is fundamentally constrained by the efficiency of data collection. While handheld interfaces have emerged as a scalable solution for in-the-wild data acquisition, they predominantly operate in an open-loop manner: operators blindly collect demonstrations without knowing the underlying policy's weaknesses, leading to inefficient coverage of critical state distributions. Conversely, interactive methods like DAgger effectively address covariate shift but rely on physical robot execution, which is costly and difficult to scale. To reconcile this trade-off, we introduce RoboPocket, a portable system that enables Robot-Free Instant Policy Iteration using single consumer smartphones. Its core innovation is a Remote Inference framework that visualizes the policy's predicted trajectory via Augmented Reality (AR) Visual Foresight. This immersive feedback allows collectors to proactively identify potential failures and focus data collection on the policy's weak regions without requiring a physical robot. Furthermore, we implement an asynchronous Online Finetuning pipeline that continuously updates the policy with incoming data, effectively closing the learning loop in minutes. Extensive experiments demonstrate that RoboPocket adheres to data scaling laws and doubles the data efficiency compared to offline scaling strategies, overcoming their long-standing efficiency bottleneck. Moreover, our instant iteration loop also boosts sample efficiency by up to 2times in distributed environments a small number of interactive corrections per person. Project page and videos: https://robo-pocket.github.io.
RoboPocket enables efficient robot-free policy iteration using smartphone-based augmented reality visualization and asynchronous online fine-tuning to improve data and sample efficiency in imitation learning.
imitation learning data collection open-loop DAgger covariate shift Remote Inference Augmented Reality Visual Foresight Online Finetuning policy iteration sample efficiency data scaling laws