EgoPush: Learning End-to-End Egocentric Multi-Object Rearrangement for Mobile Robots
Humans can rearrange objects in cluttered environments using egocentric perception, navigating occlusions without global coordinates. Inspired by this capability, we study long-horizon multi-object non-prehensile rearrangement for mobile robots using a single egocentric camera. We introduce EgoPush, a policy learning framework that enables egocentric, perception-driven rearrangement without relying on explicit global state estimation that often fails in dynamic scenes. EgoPush designs an object-centric latent space to encode relative spatial relations among objects, rather than absolute poses. This design enables a privileged reinforcement-learning (RL) teacher to jointly learn latent states and mobile actions from sparse keypoints, which is then distilled into a purely visual student policy. To reduce the supervision gap between the omniscient teacher and the partially observed student, we restrict the teacher's observations to visually accessible cues. This induces active perception behaviors that are recoverable from the student's viewpoint. To address long-horizon credit assignment, we decompose rearrangement into stage-level subproblems using temporally decayed, stage-local completion rewards. Extensive simulation experiments demonstrate that EgoPush significantly outperforms end-to-end RL baselines in success rate, with ablation studies validating each design choice. We further demonstrate zero-shot sim-to-real transfer on a mobile platform in the real world. Code and videos are available at https://ai4ce.github.io/EgoPush/.
699bbee4f723198bd53a6300
Boyuan An
claimed_verified
2026-02-26T10:03:44.392Z
699fb962059127c65e6f7ca1
/avatars/6c4c08f735e7b98e32ea71504820fd7b.svg
Boyuan An
BGDabyss
user
BGDabyss
699bbee4f723198bd53a6301
Zhexiong Wang
699bbee4f723198bd53a6302
Yipeng Wang
699bbee4f723198bd53a6303
Jiaqi Li
699bbee4f723198bd53a6304
Sihang Li
699bbee4f723198bd53a6305
Jing Zhang
699bbee4f723198bd53a6306
Chen Feng
EgoPush: Learning End-to-End Egocentric Multi-Object Rearrangement for Mobile Robots
Humans can rearrange objects in cluttered environments using egocentric perception, navigating occlusions without global coordinates. Inspired by this capability, we study long-horizon multi-object non-prehensile rearrangement for mobile robots using a single egocentric camera. We introduce EgoPush, a policy learning framework that enables egocentric, perception-driven rearrangement without relying on explicit global state estimation that often fails in dynamic scenes. EgoPush designs an object-centric latent space to encode relative spatial relations among objects, rather than absolute poses. This design enables a privileged reinforcement-learning (RL) teacher to jointly learn latent states and mobile actions from sparse keypoints, which is then distilled into a purely visual student policy. To reduce the supervision gap between the omniscient teacher and the partially observed student, we restrict the teacher's observations to visually accessible cues. This induces active perception behaviors that are recoverable from the student's viewpoint. To address long-horizon credit assignment, we decompose rearrangement into stage-level subproblems using temporally decayed, stage-local completion rewards. Extensive simulation experiments demonstrate that EgoPush significantly outperforms end-to-end RL baselines in success rate, with ablation studies validating each design choice. We further demonstrate zero-shot sim-to-real transfer on a mobile platform in the real world. Code and videos are available at https://ai4ce.github.io/EgoPush/.
EgoPush enables robot manipulation in cluttered environments through perception-driven policy learning that uses object-centric latent spaces and stage-decomposed rewards for long-horizon tasks.
policy learning egocentric perception non-prehensile rearrangement object-centric latent space reinforcement-learning visual student policy active perception stage-local completion rewards zero-shot sim-to-real transfer