PhysBrain: Human Egocentric Data as a Bridge from Vision Language Models to Physical Intelligence
Robotic generalization relies on physical intelligence: the ability to reason about state changes, contact-rich interactions, and long-horizon planning under egocentric perception and action. However, most VLMs are trained primarily on third-person data, creating a fundamental viewpoint mismatch for humanoid robots. Scaling robot egocentric data collection remains impractical due to high cost and limited diversity, whereas large-scale human egocentric videos offer a scalable alternative that naturally capture rich interaction context and causal structure. The key challenge is to convert raw egocentric videos into structured and reliable embodiment training supervision. Accordingly, we propose an Egocentric2Embodiment translation pipeline that transforms first-person videos into multi-level, schema-driven VQA supervision with enforced evidence grounding and temporal consistency, enabling the construction of the Egocentric2Embodiment dataset (E2E-3M) at scale. An egocentric-aware embodied brain, termed PhysBrain, is obtained by training on the E2E-3M dataset. PhysBrain exhibits substantially improved egocentric understanding, particularly for planning on EgoThink. It provides an egocentric-aware initialization that enables more sample-efficient VLA fine-tuning and higher SimplerEnv success rates (53.9\%), demonstrating effective transfer from human egocentric supervision to downstream robot control.
6944ba3efbf17e708e185f60
Xiaopeng Lin
claimed_verified
2025-12-19T08:58:14.947Z
6944c28d3cd5eeb7a7838663
/avatars/63f9f8c4e8462cbc9726bdee249fbcde.svg
lin
birdxp
user
birdxp
6944ba3efbf17e708e185f61
Shijie Lian
claimed_verified
2025-12-19T08:58:18.093Z
65ec01fd770aa0e25d9374dc
https://cdn-avatars.huggingface.co/v1/production/uploads/65ec01fd770aa0e25d9374dc/yvLWwBEdAdHb-8EdUHg3n.jpeg
Shijie Lian
LiamLian0727
user
LiamLian0727
6944ba3efbf17e708e185f62
Bin Yu
claimed_verified
2025-12-22T11:00:33.691Z
63d3b5f1640bb0f77173baea
https://cdn-avatars.huggingface.co/v1/production/uploads/1674819020331-noauth.jpeg
yubin
VLyb
user
VLyb
6944ba3efbf17e708e185f63
Ruoqi Yang
claimed_verified
2025-12-22T11:00:35.617Z
684a7f750fff63d29d18444c
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/kXTIoUPOM7LDztBcRL6YI.png
Ruoqi Yang
ruoqi25
user
ruoqi25
6944ba3efbf17e708e185f64
Changti Wu
claimed_verified
2026-02-13T09:38:25.273Z
67b55e66d454cc4d10d21cfd
/avatars/3b18014fa7e603a5940175896f89372a.svg
Changti Wu
MaplesWCT
user
MaplesWCT
6944ba3efbf17e708e185f65
Yuzhuo Miao
6944ba3efbf17e708e185f66
Yurun Jin
6944ba3efbf17e708e185f67
Yukun Shi
6944ba3efbf17e708e185f68
Cong Huang
6944ba3efbf17e708e185f69
Bojun Cheng
6944ba3efbf17e708e185f6a
Kai Chen
PhysBrain: Human Egocentric Data as a Bridge from Vision Language Models to Physical Intelligence
Robotic generalization relies on physical intelligence: the ability to reason about state changes, contact-rich interactions, and long-horizon planning under egocentric perception and action. However, most VLMs are trained primarily on third-person data, creating a fundamental viewpoint mismatch for humanoid robots. Scaling robot egocentric data collection remains impractical due to high cost and limited diversity, whereas large-scale human egocentric videos offer a scalable alternative that naturally capture rich interaction context and causal structure. The key challenge is to convert raw egocentric videos into structured and reliable embodiment training supervision. Accordingly, we propose an Egocentric2Embodiment translation pipeline that transforms first-person videos into multi-level, schema-driven VQA supervision with enforced evidence grounding and temporal consistency, enabling the construction of the Egocentric2Embodiment dataset (E2E-3M) at scale. An egocentric-aware embodied brain, termed PhysBrain, is obtained by training on the E2E-3M dataset. PhysBrain exhibits substantially improved egocentric understanding, particularly for planning on EgoThink. It provides an egocentric-aware initialization that enables more sample-efficient VLA fine-tuning and higher SimplerEnv success rates (53.9\%), demonstrating effective transfer from human egocentric supervision to downstream robot control.
Proposed Egocentric2Embodiment pipeline translates human egocentric videos into structured training data for robots, enhancing their egocentric understanding and task performance.
Egocentric2Embodiment VQA supervision evidence grounding temporal consistency Egocentric2Embodiment dataset E2E-3M PhysBrain egocentric-aware VLA fine-tuning SimplerEnv success rates