PhysWorld: From Real Videos to World Models of Deformable Objects via
  Physics-Aware Demonstration Synthesis
Interactive world models that simulate object dynamics are crucial for
robotics, VR, and AR. However, it remains a significant challenge to learn
physics-consistent dynamics models from limited real-world video data,
especially for deformable objects with spatially-varying physical properties.
To overcome the challenge of data scarcity, we propose PhysWorld, a novel
framework that utilizes a simulator to synthesize physically plausible and
diverse demonstrations to learn efficient world models. Specifically, we first
construct a physics-consistent digital twin within MPM simulator via
constitutive model selection and global-to-local optimization of physical
properties. Subsequently, we apply part-aware perturbations to the physical
properties and generate various motion patterns for the digital twin,
synthesizing extensive and diverse demonstrations. Finally, using these
demonstrations, we train a lightweight GNN-based world model that is embedded
with physical properties. The real video can be used to further refine the
physical properties. PhysWorld achieves accurate and fast future predictions
for various deformable objects, and also generalizes well to novel
interactions. Experiments show that PhysWorld has competitive performance while
enabling inference speeds 47 times faster than the recent state-of-the-art
method, i.e., PhysTwin.
68fed0996cdff8b857f47101
Yu Yang
68fed0996cdff8b857f47102
Zhilu Zhang
claimed_verified
2025-10-27T10:27:36.894Z
643eac99510c894d9110a9ed
https://cdn-avatars.huggingface.co/v1/production/uploads/643eac99510c894d9110a9ed/VYPrfvMoe-gOoQhLU_v84.jpeg
Zhilu Zhang
cszhilu1998
user
cszhilu1998
68fed0996cdff8b857f47103
Xiang Zhang
68fed0996cdff8b857f47104
Yihan Zeng
68fed0996cdff8b857f47105
Hui Li
68fed0996cdff8b857f47106
Wangmeng Zuo
PhysWorld: From Real Videos to World Models of Deformable Objects via
  Physics-Aware Demonstration Synthesis
Interactive world models that simulate object dynamics are crucial for
robotics, VR, and AR. However, it remains a significant challenge to learn
physics-consistent dynamics models from limited real-world video data,
especially for deformable objects with spatially-varying physical properties.
To overcome the challenge of data scarcity, we propose PhysWorld, a novel
framework that utilizes a simulator to synthesize physically plausible and
diverse demonstrations to learn efficient world models. Specifically, we first
construct a physics-consistent digital twin within MPM simulator via
constitutive model selection and global-to-local optimization of physical
properties. Subsequently, we apply part-aware perturbations to the physical
properties and generate various motion patterns for the digital twin,
synthesizing extensive and diverse demonstrations. Finally, using these
demonstrations, we train a lightweight GNN-based world model that is embedded
with physical properties. The real video can be used to further refine the
physical properties. PhysWorld achieves accurate and fast future predictions
for various deformable objects, and also generalizes well to novel
interactions. Experiments show that PhysWorld has competitive performance while
enabling inference speeds 47 times faster than the recent state-of-the-art
method, i.e., PhysTwin.
PhysWorld uses a simulator to generate diverse demonstrations for training a GNN-based world model, enabling accurate and fast predictions for deformable objects with competitive performance and faster inference speeds.
physics-consistent dynamics models deformable objects spatially-varying physical properties MPM simulator constitutive model selection global-to-local optimization part-aware perturbations GNN-based world model future predictions novel interactions PhysTwin