Kinema4D: Kinematic 4D World Modeling for Spatiotemporal Embodied Simulation
Simulating robot-world interactions is a cornerstone of Embodied AI. Recently, a few works have shown promise in leveraging video generations to transcend the rigid visual/physical constraints of traditional simulators. However, they primarily operate in 2D space or are guided by static environmental cues, ignoring the fundamental reality that robot-world interactions are inherently 4D spatiotemporal events that require precise interactive modeling. To restore this 4D essence while ensuring the precise robot control, we introduce Kinema4D, a new action-conditioned 4D generative robotic simulator that disentangles the robot-world interaction into: i) Precise 4D representation of robot controls: we drive a URDF-based 3D robot via kinematics, producing a precise 4D robot control trajectory. ii) Generative 4D modeling of environmental reactions: we project the 4D robot trajectory into a pointmap as a spatiotemporal visual signal, controlling the generative model to synthesize complex environments' reactive dynamics into synchronized RGB/pointmap sequences. To facilitate training, we curated a large-scale dataset called Robo4D-200k, comprising 201,426 robot interaction episodes with high-quality 4D annotations. Extensive experiments demonstrate that our method effectively simulates physically-plausible, geometry-consistent, and embodiment-agnostic interactions that faithfully mirror diverse real-world dynamics. For the first time, it shows potential zero-shot transfer capability, providing a high-fidelity foundation for advancing next-generation embodied simulation.
69ba487a5d2b9c28ea09f264
Mutian Xu
claimed_verified
2026-03-18T08:35:44.956Z
64749a0d5aba8edfb2eeaba7
https://cdn-avatars.huggingface.co/v1/production/uploads/64749a0d5aba8edfb2eeaba7/Tiy4DEdp3KQYh7Ij8Vmkn.png
Mutian Xu
Minoday
user
Minoday
69ba487a5d2b9c28ea09f265
Tianbao Zhang
admin_assigned
2026-03-18T09:57:26.700Z
656d84ee02a56b531ad17ed4
/avatars/e21d8df87e231b79a23b8d2483b04cfe.svg
Tianbao Zhang
ZTBob
user
ZTBob
69ba487a5d2b9c28ea09f266
Tianqi Liu
69ba487a5d2b9c28ea09f267
Zhaoxi Chen
admin_assigned
2026-03-18T09:57:36.798Z
62fc8cf7ee999004b5a8b982
/avatars/6c5dda9e58747054a989f077a078f3dc.svg
Zhaoxi Chen
FrozenBurning
user
FrozenBurning
69ba487a5d2b9c28ea09f268
Xiaoguang Han
69ba487a5d2b9c28ea09f269
Ziwei Liu
admin_assigned
2026-03-18T09:57:46.730Z
62ab1ac1d48b4d8b048a3473
https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
Ziwei Liu
liuziwei7
user
liuziwei7
Kinema4D: Kinematic 4D World Modeling for Spatiotemporal Embodied Simulation
Simulating robot-world interactions is a cornerstone of Embodied AI. Recently, a few works have shown promise in leveraging video generations to transcend the rigid visual/physical constraints of traditional simulators. However, they primarily operate in 2D space or are guided by static environmental cues, ignoring the fundamental reality that robot-world interactions are inherently 4D spatiotemporal events that require precise interactive modeling. To restore this 4D essence while ensuring the precise robot control, we introduce Kinema4D, a new action-conditioned 4D generative robotic simulator that disentangles the robot-world interaction into: i) Precise 4D representation of robot controls: we drive a URDF-based 3D robot via kinematics, producing a precise 4D robot control trajectory. ii) Generative 4D modeling of environmental reactions: we project the 4D robot trajectory into a pointmap as a spatiotemporal visual signal, controlling the generative model to synthesize complex environments' reactive dynamics into synchronized RGB/pointmap sequences. To facilitate training, we curated a large-scale dataset called Robo4D-200k, comprising 201,426 robot interaction episodes with high-quality 4D annotations. Extensive experiments demonstrate that our method effectively simulates physically-plausible, geometry-consistent, and embodiment-agnostic interactions that faithfully mirror diverse real-world dynamics. For the first time, it shows potential zero-shot transfer capability, providing a high-fidelity foundation for advancing next-generation embodied simulation.
Kinema4D introduces a 4D generative robotic simulator that models robot-world interactions through precise kinematic control and spatiotemporal environmental reaction synthesis, enabling physically plausible and embodiment-agnostic simulations with zero-shot transfer capability.
4D generative robotic simulator kinematics URDF-based 3D robot 4D robot control trajectory pointmap spatiotemporal visual signal generative model RGB/pointmap sequences Robo4D-200k zero-shot transfer