Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modeling
We introduce Latent Particle World Model (LPWM), a self-supervised object-centric world model scaled to real-world multi-object datasets and applicable in decision-making. LPWM autonomously discovers keypoints, bounding boxes, and object masks directly from video data, enabling it to learn rich scene decompositions without supervision. Our architecture is trained end-to-end purely from videos and supports flexible conditioning on actions, language, and image goals. LPWM models stochastic particle dynamics via a novel latent action module and achieves state-of-the-art results on diverse real-world and synthetic datasets. Beyond stochastic video modeling, LPWM is readily applicable to decision-making, including goal-conditioned imitation learning, as we demonstrate in the paper. Code, data, pre-trained models and video rollouts are available: https://taldatech.github.io/lpwm-web
69aaa387cedd2750d3d9aab3
Tal Daniel
claimed_verified
2026-03-06T13:54:27.670Z
687e75de730fb0e239c379a8
/avatars/57508ee60196871f2cc3f496aa037e1a.svg
Tal Daniel
taldatech
user
taldatech
69aaa387cedd2750d3d9aab4
Carl Qi
69aaa387cedd2750d3d9aab5
Dan Haramati
claimed_verified
2026-03-09T10:01:48.099Z
67be4eb6a344bd7c64b50c3e
https://cdn-avatars.huggingface.co/v1/production/uploads/67be4eb6a344bd7c64b50c3e/eSc7NrHSg_wMuO_EubcTd.jpeg
Dan Haramati
DanHrmti
user
DanHrmti
69aaa387cedd2750d3d9aab6
Amir Zadeh
69aaa387cedd2750d3d9aab7
Chuan Li
69aaa387cedd2750d3d9aab8
Aviv Tamar
69aaa387cedd2750d3d9aab9
Deepak Pathak
69aaa387cedd2750d3d9aaba
David Held
Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modeling
We introduce Latent Particle World Model (LPWM), a self-supervised object-centric world model scaled to real-world multi-object datasets and applicable in decision-making. LPWM autonomously discovers keypoints, bounding boxes, and object masks directly from video data, enabling it to learn rich scene decompositions without supervision. Our architecture is trained end-to-end purely from videos and supports flexible conditioning on actions, language, and image goals. LPWM models stochastic particle dynamics via a novel latent action module and achieves state-of-the-art results on diverse real-world and synthetic datasets. Beyond stochastic video modeling, LPWM is readily applicable to decision-making, including goal-conditioned imitation learning, as we demonstrate in the paper. Code, data, pre-trained models and video rollouts are available: https://taldatech.github.io/lpwm-web
LPWM autonomously discovers object representations from video data and models stochastic particle dynamics for decision-making applications.
object-centric world model self-supervised learning keypoints bounding boxes object masks end-to-end training latent action module stochastic particle dynamics goal-conditioned imitation learning