World Action Models are Zero-shot Policies
State-of-the-art Vision-Language-Action (VLA) models excel at semantic generalization but struggle to generalize to unseen physical motions in novel environments. We introduce DreamZero, a World Action Model (WAM) built upon a pretrained video diffusion backbone. Unlike VLAs, WAMs learn physical dynamics by predicting future world states and actions, using video as a dense representation of how the world evolves. By jointly modeling video and action, DreamZero learns diverse skills effectively from heterogeneous robot data without relying on repetitive demonstrations. This results in over 2x improvement in generalization to new tasks and environments compared to state-of-the-art VLAs in real robot experiments. Crucially, through model and system optimizations, we enable a 14B autoregressive video diffusion model to perform real-time closed-loop control at 7Hz. Finally, we demonstrate two forms of cross-embodiment transfer: video-only demonstrations from other robots or humans yield a relative improvement of over 42% on unseen task performance with just 10-20 minutes of data. More surprisingly, DreamZero enables few-shot embodiment adaptation, transferring to a new embodiment with only 30 minutes of play data while retaining zero-shot generalization.
6996793f1268a6b79e0d028d
Seonghyeon Ye
6996793f1268a6b79e0d028e
Yunhao Ge
6996793f1268a6b79e0d028f
Kaiyuan Zheng
6996793f1268a6b79e0d0290
Shenyuan Gao
6996793f1268a6b79e0d0291
Sihyun Yu
6996793f1268a6b79e0d0292
George Kurian
6996793f1268a6b79e0d0293
Suneel Indupuru
6996793f1268a6b79e0d0294
You Liang Tan
6996793f1268a6b79e0d0295
Chuning Zhu
6996793f1268a6b79e0d0296
Jiannan Xiang
6996793f1268a6b79e0d0297
Ayaan Malik
6996793f1268a6b79e0d0298
Kyungmin Lee
6996793f1268a6b79e0d0299
William Liang
6996793f1268a6b79e0d029a
Nadun Ranawaka
6996793f1268a6b79e0d029b
Jiasheng Gu
6996793f1268a6b79e0d029c
Yinzhen Xu
6996793f1268a6b79e0d029d
Guanzhi Wang
6996793f1268a6b79e0d029e
Fengyuan Hu
6996793f1268a6b79e0d029f
Avnish Narayan
6996793f1268a6b79e0d02a0
Johan Bjorck
6996793f1268a6b79e0d02a1
Jing Wang
6996793f1268a6b79e0d02a2
Gwanghyun Kim
6996793f1268a6b79e0d02a3
Dantong Niu
6996793f1268a6b79e0d02a4
Ruijie Zheng
6996793f1268a6b79e0d02a5
Yuqi Xie
6996793f1268a6b79e0d02a6
Jimmy Wu
6996793f1268a6b79e0d02a7
Qi Wang
6996793f1268a6b79e0d02a8
Ryan Julian
6996793f1268a6b79e0d02a9
Danfei Xu
6996793f1268a6b79e0d02aa
Yilun Du
6996793f1268a6b79e0d02ab
Yevgen Chebotar
6996793f1268a6b79e0d02ac
Scott Reed
6996793f1268a6b79e0d02ad
Jan Kautz
6996793f1268a6b79e0d02ae
Yuke Zhu
6996793f1268a6b79e0d02af
Linxi "Jim" Fan
6996793f1268a6b79e0d02b0
Joel Jang
World Action Models are Zero-shot Policies
State-of-the-art Vision-Language-Action (VLA) models excel at semantic generalization but struggle to generalize to unseen physical motions in novel environments. We introduce DreamZero, a World Action Model (WAM) built upon a pretrained video diffusion backbone. Unlike VLAs, WAMs learn physical dynamics by predicting future world states and actions, using video as a dense representation of how the world evolves. By jointly modeling video and action, DreamZero learns diverse skills effectively from heterogeneous robot data without relying on repetitive demonstrations. This results in over 2x improvement in generalization to new tasks and environments compared to state-of-the-art VLAs in real robot experiments. Crucially, through model and system optimizations, we enable a 14B autoregressive video diffusion model to perform real-time closed-loop control at 7Hz. Finally, we demonstrate two forms of cross-embodiment transfer: video-only demonstrations from other robots or humans yield a relative improvement of over 42% on unseen task performance with just 10-20 minutes of data. More surprisingly, DreamZero enables few-shot embodiment adaptation, transferring to a new embodiment with only 30 minutes of play data while retaining zero-shot generalization.
DreamZero is a World Action Model that leverages video diffusion to enable better generalization of physical motions across novel environments and embodiments compared to vision-language-action models.
World Action Model video diffusion video backbone physical dynamics autoregressive video diffusion model closed-loop control cross-embodiment transfer few-shot embodiment adaptation