Action Images: End-to-End Policy Learning via Multiview Video Generation
World action models (WAMs) have emerged as a promising direction for robot policy learning, as they can leverage powerful video backbones to model the future states. However, existing approaches often rely on separate action modules, or use action representations that are not pixel-grounded, making it difficult to fully exploit the pretrained knowledge of video models and limiting transfer across viewpoints and environments. In this work, we present Action Images, a unified world action model that formulates policy learning as multiview video generation. Instead of encoding control as low-dimensional tokens, we translate 7-DoF robot actions into interpretable action images: multi-view action videos that are grounded in 2D pixels and explicitly track robot-arm motion. This pixel-grounded action representation allows the video backbone itself to act as a zero-shot policy, without a separate policy head or action module. Beyond control, the same unified model supports video-action joint generation, action-conditioned video generation, and action labeling under a shared representation. On RLBench and real-world evaluations, our model achieves the strongest zero-shot success rates and improves video-action joint generation quality over prior video-space world models, suggesting that interpretable action images are a promising route to policy learning.
69d5c0f8b9b236b710ae813f
Haoyu Zhen
69d5c0f8b9b236b710ae8140
Zixian Gao
69d5c0f8b9b236b710ae8141
Qiao Sun
69d5c0f8b9b236b710ae8142
Yilin Zhao
69d5c0f8b9b236b710ae8143
Yuncong Yang
69d5c0f8b9b236b710ae8144
Yilun Du
69d5c0f8b9b236b710ae8145
Tsun-Hsuan Wang
69d5c0f8b9b236b710ae8146
Yi-Ling Qiao
69d5c0f8b9b236b710ae8147
Chuang Gan
Action Images: End-to-End Policy Learning via Multiview Video Generation
World action models (WAMs) have emerged as a promising direction for robot policy learning, as they can leverage powerful video backbones to model the future states. However, existing approaches often rely on separate action modules, or use action representations that are not pixel-grounded, making it difficult to fully exploit the pretrained knowledge of video models and limiting transfer across viewpoints and environments. In this work, we present Action Images, a unified world action model that formulates policy learning as multiview video generation. Instead of encoding control as low-dimensional tokens, we translate 7-DoF robot actions into interpretable action images: multi-view action videos that are grounded in 2D pixels and explicitly track robot-arm motion. This pixel-grounded action representation allows the video backbone itself to act as a zero-shot policy, without a separate policy head or action module. Beyond control, the same unified model supports video-action joint generation, action-conditioned video generation, and action labeling under a shared representation. On RLBench and real-world evaluations, our model achieves the strongest zero-shot success rates and improves video-action joint generation quality over prior video-space world models, suggesting that interpretable action images are a promising route to policy learning.
World action models that formulate policy learning as multiview video generation use pixel-grounded action images to enable zero-shot policy learning without separate action modules.
world action models video backbones future states robot policy learning action representations pixel-grounded multiview video generation zero-shot policy action images 7-DoF robot actions video-action joint generation action-conditioned video generation action labeling