VideoVLA: Video Generators Can Be Generalizable Robot Manipulators
Generalization in robot manipulation is essential for deploying robots in open-world environments and advancing toward artificial general intelligence. While recent Vision-Language-Action (VLA) models leverage large pre-trained understanding models for perception and instruction following, their ability to generalize to novel tasks, objects, and settings remains limited. In this work, we present VideoVLA, a simple approach that explores the potential of transforming large video generation models into robotic VLA manipulators. Given a language instruction and an image, VideoVLA predicts an action sequence as well as the future visual outcomes. Built on a multi-modal Diffusion Transformer, VideoVLA jointly models video, language, and action modalities, using pre-trained video generative models for joint visual and action forecasting. Our experiments show that high-quality imagined futures correlate with reliable action predictions and task success, highlighting the importance of visual imagination in manipulation. VideoVLA demonstrates strong generalization, including imitating other embodiments' skills and handling novel objects. This dual-prediction strategy - forecasting both actions and their visual consequences - explores a paradigm shift in robot learning and unlocks generalization capabilities in manipulation systems.
69379eec19d912300c34a303
Yichao Shen
admin_assigned
2025-12-09T11:52:21.421Z
60b30b311d57c8cd7204458f
/avatars/9debe5d788096fa02119fdbf63437bc0.svg
Yichao Shen
ys3197
user
ys3197
69379eec19d912300c34a304
Fangyun Wei
69379eec19d912300c34a305
Zhiying Du
69379eec19d912300c34a306
Yaobo Liang
69379eec19d912300c34a307
Yan Lu
69379eec19d912300c34a308
Jiaolong Yang
admin_assigned
2025-12-09T11:52:48.871Z
66a59f6b94d2b190a22b7c80
/avatars/9b59c0bb3e171d5246e3cb3aa8377a77.svg
jiaolong yang
godjiaolongge
user
godjiaolongge
69379eec19d912300c34a309
Nanning Zheng
69379eec19d912300c34a30a
Baining Guo
VideoVLA: Video Generators Can Be Generalizable Robot Manipulators
Generalization in robot manipulation is essential for deploying robots in open-world environments and advancing toward artificial general intelligence. While recent Vision-Language-Action (VLA) models leverage large pre-trained understanding models for perception and instruction following, their ability to generalize to novel tasks, objects, and settings remains limited. In this work, we present VideoVLA, a simple approach that explores the potential of transforming large video generation models into robotic VLA manipulators. Given a language instruction and an image, VideoVLA predicts an action sequence as well as the future visual outcomes. Built on a multi-modal Diffusion Transformer, VideoVLA jointly models video, language, and action modalities, using pre-trained video generative models for joint visual and action forecasting. Our experiments show that high-quality imagined futures correlate with reliable action predictions and task success, highlighting the importance of visual imagination in manipulation. VideoVLA demonstrates strong generalization, including imitating other embodiments' skills and handling novel objects. This dual-prediction strategy - forecasting both actions and their visual consequences - explores a paradigm shift in robot learning and unlocks generalization capabilities in manipulation systems.
VideoVLA uses a multi-modal Diffusion Transformer to predict actions and visual outcomes from language and image inputs, enabling strong generalization in robotic manipulation tasks.
VideoVLA multi-modal Diffusion Transformer visual imagination manipulation action sequence future visual outcomes generalization imitation novel objects