10 Open Challenges Steering the Future of Vision-Language-Action Models
Due to their ability of follow natural language instructions,
vision-language-action (VLA) models are increasingly prevalent in the embodied
AI arena, following the widespread success of their precursors -- LLMs and
VLMs. In this paper, we discuss 10 principal milestones in the ongoing
development of VLA models -- multimodality, reasoning, data, evaluation,
cross-robot action generalization, efficiency, whole-body coordination, safety,
agents, and coordination with humans. Furthermore, we discuss the emerging
trends of using spatial understanding, modeling world dynamics, post training,
and data synthesis -- all aiming to reach these milestones. Through these
discussions, we hope to bring attention to the research avenues that may
accelerate the development of VLA models into wider acceptability.
6912db4ea644ba07c499c80e
Soujanya Poria
6912db4ea644ba07c499c80f
Navonil Majumder
6912db4ea644ba07c499c810
Chia-Yu Hung
6912db4ea644ba07c499c811
Amir Ali Bagherzadeh
6912db4ea644ba07c499c812
Chuan Li
6912db4ea644ba07c499c813
Kenneth Kwok
6912db4ea644ba07c499c814
Ziwei Wang
6912db4ea644ba07c499c815
Cheston Tan
6912db4ea644ba07c499c816
Jiajun Wu
6912db4ea644ba07c499c817
David Hsu
10 Open Challenges Steering the Future of Vision-Language-Action Models
Due to their ability of follow natural language instructions,
vision-language-action (VLA) models are increasingly prevalent in the embodied
AI arena, following the widespread success of their precursors -- LLMs and
VLMs. In this paper, we discuss 10 principal milestones in the ongoing
development of VLA models -- multimodality, reasoning, data, evaluation,
cross-robot action generalization, efficiency, whole-body coordination, safety,
agents, and coordination with humans. Furthermore, we discuss the emerging
trends of using spatial understanding, modeling world dynamics, post training,
and data synthesis -- all aiming to reach these milestones. Through these
discussions, we hope to bring attention to the research avenues that may
accelerate the development of VLA models into wider acceptability.
VLA models, combining vision, language, and action, are advancing through milestones like multimodality, reasoning, and safety, with trends focusing on spatial understanding and human coordination.
vision-language-action (VLA) models embodied AI multimodality reasoning data evaluation cross-robot action generalization efficiency whole-body coordination safety agents coordination with humans spatial understanding modeling world dynamics post training data synthesis