Look Before Acting: Enhancing Vision Foundation Representations for Vision-Language-Action Models
Vision-Language-Action (VLA) models have recently emerged as a promising paradigm for robotic manipulation, in which reliable action prediction critically depends on accurately interpreting and integrating visual observations conditioned on language instructions. Although recent works have sought to enhance the visual capabilities of VLA models, most approaches treat the LLM backbone as a black box, providing limited insight into how visual information is grounded into action generation. Therefore, we perform a systematic analysis of multiple VLA models across different action-generation paradigms and observe that sensitivity to visual tokens progressively decreases in deeper layers during action generation. Motivated by this observation, we propose DeepVision-VLA, built on a Vision-Language Mixture-of-Transformers (VL-MoT) framework. This framework enables shared attention between the vision foundation model and the VLA backbone, injecting multi-level visual features from the vision expert into deeper layers of the VLA backbone to enhance visual representations for precise and complex manipulation. In addition, we introduce Action-Guided Visual Pruning (AGVP), which leverages shallow-layer attention to prune irrelevant visual tokens while preserving task-relevant ones, reinforcing critical visual cues for manipulation with minimal computational overhead. DeepVision-VLA outperforms prior state-of-the-art methods by 9.0\% and 7.5\% on simulated and real-world tasks, respectively, providing new insights for the design of visually enhanced VLA models.
69b918c46b3f36c249fbc639
Yulin Luo
claimed_verified
2026-03-17T17:34:34.916Z
64e436a99ec4cf5000b30c14
/avatars/457b5950dd3bc0611ce414e84c71ce98.svg
lin Luo
lyl010221-pku
user
lyl010221-pku
69b918c46b3f36c249fbc63a
Hao Chen
69b918c46b3f36c249fbc63b
Zhuangzhe Wu
claimed_verified
2026-03-17T09:36:46.776Z
6830a4d2b4232fb67ce33334
/avatars/25d8e5ec9b2df42ea4edcc7ce919e35f.svg
zhuangzhe wu
zhuangzhe1229
user
zhuangzhe1229
69b918c46b3f36c249fbc63c
Bowen Sui
69b918c46b3f36c249fbc63d
Jiaming Liu
69b918c46b3f36c249fbc63e
Chenyang Gu
claimed_verified
2026-03-17T09:36:48.955Z
668d4f97fecd7e04837f65d3
https://cdn-avatars.huggingface.co/v1/production/uploads/668d4f97fecd7e04837f65d3/dF0NxmelIbWR7N8S4p5Pe.jpeg
Gu Chenyang
GayStarc
user
GayStarc
69b918c46b3f36c249fbc63f
Zhuoyang Liu
69b918c46b3f36c249fbc640
Qiuxuan Feng
69b918c46b3f36c249fbc641
Jiale Yu
69b918c46b3f36c249fbc642
Shuo Gu
69b918c46b3f36c249fbc643
Peng Jia
69b918c46b3f36c249fbc644
Pheng-Ann Heng
69b918c46b3f36c249fbc645
Shanghang Zhang
Look Before Acting: Enhancing Vision Foundation Representations for Vision-Language-Action Models
Vision-Language-Action (VLA) models have recently emerged as a promising paradigm for robotic manipulation, in which reliable action prediction critically depends on accurately interpreting and integrating visual observations conditioned on language instructions. Although recent works have sought to enhance the visual capabilities of VLA models, most approaches treat the LLM backbone as a black box, providing limited insight into how visual information is grounded into action generation. Therefore, we perform a systematic analysis of multiple VLA models across different action-generation paradigms and observe that sensitivity to visual tokens progressively decreases in deeper layers during action generation. Motivated by this observation, we propose DeepVision-VLA, built on a Vision-Language Mixture-of-Transformers (VL-MoT) framework. This framework enables shared attention between the vision foundation model and the VLA backbone, injecting multi-level visual features from the vision expert into deeper layers of the VLA backbone to enhance visual representations for precise and complex manipulation. In addition, we introduce Action-Guided Visual Pruning (AGVP), which leverages shallow-layer attention to prune irrelevant visual tokens while preserving task-relevant ones, reinforcing critical visual cues for manipulation with minimal computational overhead. DeepVision-VLA outperforms prior state-of-the-art methods by 9.0\% and 7.5\% on simulated and real-world tasks, respectively, providing new insights for the design of visually enhanced VLA models.