Toward Ambulatory Vision: Learning Visually-Grounded Active View Selection
Vision Language Models (VLMs) excel at visual question answering (VQA) but remain limited to snapshot vision, reasoning from static images. In contrast, embodied agents require ambulatory vision, actively moving to obtain more informative views. We introduce Visually Grounded Active View Selection (VG-AVS), a task that selects the most informative next viewpoint using only the visual information in the current image, without relying on scene memory or external knowledge. To support this task, we construct a synthetic dataset with automatically generated paired query-target views and question-answer prompts. We also propose a framework that fine-tunes pretrained VLMs through supervised fine-tuning (SFT) followed by RL-based policy optimization. Our approach achieves strong question answering performance based on viewpoint selection and generalizes robustly to unseen synthetic and real scenes. Furthermore, incorporating our learned VG-AVS framework into existing scene-exploration-based EQA systems improves downstream question-answering accuracy.
6940dba565f1e24a117805f7
Juil Koo
6940dba565f1e24a117805f8
Daehyeon Choi
claimed_verified
2025-12-16T09:38:47.490Z
6616702547ea6347974667e5
/avatars/6bf95e50ba19df163ef89867ed63fecc.svg
Daehyeon Choi
daehyeonchoi
user
daehyeonchoi
6940dba565f1e24a117805f9
Sangwoo Youn
claimed_verified
2025-12-16T09:37:42.523Z
686269b24c9139a755a319dd
/avatars/9533d1c4ef685db3097a8cd65092f269.svg
Sangwoo Youn
sang-w00
user
sang-w00
6940dba565f1e24a117805fa
Phillip Y. Lee
claimed_verified
2025-12-16T09:37:44.580Z
6342796a0875f2c99cfd313b
/avatars/98575092404c4197b20c929a6499a015.svg
Yuseung "Phillip" Lee
phillipinseoul
user
phillipinseoul
6940dba565f1e24a117805fb
Minhyuk Sung
Toward Ambulatory Vision: Learning Visually-Grounded Active View Selection
Vision Language Models (VLMs) excel at visual question answering (VQA) but remain limited to snapshot vision, reasoning from static images. In contrast, embodied agents require ambulatory vision, actively moving to obtain more informative views. We introduce Visually Grounded Active View Selection (VG-AVS), a task that selects the most informative next viewpoint using only the visual information in the current image, without relying on scene memory or external knowledge. To support this task, we construct a synthetic dataset with automatically generated paired query-target views and question-answer prompts. We also propose a framework that fine-tunes pretrained VLMs through supervised fine-tuning (SFT) followed by RL-based policy optimization. Our approach achieves strong question answering performance based on viewpoint selection and generalizes robustly to unseen synthetic and real scenes. Furthermore, incorporating our learned VG-AVS framework into existing scene-exploration-based EQA systems improves downstream question-answering accuracy.
VG-AVS, a task and framework fine-tunes VLMs to select the most informative next viewpoint for visual question answering, enhancing performance and generalization.
Vision Language Models VLMs visual question answering VQA snapshot vision ambulatory vision Visually Grounded Active View Selection VG-AVS synthetic dataset supervised fine-tuning SFT RL-based policy optimization EQA systems