Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMs
While recent vision-language models (VLMs) demonstrate strong image understanding, their ability to "think with images", i.e., to reason through multi-step visual interactions, remains limited. We introduce VISTA-Gym, a scalable training environment for incentivizing tool-integrated visual reasoning capabilities in VLMs. VISTA-Gym unifies diverse real-world multimodal reasoning tasks (7 tasks from 13 datasets in total) with a standardized interface for visual tools (e.g., grounding, parsing), executable interaction loops, verifiable feedback signals, and efficient trajectory logging, enabling visual agentic reinforcement learning at scale. While recent VLMs exhibit strong text-only reasoning, both proprietary and open-source models still struggle with tool selection, invocation, and coordination. With VISTA-Gym, we train VISTA-R1 to interleave tool-use with agentic reasoning via multi-turn trajectory sampling and end-to-end reinforcement learning. Extensive experiments across 11 public reasoning-intensive VQA benchmarks show that VISTA-R1-8B outperforms state-of-the-art baselines with similar sizes by 9.51%-18.72%, demonstrating VISTA-Gym as an effective training ground to unlock the tool-integrated reasoning capabilities for VLMs.
69268437243b2216fb75cab7
Meng Lu
69268437243b2216fb75cab8
Ran Xu
claimed_verified
2026-02-04T12:35:07.516Z
6621dd7623518bf2898788da
/avatars/837aca71471a19ea80ae55d619781c5a.svg
Ran Xu
ritaranx
user
ritaranx
69268437243b2216fb75cab9
Yi Fang
69268437243b2216fb75caba
Wenxuan Zhang
69268437243b2216fb75cabb
Yue Yu
69268437243b2216fb75cabc
Gaurav Srivastava
69268437243b2216fb75cabd
Yuchen Zhuang
69268437243b2216fb75cabe
Mohamed Elhoseiny
69268437243b2216fb75cabf
Charles Fleming
69268437243b2216fb75cac0
Carl Yang
69268437243b2216fb75cac1
Zhengzhong Tu
69268437243b2216fb75cac2
Yang Xie
69268437243b2216fb75cac3
Guanghua Xiao
69268437243b2216fb75cac4
Hanrui Wang
69268437243b2216fb75cac5
Di Jin
69268437243b2216fb75cac6
Wenqi Shi
claimed_verified
2025-11-26T09:27:25.162Z
65cae89119683f9817c049ea
https://cdn-avatars.huggingface.co/v1/production/uploads/65cae89119683f9817c049ea/A0XxjmaJldu28JhFvWmpP.jpeg
Wenqi Shi
wshi83
user
wshi83
69268437243b2216fb75cac7
Xuan Wang
Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMs
While recent vision-language models (VLMs) demonstrate strong image understanding, their ability to "think with images", i.e., to reason through multi-step visual interactions, remains limited. We introduce VISTA-Gym, a scalable training environment for incentivizing tool-integrated visual reasoning capabilities in VLMs. VISTA-Gym unifies diverse real-world multimodal reasoning tasks (7 tasks from 13 datasets in total) with a standardized interface for visual tools (e.g., grounding, parsing), executable interaction loops, verifiable feedback signals, and efficient trajectory logging, enabling visual agentic reinforcement learning at scale. While recent VLMs exhibit strong text-only reasoning, both proprietary and open-source models still struggle with tool selection, invocation, and coordination. With VISTA-Gym, we train VISTA-R1 to interleave tool-use with agentic reasoning via multi-turn trajectory sampling and end-to-end reinforcement learning. Extensive experiments across 11 public reasoning-intensive VQA benchmarks show that VISTA-R1-8B outperforms state-of-the-art baselines with similar sizes by 9.51%-18.72%, demonstrating VISTA-Gym as an effective training ground to unlock the tool-integrated reasoning capabilities for VLMs.
VISTA-Gym enhances vision-language models' tool-integrated visual reasoning through a scalable training environment with diverse multimodal tasks and reinforcement learning.
vision-language models VLMs VISTA-Gym multimodal reasoning visual tools executable interaction loops verifiable feedback signals trajectory logging visual agentic reinforcement learning tool selection tool invocation tool coordination multi-turn trajectory sampling end-to-end reinforcement learning VQA benchmarks VISTA-R1-8B