ToolScope: An Agentic Framework for Vision-Guided and Long-Horizon Tool
  Use
Recently, large language models (LLMs) have demonstrated remarkable
problem-solving capabilities by autonomously integrating with external tools
for collaborative reasoning. However, due to the inherently complex and diverse
nature of multimodal information, enabling multimodal large language models
(MLLMs) to flexibly and efficiently utilize external tools during reasoning
remains an underexplored challenge. In this work, we introduce ToolScope, an
agentic framework designed to unify global planning with local multimodal
perception, adopting a specialized Perceive tool to mitigates visual context
degradation in long-horizon VQA task. ToolScope comprises three primary
components: the Global Navigator, the Agentic Executor, and the Response
Synthesizer. The Global Navigator functions as a "telescope", offering
high-level strategic guidance. The Agentic Executor operates iteratively to
augment MLLM with local perception through the integration of external
tools-Search, Code, and Perceive. Finally, the Response Synthesizer
consolidates and organizes the reasoning process into a coherent, user-friendly
output. We evaluate ToolScope on four VQA benchmarks across diverse domains,
including VQA 2.0, ScienceQA, MAT-Search and MathVista. It demonstrates strong
generalization capabilities, achieving an average performance improvement of up
to +6.69% across all datasets.
6908501d812eca10f9cc5efe
Mengjie Deng
claimed_verified
2025-11-03T20:51:52.660Z
65db23d1f386d08eb0d1cec5
/avatars/b495ec5b35b15fea245ef490b83d1856.svg
Mengjie Deng
MengjieDeng
user
MengjieDeng
6908501d812eca10f9cc5eff
Guanting Dong
claimed_verified
2026-02-27T16:47:14.138Z
61cd4b833dd34ba1985e0753
https://cdn-avatars.huggingface.co/v1/production/uploads/61cd4b833dd34ba1985e0753/BfHfrwotoMESpXZOHiIe4.png
KABI
dongguanting
user
dongguanting
6908501d812eca10f9cc5f00
Zhicheng Dou
claimed_verified
2025-11-06T11:48:51.627Z
66f0bf59e9d50ec57febf751
/avatars/be97941e60064e5dd806c6fe9db3c537.svg
Zhicheng Dou
douzc
user
douzc
ToolScope: An Agentic Framework for Vision-Guided and Long-Horizon Tool
  Use
Recently, large language models (LLMs) have demonstrated remarkable
problem-solving capabilities by autonomously integrating with external tools
for collaborative reasoning. However, due to the inherently complex and diverse
nature of multimodal information, enabling multimodal large language models
(MLLMs) to flexibly and efficiently utilize external tools during reasoning
remains an underexplored challenge. In this work, we introduce ToolScope, an
agentic framework designed to unify global planning with local multimodal
perception, adopting a specialized Perceive tool to mitigates visual context
degradation in long-horizon VQA task. ToolScope comprises three primary
components: the Global Navigator, the Agentic Executor, and the Response
Synthesizer. The Global Navigator functions as a "telescope", offering
high-level strategic guidance. The Agentic Executor operates iteratively to
augment MLLM with local perception through the integration of external
tools-Search, Code, and Perceive. Finally, the Response Synthesizer
consolidates and organizes the reasoning process into a coherent, user-friendly
output. We evaluate ToolScope on four VQA benchmarks across diverse domains,
including VQA 2.0, ScienceQA, MAT-Search and MathVista. It demonstrates strong
generalization capabilities, achieving an average performance improvement of up
to +6.69% across all datasets.
ToolScope, an agentic framework for multimodal large language models, enhances visual question answering by integrating external tools and achieving significant performance improvements across various benchmarks.
large language models multimodal large language models ToolScope Global Navigator Agentic Executor Response Synthesizer VQA visual context degradation long-horizon VQA Search Code Perceive VQA 2.0 ScienceQA MAT-Search MathVista