AdaReasoner: Dynamic Tool Orchestration for Iterative Visual Reasoning
When humans face problems beyond their immediate capabilities, they rely on tools, providing a promising paradigm for improving visual reasoning in multimodal large language models (MLLMs). Effective reasoning, therefore, hinges on knowing which tools to use, when to invoke them, and how to compose them over multiple steps, even when faced with new tools or new tasks. We introduce AdaReasoner, a family of multimodal models that learn tool use as a general reasoning skill rather than as tool-specific or explicitly supervised behavior. AdaReasoner is enabled by (i) a scalable data curation pipeline exposing models to long-horizon, multi-step tool interactions; (ii) Tool-GRPO, a reinforcement learning algorithm that optimizes tool selection and sequencing based on end-task success; and (iii) an adaptive learning mechanism that dynamically regulates tool usage. Together, these components allow models to infer tool utility from task context and intermediate outcomes, enabling coordination of multiple tools and generalization to unseen tools. Empirically, AdaReasoner exhibits strong tool-adaptive and generalization behaviors: it autonomously adopts beneficial tools, suppresses irrelevant ones, and adjusts tool usage frequency based on task demands, despite never being explicitly trained to do so. These capabilities translate into state-of-the-art performance across challenging benchmarks, improving the 7B base model by +24.9\% on average and surpassing strong proprietary systems such as GPT-5 on multiple tasks, including VSP and Jigsaw.
6978a169026bdf0473117088
Mingyang Song
claimed_verified
2026-01-28T11:29:30.581Z
66aca01e33f6b27979856f6f
https://cdn-avatars.huggingface.co/v1/production/uploads/66aca01e33f6b27979856f6f/IyOxv89TudwscGH7tdue3.jpeg
Mingyang Song
hitsmy
user
hitsmy
6978a169026bdf0473117089
Haoyu Sun
claimed_verified
2026-01-28T11:29:26.154Z
63a2a51ef30c464227924fc6
/avatars/e109e85abd25b97bb29dbbe007119e34.svg
Haoyu Sun
Mikivis
user
Mikivis
6978a169026bdf047311708a
Jiawei Gu
claimed_verified
2026-01-28T11:29:28.392Z
645b4819f9d4ec91fdd54852
/avatars/e12efb8e030688a0afcc72176b453fb3.svg
Jiawei Gu
kuvvi
user
kuvvi
6978a169026bdf047311708b
Linjie Li
6978a169026bdf047311708c
Luxin Xu
6978a169026bdf047311708d
Ranjay Krishna
6978a169026bdf047311708e
Yu Cheng
AdaReasoner: Dynamic Tool Orchestration for Iterative Visual Reasoning
When humans face problems beyond their immediate capabilities, they rely on tools, providing a promising paradigm for improving visual reasoning in multimodal large language models (MLLMs). Effective reasoning, therefore, hinges on knowing which tools to use, when to invoke them, and how to compose them over multiple steps, even when faced with new tools or new tasks. We introduce AdaReasoner, a family of multimodal models that learn tool use as a general reasoning skill rather than as tool-specific or explicitly supervised behavior. AdaReasoner is enabled by (i) a scalable data curation pipeline exposing models to long-horizon, multi-step tool interactions; (ii) Tool-GRPO, a reinforcement learning algorithm that optimizes tool selection and sequencing based on end-task success; and (iii) an adaptive learning mechanism that dynamically regulates tool usage. Together, these components allow models to infer tool utility from task context and intermediate outcomes, enabling coordination of multiple tools and generalization to unseen tools. Empirically, AdaReasoner exhibits strong tool-adaptive and generalization behaviors: it autonomously adopts beneficial tools, suppresses irrelevant ones, and adjusts tool usage frequency based on task demands, despite never being explicitly trained to do so. These capabilities translate into state-of-the-art performance across challenging benchmarks, improving the 7B base model by +24.9\% on average and surpassing strong proprietary systems such as GPT-5 on multiple tasks, including VSP and Jigsaw.
AdaReasoner enables multimodal models to learn tool usage as a general reasoning skill through scalable data curation, reinforcement learning for tool selection, and adaptive learning mechanisms that improve performance on complex visual reasoning tasks.
multimodal large language models tool use reinforcement learning end-task success adaptive learning mechanism visual reasoning multimodal models tool selection tool sequencing long-horizon interactions