CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimization
Agentic vision-language models are increasingly trained to "think with images" by calling image operations. However, we show that high final-answer accuracy often hides unfaithful visual reasoning: models may invoke tools on irrelevant regions or ignore tool outputs entirely, yet still guess the correct answer. In this work, we first propose a faithfulness evaluation protocol that measures whether intermediate visual tool outputs (e.g., crops) actually contain the queried evidence. This reveals that recent visual agents achieve high final-answer accuracy but exhibit low rates of faithful tool-use on visual search benchmarks. We then introduce CodeV, a code-based visual agent trained with Tool-Aware Policy Optimization (TAPO). TAPO is a process-level RL framework that augments GRPO with dense rewards defined directly on visual tool inputs and outputs, rather than on chain-of-thought tokens, making supervision easier to verify and less susceptible to reward hacking. CodeV represents visual tools as executable Python code, and TAPO assigns step-wise rewards based solely on the question and tool output, encouraging both necessary and evidence-consistent tool use. In a two-stage SFT+RL pipeline, CodeV achieves competitive or superior accuracy while substantially increasing faithful tool-use rates on related visual search benchmarks. Beyond visual search, CodeV attains strong performance on a range of multimodal reasoning and math benchmarks, suggesting that explicitly supervising intermediate tool behavior is crucial for building trustworthy, agentic visual reasoning systems.
692fe32326742347f61dadd3
Xinhai Hou
claimed_verified
2025-12-03T09:13:12.152Z
6633dedc9c2b8424a93b715c
https://cdn-avatars.huggingface.co/v1/production/uploads/6633dedc9c2b8424a93b715c/4yosLG5H0DeVw_6lV6IiQ.jpeg
Xinhai Hou
RenlyH
user
RenlyH
692fe32326742347f61dadd4
Shaoyuan Xu
692fe32326742347f61dadd5
Manan Biyani
692fe32326742347f61dadd6
Mayan Li
692fe32326742347f61dadd7
Jia Liu
692fe32326742347f61dadd8
Todd C. Hollon
692fe32326742347f61dadd9
Bryan Wang
CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimization
Agentic vision-language models are increasingly trained to "think with images" by calling image operations. However, we show that high final-answer accuracy often hides unfaithful visual reasoning: models may invoke tools on irrelevant regions or ignore tool outputs entirely, yet still guess the correct answer. In this work, we first propose a faithfulness evaluation protocol that measures whether intermediate visual tool outputs (e.g., crops) actually contain the queried evidence. This reveals that recent visual agents achieve high final-answer accuracy but exhibit low rates of faithful tool-use on visual search benchmarks. We then introduce CodeV, a code-based visual agent trained with Tool-Aware Policy Optimization (TAPO). TAPO is a process-level RL framework that augments GRPO with dense rewards defined directly on visual tool inputs and outputs, rather than on chain-of-thought tokens, making supervision easier to verify and less susceptible to reward hacking. CodeV represents visual tools as executable Python code, and TAPO assigns step-wise rewards based solely on the question and tool output, encouraging both necessary and evidence-consistent tool use. In a two-stage SFT+RL pipeline, CodeV achieves competitive or superior accuracy while substantially increasing faithful tool-use rates on related visual search benchmarks. Beyond visual search, CodeV attains strong performance on a range of multimodal reasoning and math benchmarks, suggesting that explicitly supervising intermediate tool behavior is crucial for building trustworthy, agentic visual reasoning systems.
CodeV, a code-based visual agent trained with Tool-Aware Policy Optimization (TAPO), improves faithful tool use and accuracy in visual and multimodal reasoning tasks.
Agentic vision-language models image operations faithfulness evaluation protocol intermediate visual tool outputs Tool-Aware Policy Optimization (TAPO) process-level RL GRPO dense rewards visual tool inputs visual tool outputs step-wise rewards SFT+RL pipeline multimodal reasoning math benchmarks