Look Where It Matters: High-Resolution Crops Retrieval for Efficient VLMs
Vision-language models (VLMs) typically process images at a native high-resolution, forcing a trade-off between accuracy and computational efficiency: high-resolution inputs capture fine details but incur significant computational costs, while low-resolution inputs advocate for efficiency, they potentially miss critical visual information, like small text. We present AwaRes, a spatial-on-demand framework that resolves this accuracy-efficiency trade-off by operating on a low-resolution global view and using tool-calling to retrieve only high-resolution segments needed for a given query. We construct supervised data automatically: a judge compares low- vs.\ high-resolution answers to label whether cropping is needed, and an oracle grounding model localizes the evidence for the correct answer, which we map to a discrete crop set to form multi-turn tool-use trajectories. We train our framework with cold-start SFT followed by multi-turn GRPO with a composite reward that combines semantic answer correctness with explicit crop-cost penalties. Project page: https://nimrodshabtay.github.io/AwaRes
69bf9ef83fd3fef6237a65b5
Nimrod Shabtay
69bf9ef83fd3fef6237a65b6
Moshe Kimhi
claimed_verified
2026-03-24T13:20:56.922Z
63926124526c29d5b5011374
https://cdn-avatars.huggingface.co/v1/production/uploads/63926124526c29d5b5011374/TD0J-6gHZTXF8fNr29UT7.png
Moshe kimhi
Kimhi
user
Kimhi
69bf9ef83fd3fef6237a65b7
Artem Spector
69bf9ef83fd3fef6237a65b8
Sivan Haray
69bf9ef83fd3fef6237a65b9
Ehud Rivlin
69bf9ef83fd3fef6237a65ba
Chaim Baskin
69bf9ef83fd3fef6237a65bb
Raja Giryes
69bf9ef83fd3fef6237a65bc
Eli Schwartz
Look Where It Matters: High-Resolution Crops Retrieval for Efficient VLMs
Vision-language models (VLMs) typically process images at a native high-resolution, forcing a trade-off between accuracy and computational efficiency: high-resolution inputs capture fine details but incur significant computational costs, while low-resolution inputs advocate for efficiency, they potentially miss critical visual information, like small text. We present AwaRes, a spatial-on-demand framework that resolves this accuracy-efficiency trade-off by operating on a low-resolution global view and using tool-calling to retrieve only high-resolution segments needed for a given query. We construct supervised data automatically: a judge compares low- vs.\ high-resolution answers to label whether cropping is needed, and an oracle grounding model localizes the evidence for the correct answer, which we map to a discrete crop set to form multi-turn tool-use trajectories. We train our framework with cold-start SFT followed by multi-turn GRPO with a composite reward that combines semantic answer correctness with explicit crop-cost penalties. Project page: https://nimrodshabtay.github.io/AwaRes
AwaRes is a spatial-on-demand framework for vision-language models that dynamically retrieves high-resolution image segments based on query needs, using tool-calling and multi-turn reinforcement learning with composite rewards.
vision-language models spatial-on-demand framework tool-calling multi-turn GRPO composite reward cold-start SFT supervised data construction judge model oracle grounding model discrete crop set multi-turn tool-use trajectories