SPARC: Separating Perception And Reasoning Circuits for Test-time Scaling of VLMs
Despite recent successes, test-time scaling - i.e., dynamically expanding the token budget during inference as needed - remains brittle for vision-language models (VLMs): unstructured chains-of-thought about images entangle perception and reasoning, leading to long, disorganized contexts where small perceptual mistakes may cascade into completely wrong answers. Moreover, expensive reinforcement learning with hand-crafted rewards is required to achieve good performance. Here, we introduce SPARC (Separating Perception And Reasoning Circuits), a modular framework that explicitly decouples visual perception from reasoning. Inspired by sequential sensory-to-cognitive processing in the brain, SPARC implements a two-stage pipeline where the model first performs explicit visual search to localize question-relevant regions, then conditions its reasoning on those regions to produce the final answer. This separation enables independent test-time scaling with asymmetric compute allocation (e.g., prioritizing perceptual processing under distribution shift), supports selective optimization (e.g., improving the perceptual stage alone when it is the bottleneck for end-to-end performance), and accommodates compressed contexts by running global search at lower image resolutions and allocating high-resolution processing only to selected regions, thereby reducing total visual tokens count and compute. Across challenging visual reasoning benchmarks, SPARC outperforms monolithic baselines and strong visual-grounding approaches. For instance, SPARC improves the accuracy of Qwen3VL-4B on the V^* VQA benchmark by 6.7 percentage points, and it surpasses "thinking with images" by 4.6 points on a challenging OOD task despite requiring a 200times lower token budget.
698a6aeb1b2dc6b37d61ae97
Niccolo Avogaro
698a6aeb1b2dc6b37d61ae98
Nayanika Debnath
698a6aeb1b2dc6b37d61ae99
Li Mi
698a6aeb1b2dc6b37d61ae9a
Thomas Frick
698a6aeb1b2dc6b37d61ae9b
Junling Wang
698a6aeb1b2dc6b37d61ae9c
Zexue He
698a6aeb1b2dc6b37d61ae9d
Hang Hua
claimed_verified
2026-02-10T09:07:31.198Z
639f8277beb95d698de007dd
/avatars/57f223ccd9d3cb03166ccf0e41361c58.svg
HangHua
hhua2
user
hhua2
698a6aeb1b2dc6b37d61ae9e
Konrad Schindler
698a6aeb1b2dc6b37d61ae9f
Mattia Rigotti
SPARC: Separating Perception And Reasoning Circuits for Test-time Scaling of VLMs
Despite recent successes, test-time scaling - i.e., dynamically expanding the token budget during inference as needed - remains brittle for vision-language models (VLMs): unstructured chains-of-thought about images entangle perception and reasoning, leading to long, disorganized contexts where small perceptual mistakes may cascade into completely wrong answers. Moreover, expensive reinforcement learning with hand-crafted rewards is required to achieve good performance. Here, we introduce SPARC (Separating Perception And Reasoning Circuits), a modular framework that explicitly decouples visual perception from reasoning. Inspired by sequential sensory-to-cognitive processing in the brain, SPARC implements a two-stage pipeline where the model first performs explicit visual search to localize question-relevant regions, then conditions its reasoning on those regions to produce the final answer. This separation enables independent test-time scaling with asymmetric compute allocation (e.g., prioritizing perceptual processing under distribution shift), supports selective optimization (e.g., improving the perceptual stage alone when it is the bottleneck for end-to-end performance), and accommodates compressed contexts by running global search at lower image resolutions and allocating high-resolution processing only to selected regions, thereby reducing total visual tokens count and compute. Across challenging visual reasoning benchmarks, SPARC outperforms monolithic baselines and strong visual-grounding approaches. For instance, SPARC improves the accuracy of Qwen3VL-4B on the V^* VQA benchmark by 6.7 percentage points, and it surpasses "thinking with images" by 4.6 points on a challenging OOD task despite requiring a 200times lower token budget.
SPARC is a modular framework that decouples visual perception from reasoning in vision-language models, enabling efficient test-time scaling through targeted compute allocation and improved performance on visual reasoning tasks.
vision-language models test-time scaling visual perception reasoning visual search sequential sensory-to-cognitive processing two-stage pipeline visual grounding asymmetric compute allocation token budget visual reasoning benchmarks Qwen3VL-4B V* VQA benchmark out-of-distribution tasks