COOPER: A Unified Model for Cooperative Perception and Reasoning in Spatial Intelligence
Visual Spatial Reasoning is crucial for enabling Multimodal Large Language Models (MLLMs) to understand object properties and spatial relationships, yet current models still struggle with 3D-aware reasoning. Existing approaches typically enhance either perception, by augmenting RGB inputs with auxiliary modalities such as depth and segmentation, or reasoning, by training on spatial VQA datasets and applying reinforcement learning, and thus treat these two aspects in isolation. In this work, we investigate whether a unified MLLM can develop an intrinsic ability to enhance spatial perception and, through adaptive interleaved reasoning, achieve stronger spatial intelligence. We propose COOPER, a unified MLLM that leverages depth and segmentation as auxiliary modalities and is trained in two stages to acquire auxiliary modality generation and adaptive, interleaved reasoning capabilities. COOPER achieves an average 6.91\% improvement in spatial reasoning while maintaining general performance. Moreover, even a variant trained only for auxiliary modality generation attains a 7.92\% gain on distance and size estimation, suggesting that learning to generate auxiliary modalities helps internalize spatial knowledge and strengthen spatial understanding.
693530fb3962c926cf6805ef
Zefeng Zhang
claimed_verified
2025-12-08T08:30:43.319Z
642c2dcec3694d2b74565c48
/avatars/31243bb505f8c511ebd7492eaf3ea1a9.svg
zhangzef
Starrrrrry
user
Starrrrrry
693530fb3962c926cf6805f0
Xiangzhao Hao
claimed_verified
2025-12-08T08:56:20.670Z
64ec6025b96ff0e175728ac0
/avatars/a1acbe22b5e5105703e7912da2cfced2.svg
hxz
CUDAOUTOFMEMORY
user
CUDAOUTOFMEMORY
693530fb3962c926cf6805f1
Hengzhu Tang
693530fb3962c926cf6805f2
Zhenyu Zhang
693530fb3962c926cf6805f3
Jiawei Sheng
693530fb3962c926cf6805f4
Xiaodong Li
693530fb3962c926cf6805f5
Zhenyang Li
693530fb3962c926cf6805f6
Li Gao
693530fb3962c926cf6805f7
Daiting Shi
693530fb3962c926cf6805f8
Dawei Yin
693530fb3962c926cf6805f9
Tingwen Liu
COOPER: A Unified Model for Cooperative Perception and Reasoning in Spatial Intelligence
Visual Spatial Reasoning is crucial for enabling Multimodal Large Language Models (MLLMs) to understand object properties and spatial relationships, yet current models still struggle with 3D-aware reasoning. Existing approaches typically enhance either perception, by augmenting RGB inputs with auxiliary modalities such as depth and segmentation, or reasoning, by training on spatial VQA datasets and applying reinforcement learning, and thus treat these two aspects in isolation. In this work, we investigate whether a unified MLLM can develop an intrinsic ability to enhance spatial perception and, through adaptive interleaved reasoning, achieve stronger spatial intelligence. We propose COOPER, a unified MLLM that leverages depth and segmentation as auxiliary modalities and is trained in two stages to acquire auxiliary modality generation and adaptive, interleaved reasoning capabilities. COOPER achieves an average 6.91\% improvement in spatial reasoning while maintaining general performance. Moreover, even a variant trained only for auxiliary modality generation attains a 7.92\% gain on distance and size estimation, suggesting that learning to generate auxiliary modalities helps internalize spatial knowledge and strengthen spatial understanding.
A unified multimodal large language model (MLLM) that integrates depth and segmentation modalities enhances spatial reasoning and perception through adaptive interleaved reasoning, improving spatial intelligence and general performance.
Multimodal Large Language Models (MLLMs) visual spatial reasoning 3D-aware reasoning spatial VQA datasets reinforcement learning auxiliary modalities depth segmentation adaptive interleaved reasoning spatial perception spatial intelligence distance estimation size estimation