ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal
  Generation
Unified multimodal models (UMMs) have emerged as a powerful paradigm for
seamlessly unifying text and image understanding and generation. However,
prevailing evaluations treat these abilities in isolation, such that tasks with
multimodal inputs and outputs are scored primarily through unimodal reasoning,
i.e., textual benchmarks emphasize language-based reasoning, while visual
benchmarks emphasize reasoning outcomes manifested in the pixels. We introduce
ROVER to address this pressing need to test reciprocal cross-modal reasoning,
the use of one modality to guide, verify, or refine outputs in the other, an
ability central to the vision of unified multimodal intelligence. ROVER is a
human-annotated benchmark that explicitly targets reciprocal cross-modal
reasoning, which contains 1312 tasks grounded in 1876 images, spanning two
complementary settings. Verbally-augmented reasoning for visual generation
evaluates whether models can use verbal prompts and reasoning chains to guide
faithful image synthesis. Visually-augmented reasoning for verbal generation
evaluates whether models can generate intermediate visualizations that
strengthen their own reasoning processes for question answering. Experiments on
17 unified models reveal two key findings: (i) Cross-modal reasoning determines
visual generation quality, with interleaved models significantly outperforming
non-interleaved ones; notably, combining strong unimodal models fails to
achieve comparable reasoning. (ii) Models show dissociation between physical
and symbolic reasoning: they succeed at interpreting perceptual concepts
literally but fail to construct visual abstractions for symbolic tasks, where
faulty reasoning harms performance. These results highlight reciprocal
cross-modal reasoning as a critical frontier for enabling true omnimodal
generation.
690986f4812eca10f9cc6384
Yongyuan Liang
claimed_verified
2025-11-18T14:09:57.218Z
6646d5819bb34d2b6b7455d3
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/JFH3ZTPvlaVSg4RJJBb6L.jpeg
Yongyuan Liang
cheryyunl
user
cheryyunl
690986f4812eca10f9cc6385
Wei Chow
690986f4812eca10f9cc6386
Feng Li
690986f4812eca10f9cc6387
Ziqiao Ma
claimed_verified
2025-11-04T20:24:35.876Z
630cfc45b66f088d547b2768
https://cdn-avatars.huggingface.co/v1/production/uploads/630cfc45b66f088d547b2768/9-dMts2xFVbPmPHJGGqBx.png
Martin Ziqiao Ma
marstin
user
marstin
690986f4812eca10f9cc6388
Xiyao Wang
claimed_verified
2025-11-04T19:50:50.234Z
655fed9fdef5905d38b84af3
/avatars/2cda4182dfd11a1e94743639e62328ea.svg
Xiyao Wang
russwang
user
russwang
690986f4812eca10f9cc6389
Jiageng Mao
690986f4812eca10f9cc638a
Jiuhai Chen
690986f4812eca10f9cc638b
Jiatao Gu
claimed_verified
2025-12-10T09:10:47.157Z
6164e72d73996c363c52e66d
https://cdn-avatars.huggingface.co/v1/production/uploads/1634002684894-noauth.png
Jiatao Gu
thomagram
user
thomagram
690986f4812eca10f9cc638c
Yue Wang
690986f4812eca10f9cc638d
Furong Huang
ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal
  Generation
Unified multimodal models (UMMs) have emerged as a powerful paradigm for
seamlessly unifying text and image understanding and generation. However,
prevailing evaluations treat these abilities in isolation, such that tasks with
multimodal inputs and outputs are scored primarily through unimodal reasoning,
i.e., textual benchmarks emphasize language-based reasoning, while visual
benchmarks emphasize reasoning outcomes manifested in the pixels. We introduce
ROVER to address this pressing need to test reciprocal cross-modal reasoning,
the use of one modality to guide, verify, or refine outputs in the other, an
ability central to the vision of unified multimodal intelligence. ROVER is a
human-annotated benchmark that explicitly targets reciprocal cross-modal
reasoning, which contains 1312 tasks grounded in 1876 images, spanning two
complementary settings. Verbally-augmented reasoning for visual generation
evaluates whether models can use verbal prompts and reasoning chains to guide
faithful image synthesis. Visually-augmented reasoning for verbal generation
evaluates whether models can generate intermediate visualizations that
strengthen their own reasoning processes for question answering. Experiments on
17 unified models reveal two key findings: (i) Cross-modal reasoning determines
visual generation quality, with interleaved models significantly outperforming
non-interleaved ones; notably, combining strong unimodal models fails to
achieve comparable reasoning. (ii) Models show dissociation between physical
and symbolic reasoning: they succeed at interpreting perceptual concepts
literally but fail to construct visual abstractions for symbolic tasks, where
faulty reasoning harms performance. These results highlight reciprocal
cross-modal reasoning as a critical frontier for enabling true omnimodal
generation.
ROVER is a benchmark that evaluates reciprocal cross-modal reasoning in unified multimodal models, showing that cross-modal interactions significantly impact visual generation quality and that models struggle with symbolic reasoning tasks.
unified multimodal models UMMs reciprocal cross-modal reasoning ROVER verbal prompts reasoning chains image synthesis intermediate visualizations question answering cross-modal reasoning visual generation quality interleaved models non-interleaved models unimodal models physical reasoning symbolic reasoning perceptual concepts visual abstractions