MM-CondChain: A Programmatically Verified Benchmark for Visually Grounded Deep Compositional Reasoning
Multimodal Large Language Models (MLLMs) are increasingly used to carry out visual workflows such as navigating GUIs, where the next step depends on verified visual compositional conditions (e.g., "if a permission dialog appears and the color of the interface is green, click Allow") and the process may branch or terminate early. Yet this capability remains under-evaluated: existing benchmarks focus on shallow-compositions or independent-constraints rather than deeply chained compositional conditionals. In this paper, we introduce MM-CondChain, a benchmark for visually grounded deep compositional reasoning. Each benchmark instance is organized as a multi-layer reasoning chain, where every layer contains a non-trivial compositional condition grounded in visual evidence and built from multiple objects, attributes, or relations. To answer correctly, an MLLM must perceive the image in detail, reason over multiple visual elements at each step, and follow the resulting execution path to the final outcome. To scalably construct such workflow-style data, we propose an agentic synthesis pipeline: a Planner orchestrates layer-by-layer generation of compositional conditions, while a Verifiable Programmatic Intermediate Representation (VPIR) ensures each layer's condition is mechanically verifiable. A Composer then assembles these verified layers into complete instructions. Using this pipeline, we construct benchmarks across three visual domains: natural images, data charts, and GUI trajectories. Experiments on a range of MLLMs show that even the strongest model attains only 53.33 Path F1, with sharp drops on hard negatives and as depth or predicate complexity grows, confirming that deep compositional reasoning remains a fundamental challenge.
69b7760a91a28162b233fc3b
Haozhan Shen
admin_assigned
2026-03-16T11:08:38.712Z
64f05f261a108efe45dfeda1
/avatars/cd9e68dbede75a34cf286569e60ac2af.svg
Haozhan Shen
SZhanZ
user
SZhanZ
69b7760a91a28162b233fc3c
Shilin Yan
admin_assigned
2026-03-16T11:08:32.130Z
65273fea0ef49cfb783fa5c1
/avatars/0c9e204bc2151c8cc533311900d05a36.svg
Shilin Yan
shilinyan
user
shilinyan
69b7760a91a28162b233fc3d
Hongwei Xue
admin_assigned
2026-03-16T11:08:45.571Z
656718aa937e98464aa5eb62
/avatars/c1bc6563aee24bb3ed4ad761c353d8ae.svg
Hongwei Xue
Hellwayxue
user
Hellwayxue
69b7760a91a28162b233fc3e
Shuaiqi Lu
69b7760a91a28162b233fc3f
Xiaojun Tang
69b7760a91a28162b233fc40
Guannan Zhang
69b7760a91a28162b233fc41
Tiancheng Zhao
69b7760a91a28162b233fc42
Jianwei Yin
MM-CondChain: A Programmatically Verified Benchmark for Visually Grounded Deep Compositional Reasoning
Multimodal Large Language Models (MLLMs) are increasingly used to carry out visual workflows such as navigating GUIs, where the next step depends on verified visual compositional conditions (e.g., "if a permission dialog appears and the color of the interface is green, click Allow") and the process may branch or terminate early. Yet this capability remains under-evaluated: existing benchmarks focus on shallow-compositions or independent-constraints rather than deeply chained compositional conditionals. In this paper, we introduce MM-CondChain, a benchmark for visually grounded deep compositional reasoning. Each benchmark instance is organized as a multi-layer reasoning chain, where every layer contains a non-trivial compositional condition grounded in visual evidence and built from multiple objects, attributes, or relations. To answer correctly, an MLLM must perceive the image in detail, reason over multiple visual elements at each step, and follow the resulting execution path to the final outcome. To scalably construct such workflow-style data, we propose an agentic synthesis pipeline: a Planner orchestrates layer-by-layer generation of compositional conditions, while a Verifiable Programmatic Intermediate Representation (VPIR) ensures each layer's condition is mechanically verifiable. A Composer then assembles these verified layers into complete instructions. Using this pipeline, we construct benchmarks across three visual domains: natural images, data charts, and GUI trajectories. Experiments on a range of MLLMs show that even the strongest model attains only 53.33 Path F1, with sharp drops on hard negatives and as depth or predicate complexity grows, confirming that deep compositional reasoning remains a fundamental challenge.
MM-CondChain benchmark evaluates multimodal large language models on deep compositional visual reasoning through multi-layer conditional workflows with mechanically verifiable conditions.
Multimodal Large Language Models visual workflows GUI navigation compositional conditionals deep compositional reasoning agentic synthesis pipeline Planner Verifiable Programmatic Intermediate Representation VPIR Composer visual domains natural images data charts GUI trajectories Path F1 hard negatives predicate complexity