Agentic-MME: What Agentic Capability Really Brings to Multimodal Intelligence?
Multimodal Large Language Models (MLLMs) are evolving from passive observers into active agents, solving problems through Visual Expansion (invoking visual tools) and Knowledge Expansion (open-web search). However, existing evaluations fall short: they lack flexible tool integration, test visual and search tools separately, and evaluate primarily by final answers. Consequently, they cannot verify if tools were actually invoked, applied correctly, or used efficiently. To address this, we introduce Agentic-MME, a process-verified benchmark for Multimodal Agentic Capabilities. It contains 418 real-world tasks across 6 domains and 3 difficulty levels to evaluate capability synergy, featuring over 2,000 stepwise checkpoints that average 10+ person-hours of manual annotation per task. Each task includes a unified evaluation framework supporting sandboxed code and APIs, alongside a human reference trajectory annotated with stepwise checkpoints along dual-axis: S-axis and V-axis. To enable true process-level verification, we audit fine-grained intermediate states rather than just final answers, and quantify efficiency via an overthinking metric relative to human trajectories. Experimental results show the best model, Gemini3-pro, achieves 56.3% overall accuracy, which falls significantly to 23.0% on Level-3 tasks, underscoring the difficulty of real-world multimodal agentic problem solving.
69d311402c523d36431fad2d
Qianshan Wei
69d311402c523d36431fad2e
Yishan Yang
69d311402c523d36431fad2f
Siyi Wang
claimed_verified
2026-04-13T07:35:11.389Z
6756ce064b5d676c93310100
/avatars/c6a4ebd713cde893929bb90e87ca5597.svg
Siyi Wang#20
Crystal1047
user
Crystal1047
69d311402c523d36431fad30
Jinglin Chen
69d311402c523d36431fad31
Binyu Wang
69d311402c523d36431fad32
Jiaming Wang
claimed_verified
2026-04-07T08:38:33.394Z
68355c5ec0003bc40230b3f2
https://cdn-avatars.huggingface.co/v1/production/uploads/68355c5ec0003bc40230b3f2/fJjAPFtmAJskQJqxWUb-T.jpeg
jasmineWang
Jessamine
user
Jessamine
69d311402c523d36431fad33
Shuang Chen
claimed_verified
2026-04-07T08:38:36.794Z
66cc19ad6f8945277c39cd86
/avatars/f031e77d7fb9dbebbc00fba9b5dd5357.svg
Shawn
csfufu
user
csfufu
69d311402c523d36431fad34
Zechen Li
69d311402c523d36431fad35
Yang Shi
claimed_verified
2026-04-07T08:38:39.677Z
673c7319d11b1c2e246ead9c
https://cdn-avatars.huggingface.co/v1/production/uploads/673c7319d11b1c2e246ead9c/IjFIO--N7Hm_BOEafhEQv.jpeg
Yang Shi
DogNeverSleep
user
DogNeverSleep
69d311402c523d36431fad36
Yuqi Tang
69d311402c523d36431fad37
Weining Wang
69d311402c523d36431fad38
Yi Yu
69d311402c523d36431fad39
Chaoyou Fu
claimed_verified
2026-04-08T08:06:29.373Z
653b268cd1041ca9188954da
https://cdn-avatars.huggingface.co/v1/production/uploads/653b268cd1041ca9188954da/5Qy0fyQdRCIUUAzV6fg1k.png
Chaoyou Fu
BradyFU
user
BradyFU
69d311402c523d36431fad3a
Qi Li
69d311402c523d36431fad3b
Yi-Fan Zhang
claimed_verified
2026-04-08T08:57:24.164Z
623d8ca4c29adf5ef6175615
https://cdn-avatars.huggingface.co/v1/production/uploads/623d8ca4c29adf5ef6175615/q7lHao7UPwU1u7YLSP56m.jpeg
Yi-Fan Zhang
yifanzhang114
user
yifanzhang114
Agentic-MME: What Agentic Capability Really Brings to Multimodal Intelligence?
Multimodal Large Language Models (MLLMs) are evolving from passive observers into active agents, solving problems through Visual Expansion (invoking visual tools) and Knowledge Expansion (open-web search). However, existing evaluations fall short: they lack flexible tool integration, test visual and search tools separately, and evaluate primarily by final answers. Consequently, they cannot verify if tools were actually invoked, applied correctly, or used efficiently. To address this, we introduce Agentic-MME, a process-verified benchmark for Multimodal Agentic Capabilities. It contains 418 real-world tasks across 6 domains and 3 difficulty levels to evaluate capability synergy, featuring over 2,000 stepwise checkpoints that average 10+ person-hours of manual annotation per task. Each task includes a unified evaluation framework supporting sandboxed code and APIs, alongside a human reference trajectory annotated with stepwise checkpoints along dual-axis: S-axis and V-axis. To enable true process-level verification, we audit fine-grained intermediate states rather than just final answers, and quantify efficiency via an overthinking metric relative to human trajectories. Experimental results show the best model, Gemini3-pro, achieves 56.3% overall accuracy, which falls significantly to 23.0% on Level-3 tasks, underscoring the difficulty of real-world multimodal agentic problem solving.
A new benchmark evaluates multimodal agentic capabilities by verifying tool usage and process efficiency rather than just final answers, revealing significant challenges in real-world multimodal problem-solving.
Multimodal Large Language Models Visual Expansion Knowledge Expansion Agentic-MME process-verified benchmark tool integration stepwise checkpoints sandboxed code overthinking metric