MMOU: A Massive Multi-Task Omni Understanding and Reasoning Benchmark for Long and Complex Real-World Videos
Multimodal Large Language Models (MLLMs) have shown strong performance in visual and audio understanding when evaluated in isolation. However, their ability to jointly reason over omni-modal (visual, audio, and textual) signals in long and complex videos remains largely unexplored. We introduce MMOU, a new benchmark designed to systematically evaluate multimodal understanding and reasoning under these challenging, real-world conditions. MMOU consists of 15,000 carefully curated questions paired with 9038 web-collected videos of varying length, spanning diverse domains and exhibiting rich, tightly coupled audio-visual content. The benchmark covers 13 fundamental skill categories, all of which require integrating evidence across modalities and time. All questions are manually annotated across multiple turns by professional annotators, ensuring high quality and reasoning fidelity. We evaluate 20+ state-of-the-art open-source and proprietary multimodal models on MMOU. The results expose substantial performance gaps: the best closed-source model achieves only 64.2% accuracy, while the strongest open-source model reaches just 46.8%. Our results highlight the challenges of long-form omni-modal understanding, revealing that current models frequently fail to apply even fundamental skills in long videos. Through detailed analysis, we further identify systematic failure modes and provide insights into where and why current models break.
69b8c95d6b3f36c249fbc3fd
Arushi Goel
69b8c95d6b3f36c249fbc3fe
Sreyan Ghosh
admin_assigned
2026-03-17T08:45:23.537Z
62c9664eb34e600d7eaa4beb
/avatars/ca23ecdec2d31c99ecce97d9b180ae0c.svg
Ghosh
Sreyan88
user
Sreyan88
69b8c95d6b3f36c249fbc3ff
Vatsal Agarwal
claimed_verified
2026-03-17T08:20:14.964Z
653213eaa0dffcea236ca422
/avatars/69d30ce923edcf99e177669c2f8bf9a5.svg
Vatsal Agarwal
vatsalag
user
vatsalag
69b8c95d6b3f36c249fbc400
Nishit Anand
claimed_verified
2026-03-17T08:20:16.996Z
6316edb01cc81c5e95eb6100
https://cdn-avatars.huggingface.co/v1/production/uploads/6316edb01cc81c5e95eb6100/r0L1kMMOUQENc1vpsbyxk.jpeg
Nishit Anand
nishitanand
user
nishitanand
69b8c95d6b3f36c249fbc401
Kaousheik Jayakumar
69b8c95d6b3f36c249fbc402
Lasha Koroshinadze
admin_assigned
2026-03-17T08:45:33.587Z
63fc96a0b9db84750cec216f
/avatars/eb8cb2518260de4851f8f6aa8bc5508c.svg
Lasha Koroshinadze
lashahub
user
lashahub
69b8c95d6b3f36c249fbc403
Yao Xu
69b8c95d6b3f36c249fbc404
Katie Lyons
69b8c95d6b3f36c249fbc405
James Case
69b8c95d6b3f36c249fbc406
Karan Sapra
admin_assigned
2026-03-17T08:45:47.463Z
66ec4204e0f57407e09b7762
/avatars/cb9b390941511ae9f1c58697f3f05c28.svg
Karan
karansapra
user
karansapra
69b8c95d6b3f36c249fbc407
Kevin J. Shih
69b8c95d6b3f36c249fbc408
Siddharth Gururani
69b8c95d6b3f36c249fbc409
Abhinav Shrivastava
69b8c95d6b3f36c249fbc40a
Ramani Duraiswami
admin_assigned
2026-03-17T08:46:00.811Z
65b8f0339b7250e205100190
/avatars/29df91154c5911a44ef503bb3580ac81.svg
Ramani Duraiswami
RamaniD
user
RamaniD
69b8c95d6b3f36c249fbc40b
Dinesh Manocha
admin_assigned
2026-03-17T08:46:07.672Z
6537a569568d8be8fa096b8c
/avatars/bfda5cb252d8b5bc3ad737d99c0d7f49.svg
Dinesh Manocha
manocha
user
manocha
69b8c95d6b3f36c249fbc40c
Andrew Tao
69b8c95d6b3f36c249fbc40d
Bryan Catanzaro
admin_assigned
2026-03-17T08:46:15.539Z
6311021788942700629e6247
https://cdn-avatars.huggingface.co/v1/production/uploads/6311021788942700629e6247/vDktfPZwSRRhYIxkUKYjP.jpeg
Bryan Catanzaro
ctnzr
user
ctnzr
69b8c95d6b3f36c249fbc40e
Mohammad Shoeybi
admin_assigned
2026-03-17T08:46:31.672Z
6641544c695975af2cbd0da6
/avatars/0ad3c18dcba585259b064fe9b00a07ce.svg
Mohammad Shoeybi
shoeybi
user
shoeybi
69b8c95d6b3f36c249fbc40f
Wei Ping
MMOU: A Massive Multi-Task Omni Understanding and Reasoning Benchmark for Long and Complex Real-World Videos
Multimodal Large Language Models (MLLMs) have shown strong performance in visual and audio understanding when evaluated in isolation. However, their ability to jointly reason over omni-modal (visual, audio, and textual) signals in long and complex videos remains largely unexplored. We introduce MMOU, a new benchmark designed to systematically evaluate multimodal understanding and reasoning under these challenging, real-world conditions. MMOU consists of 15,000 carefully curated questions paired with 9038 web-collected videos of varying length, spanning diverse domains and exhibiting rich, tightly coupled audio-visual content. The benchmark covers 13 fundamental skill categories, all of which require integrating evidence across modalities and time. All questions are manually annotated across multiple turns by professional annotators, ensuring high quality and reasoning fidelity. We evaluate 20+ state-of-the-art open-source and proprietary multimodal models on MMOU. The results expose substantial performance gaps: the best closed-source model achieves only 64.2% accuracy, while the strongest open-source model reaches just 46.8%. Our results highlight the challenges of long-form omni-modal understanding, revealing that current models frequently fail to apply even fundamental skills in long videos. Through detailed analysis, we further identify systematic failure modes and provide insights into where and why current models break.