LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMs
Recent advancements in omnimodal large language models (OmniLLMs) have significantly improved the comprehension of audio and video inputs. However, current evaluations primarily focus on short audio and video clips ranging from 10 seconds to 5 minutes, failing to reflect the demands of real-world applications, where videos typically run for tens of minutes. To address this critical gap, we introduce LVOmniBench, a new benchmark designed specifically for the cross-modal comprehension of long-form audio and video. This dataset comprises high-quality videos sourced from open platforms that feature rich audio-visual dynamics. Through rigorous manual selection and annotation, LVOmniBench comprises 275 videos, ranging in duration from 10 to 90 minutes, and 1,014 question-answer (QA) pairs. LVOmniBench aims to rigorously evaluate the capabilities of OmniLLMs across domains, including long-term memory, temporal localization, fine-grained understanding, and multimodal perception. Our extensive evaluation reveals that current OmniLLMs encounter significant challenges when processing extended audio-visual inputs. Open-source models generally achieve accuracies below 35%, whereas the Gemini 3 Pro reaches a peak accuracy of approximately 65%. We anticipate that this dataset, along with our empirical findings, will stimulate further research and the development of advanced models capable of resolving complex cross-modal understanding problems within long-form audio-visual contexts.
69bcb548081b9ad49af57262
Keda Tao
69bcb548081b9ad49af57263
Yuhua Zheng
69bcb548081b9ad49af57264
Jia Xu
69bcb548081b9ad49af57265
Wenjie Du
claimed_verified
2026-03-23T10:53:26.010Z
6624ce8248e016b5ea4ba952
https://cdn-avatars.huggingface.co/v1/production/uploads/6624ce8248e016b5ea4ba952/WbpwxXABS5gomQJtSiiCo.jpeg
wenjiedu
Kurt232
user
Kurt232
69bcb548081b9ad49af57266
Kele Shao
69bcb548081b9ad49af57267
Hesong Wang
69bcb548081b9ad49af57268
Xueyi Chen
69bcb548081b9ad49af57269
Xin Jin
69bcb548081b9ad49af5726a
Junhan Zhu
claimed_verified
2026-04-15T12:09:29.647Z
67d5848f179ad2756600eca3
/avatars/158168a753271b6e024e1fbdf52c9e73.svg
Junhan ZHU
Alrightlone
user
Alrightlone
69bcb548081b9ad49af5726b
Bohan Yu
69bcb548081b9ad49af5726c
Weiqiang Wang
69bcb548081b9ad49af5726d
Jian Liu
69bcb548081b9ad49af5726e
Can Qin
69bcb548081b9ad49af5726f
Yulun Zhang
69bcb548081b9ad49af57270
Ming-Hsuan Yang
69bcb548081b9ad49af57271
Huan Wang
LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMs
Recent advancements in omnimodal large language models (OmniLLMs) have significantly improved the comprehension of audio and video inputs. However, current evaluations primarily focus on short audio and video clips ranging from 10 seconds to 5 minutes, failing to reflect the demands of real-world applications, where videos typically run for tens of minutes. To address this critical gap, we introduce LVOmniBench, a new benchmark designed specifically for the cross-modal comprehension of long-form audio and video. This dataset comprises high-quality videos sourced from open platforms that feature rich audio-visual dynamics. Through rigorous manual selection and annotation, LVOmniBench comprises 275 videos, ranging in duration from 10 to 90 minutes, and 1,014 question-answer (QA) pairs. LVOmniBench aims to rigorously evaluate the capabilities of OmniLLMs across domains, including long-term memory, temporal localization, fine-grained understanding, and multimodal perception. Our extensive evaluation reveals that current OmniLLMs encounter significant challenges when processing extended audio-visual inputs. Open-source models generally achieve accuracies below 35%, whereas the Gemini 3 Pro reaches a peak accuracy of approximately 65%. We anticipate that this dataset, along with our empirical findings, will stimulate further research and the development of advanced models capable of resolving complex cross-modal understanding problems within long-form audio-visual contexts.
Long-form audio-visual comprehension benchmark reveals significant challenges for current omnimodal large language models in handling extended multi-modal inputs.
omnimodal large language models cross-modal comprehension long-form audio-visual inputs multimodal perception temporal localization fine-grained understanding long-term memory