VABench: A Comprehensive Benchmark for Audio-Video Generation
Recent advances in video generation have been remarkable, enabling models to produce visually compelling videos with synchronized audio. While existing video generation benchmarks provide comprehensive metrics for visual quality, they lack convincing evaluations for audio-video generation, especially for models aiming to generate synchronized audio-video outputs. To address this gap, we introduce VABench, a comprehensive and multi-dimensional benchmark framework designed to systematically evaluate the capabilities of synchronous audio-video generation. VABench encompasses three primary task types: text-to-audio-video (T2AV), image-to-audio-video (I2AV), and stereo audio-video generation. It further establishes two major evaluation modules covering 15 dimensions. These dimensions specifically assess pairwise similarities (text-video, text-audio, video-audio), audio-video synchronization, lip-speech consistency, and carefully curated audio and video question-answering (QA) pairs, among others. Furthermore, VABench covers seven major content categories: animals, human sounds, music, environmental sounds, synchronous physical sounds, complex scenes, and virtual worlds. We provide a systematic analysis and visualization of the evaluation results, aiming to establish a new standard for assessing video generation models with synchronous audio capabilities and to promote the comprehensive advancement of the field.
69436b5c542d62d58a7bf677
Daili Hua
69436b5c542d62d58a7bf678
Xizhi Wang
69436b5c542d62d58a7bf679
Bohan Zeng
claimed_verified
2026-01-30T09:37:46.541Z
6671214c92412fd4640714eb
/avatars/48fa84e7bc3bb92ad0192aa26b32de10.svg
bohan zeng
zbhpku
user
zbhpku
69436b5c542d62d58a7bf67a
Xinyi Huang
69436b5c542d62d58a7bf67b
Hao Liang
claimed_verified
2026-04-07T08:49:43.521Z
6751a4fedf636b0140a9b873
/avatars/d75f7f6cfbfb4d646e0e557d1cfacdce.svg
Hao Liang
lhpku20010120
user
lhpku20010120
69436b5c542d62d58a7bf67c
Junbo Niu
69436b5c542d62d58a7bf67d
Xinlong Chen
69436b5c542d62d58a7bf67e
Quanqing Xu
69436b5c542d62d58a7bf67f
Wentao Zhang
VABench: A Comprehensive Benchmark for Audio-Video Generation
Recent advances in video generation have been remarkable, enabling models to produce visually compelling videos with synchronized audio. While existing video generation benchmarks provide comprehensive metrics for visual quality, they lack convincing evaluations for audio-video generation, especially for models aiming to generate synchronized audio-video outputs. To address this gap, we introduce VABench, a comprehensive and multi-dimensional benchmark framework designed to systematically evaluate the capabilities of synchronous audio-video generation. VABench encompasses three primary task types: text-to-audio-video (T2AV), image-to-audio-video (I2AV), and stereo audio-video generation. It further establishes two major evaluation modules covering 15 dimensions. These dimensions specifically assess pairwise similarities (text-video, text-audio, video-audio), audio-video synchronization, lip-speech consistency, and carefully curated audio and video question-answering (QA) pairs, among others. Furthermore, VABench covers seven major content categories: animals, human sounds, music, environmental sounds, synchronous physical sounds, complex scenes, and virtual worlds. We provide a systematic analysis and visualization of the evaluation results, aiming to establish a new standard for assessing video generation models with synchronous audio capabilities and to promote the comprehensive advancement of the field.
VABench is a benchmark framework for evaluating audio-video generation models, covering text-to-audio-video, image-to-audio-video, and stereo audio-video tasks with 15 evaluation dimensions.
text-to-audio-video image-to-audio-video stereo audio-video generation pairwise similarities audio-video synchronization lip-speech consistency question-answering pairs animals human sounds music environmental sounds synchronous physical sounds complex scenes virtual worlds