Video Reality Test: Can AI-Generated ASMR Videos fool VLMs and Humans?
Recent advances in video generation have produced vivid content that are often indistinguishable from real videos, making AI-generated video detection an emerging societal challenge. Prior AIGC detection benchmarks mostly evaluate video without audio, target broad narrative domains, and focus on classification solely. Yet it remains unclear whether state-of-the-art video generation models can produce immersive, audio-paired videos that reliably deceive humans and VLMs. To this end, we introduce Video Reality Test, an ASMR-sourced video benchmark suite for testing perceptual realism under tight audio-visual coupling, featuring the following dimensions: (i) Immersive ASMR video-audio sources. Built on carefully curated real ASMR videos, the benchmark targets fine-grained action-object interactions with diversity across objects, actions, and backgrounds. (ii) Peer-Review evaluation. An adversarial creator-reviewer protocol where video generation models act as creators aiming to fool reviewers, while VLMs serve as reviewers seeking to identify fakeness. Our experimental findings show: The best creator Veo3.1-Fast even fools most VLMs: the strongest reviewer (Gemini 2.5-Pro) achieves only 56\% accuracy (random 50\%), far below that of human experts (81.25\%). Adding audio improves real-fake discrimination, yet superficial cues such as watermarks can still significantly mislead models. These findings delineate the current boundary of video generation realism and expose limitations of VLMs in perceptual fidelity and audio-visual consistency. Our code is available at https://github.com/video-reality-test/video-reality-test.
6940d82465f1e24a117805c2
Jiaqi Wang
claimed_verified
2025-12-25T20:53:14.863Z
65b862af23d948d884ae9169
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/aAAokhlISNMiNuO_0LzrM.png
jiaqi wang
kolerk
user
kolerk
6940d82465f1e24a117805c3
Weijia Wu
6940d82465f1e24a117805c4
Yi Zhan
6940d82465f1e24a117805c5
Rui Zhao
6940d82465f1e24a117805c6
Ming Hu
6940d82465f1e24a117805c7
James Cheng
6940d82465f1e24a117805c8
Wei Liu
6940d82465f1e24a117805c9
Philip Torr
6940d82465f1e24a117805ca
Kevin Qinghong Lin
claimed_verified
2025-12-17T14:04:37.407Z
64440be5af034cdfd69ca3a7
https://cdn-avatars.huggingface.co/v1/production/uploads/64440be5af034cdfd69ca3a7/qmx24QiDFT29vleCxL9TX.jpeg
Qinghong (Kevin) Lin
KevinQHLin
user
KevinQHLin
Video Reality Test: Can AI-Generated ASMR Videos fool VLMs and Humans?
Recent advances in video generation have produced vivid content that are often indistinguishable from real videos, making AI-generated video detection an emerging societal challenge. Prior AIGC detection benchmarks mostly evaluate video without audio, target broad narrative domains, and focus on classification solely. Yet it remains unclear whether state-of-the-art video generation models can produce immersive, audio-paired videos that reliably deceive humans and VLMs. To this end, we introduce Video Reality Test, an ASMR-sourced video benchmark suite for testing perceptual realism under tight audio-visual coupling, featuring the following dimensions: (i) Immersive ASMR video-audio sources. Built on carefully curated real ASMR videos, the benchmark targets fine-grained action-object interactions with diversity across objects, actions, and backgrounds. (ii) Peer-Review evaluation. An adversarial creator-reviewer protocol where video generation models act as creators aiming to fool reviewers, while VLMs serve as reviewers seeking to identify fakeness. Our experimental findings show: The best creator Veo3.1-Fast even fools most VLMs: the strongest reviewer (Gemini 2.5-Pro) achieves only 56\% accuracy (random 50\%), far below that of human experts (81.25\%). Adding audio improves real-fake discrimination, yet superficial cues such as watermarks can still significantly mislead models. These findings delineate the current boundary of video generation realism and expose limitations of VLMs in perceptual fidelity and audio-visual consistency. Our code is available at https://github.com/video-reality-test/video-reality-test.
The Video Reality Test benchmark evaluates the realism and detection of AI-generated ASMR videos with audio, revealing that even the best models can deceive VLMs and humans, highlighting limitations in perceptual fidelity and audio-visual consistency.
ASMR audio-visual coupling Veo3.1-Fast Gemini 2.5-Pro perceptual realism real-fake discrimination audio-visual consistency