AVMeme Exam: A Multimodal Multilingual Multicultural Benchmark for LLMs' Contextual and Cultural Knowledge and Thinking
Internet audio-visual clips convey meaning through time-varying sound and motion, which extend beyond what text alone can represent. To examine whether AI models can understand such signals in human cultural contexts, we introduce AVMeme Exam, a human-curated benchmark of over one thousand iconic Internet sounds and videos spanning speech, songs, music, and sound effects. Each meme is paired with a unique Q&A assessing levels of understanding from surface content to context and emotion to usage and world knowledge, along with metadata such as original year, transcript, summary, and sensitivity. We systematically evaluate state-of-the-art multimodal large language models (MLLMs) alongside human participants using this benchmark. Our results reveal a consistent limitation: current models perform poorly on textless music and sound effects, and struggle to think in context and in culture compared to surface content. These findings highlight a key gap in human-aligned multimodal intelligence and call for models that can perceive contextually and culturally beyond the surface of what they hear and see. Project page: avmemeexam.github.io/public
6978296a026bdf0473116dc0
Xilin Jiang
6978296a026bdf0473116dc1
Qiaolin Wang
claimed_verified
2026-01-28T11:31:22.668Z
68fc33d3705bf8aa76ef5215
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/8kks5zJgOMoU2GuXUgRZq.jpeg
Qiaolin Wang
QiaolinWang
user
QiaolinWang
6978296a026bdf0473116dc2
Junkai Wu
claimed_verified
2026-01-27T09:42:46.216Z
665867a5d258e0fe41dcfb7a
/avatars/dfc23ec27c1e5177f44b2c881e12d458.svg
Junkai Wu
wjk0925
user
wjk0925
6978296a026bdf0473116dc3
Xiaomin He
6978296a026bdf0473116dc4
Zhongweiyang Xu
6978296a026bdf0473116dc5
Yinghao Ma
6978296a026bdf0473116dc6
Minshuo Piao
6978296a026bdf0473116dc7
Kaiyi Yang
6978296a026bdf0473116dc8
Xiuwen Zheng
6978296a026bdf0473116dc9
Riki Shimizu
6978296a026bdf0473116dca
Yicong Chen
6978296a026bdf0473116dcb
Arsalan Firoozi
6978296a026bdf0473116dcc
Gavin Mischler
6978296a026bdf0473116dcd
Sukru Samet Dindar
6978296a026bdf0473116dce
Richard Antonello
6978296a026bdf0473116dcf
Linyang He
6978296a026bdf0473116dd0
Tsun-An Hsieh
6978296a026bdf0473116dd1
Xulin Fan
6978296a026bdf0473116dd2
Yulun Wu
6978296a026bdf0473116dd3
Yuesheng Ma
6978296a026bdf0473116dd4
Chaitanya Amballa
6978296a026bdf0473116dd5
Weixiong Chen
claimed_verified
2026-03-05T09:58:44.694Z
6628452bb11cb848a44a49c8
/avatars/e9295bc28549c8501588430370f88ea5.svg
Weixiong Chen
Leon299
user
Leon299
6978296a026bdf0473116dd6
Jiarui Hai
6978296a026bdf0473116dd7
Ruisi Li
6978296a026bdf0473116dd8
Vishal Choudhari
6978296a026bdf0473116dd9
Cong Han
6978296a026bdf0473116dda
Yinghao Aaron Li
6978296a026bdf0473116ddb
Adeen Flinker
6978296a026bdf0473116ddc
Mounya Elhilali
6978296a026bdf0473116ddd
Emmanouil Benetos
6978296a026bdf0473116dde
Mark Hasegawa-Johnson
6978296a026bdf0473116ddf
Romit Roy Choudhury
6978296a026bdf0473116de0
Nima Mesgarani
AVMeme Exam: A Multimodal Multilingual Multicultural Benchmark for LLMs' Contextual and Cultural Knowledge and Thinking
Internet audio-visual clips convey meaning through time-varying sound and motion, which extend beyond what text alone can represent. To examine whether AI models can understand such signals in human cultural contexts, we introduce AVMeme Exam, a human-curated benchmark of over one thousand iconic Internet sounds and videos spanning speech, songs, music, and sound effects. Each meme is paired with a unique Q&A assessing levels of understanding from surface content to context and emotion to usage and world knowledge, along with metadata such as original year, transcript, summary, and sensitivity. We systematically evaluate state-of-the-art multimodal large language models (MLLMs) alongside human participants using this benchmark. Our results reveal a consistent limitation: current models perform poorly on textless music and sound effects, and struggle to think in context and in culture compared to surface content. These findings highlight a key gap in human-aligned multimodal intelligence and call for models that can perceive contextually and culturally beyond the surface of what they hear and see. Project page: avmemeexam.github.io/public
Current multimodal models demonstrate limited understanding of cultural and contextual audio-visual content, particularly excelling only in surface-level analysis rather than deeper semantic comprehension.
multimodal large language models audio-visual clips human-curated benchmark iconic Internet sounds cultural context surface content semantic comprehension