WorldVQA: Measuring Atomic World Knowledge in Multimodal Large Language Models
We introduce WorldVQA, a benchmark designed to evaluate the atomic visual world knowledge of Multimodal Large Language Models (MLLMs). Unlike current evaluations, which often conflate visual knowledge retrieval with reasoning, WorldVQA decouples these capabilities to strictly measure "what the model memorizes." The benchmark assesses the atomic capability of grounding and naming visual entities across a stratified taxonomy, spanning from common head-class objects to long-tail rarities. We expect WorldVQA to serve as a rigorous test for visual factuality, thereby establishing a standard for assessing the encyclopedic breadth and hallucination rates of current and next-generation frontier models.
6982bdf29084cb4f0ecb56f7
Runjie Zhou
6982bdf29084cb4f0ecb56f8
Youbo Shao
6982bdf29084cb4f0ecb56f9
Haoyu Lu
6982bdf29084cb4f0ecb56fa
Bowei Xing
6982bdf29084cb4f0ecb56fb
Tongtong Bai
6982bdf29084cb4f0ecb56fc
Yujie Chen
6982bdf29084cb4f0ecb56fd
Jie Zhao
6982bdf29084cb4f0ecb56fe
Lin Sui
6982bdf29084cb4f0ecb56ff
Haotian Yao
6982bdf29084cb4f0ecb5700
Zijia Zhao
6982bdf29084cb4f0ecb5701
Hao Yang
6982bdf29084cb4f0ecb5702
Haoning Wu
6982bdf29084cb4f0ecb5703
Zaida Zhou
6982bdf29084cb4f0ecb5704
Jinguo Zhu
6982bdf29084cb4f0ecb5705
Zhiqi Huang
6982bdf29084cb4f0ecb5706
Yiping Bao
6982bdf29084cb4f0ecb5707
Yangyang Liu
6982bdf29084cb4f0ecb5708
Y. Charles
6982bdf29084cb4f0ecb5709
Xinyu Zhou
WorldVQA: Measuring Atomic World Knowledge in Multimodal Large Language Models
We introduce WorldVQA, a benchmark designed to evaluate the atomic visual world knowledge of Multimodal Large Language Models (MLLMs). Unlike current evaluations, which often conflate visual knowledge retrieval with reasoning, WorldVQA decouples these capabilities to strictly measure "what the model memorizes." The benchmark assesses the atomic capability of grounding and naming visual entities across a stratified taxonomy, spanning from common head-class objects to long-tail rarities. We expect WorldVQA to serve as a rigorous test for visual factuality, thereby establishing a standard for assessing the encyclopedic breadth and hallucination rates of current and next-generation frontier models.
WorldVQA is a benchmark for evaluating the visual world knowledge of multimodal large language models by separating visual knowledge retrieval from reasoning to measure memorized facts.
Multimodal Large Language Models visual world knowledge visual knowledge retrieval reasoning atomic visual world knowledge visual factuality encyclopedic breadth hallucination rates