Appear2Meaning: A Cross-Cultural Benchmark for Structured Cultural Metadata Inference from Images
Recent advances in vision-language models (VLMs) have improved image captioning for cultural heritage. However, inferring structured cultural metadata (e.g., creator, origin, period) from visual input remains underexplored. We introduce a multi-category, cross-cultural benchmark for this task and evaluate VLMs using an LLM-as-Judge framework that measures semantic alignment with reference annotations. To assess cultural reasoning, we report exact-match, partial-match, and attribute-level accuracy across cultural regions. Results show that models capture fragmented signals and exhibit substantial performance variation across cultures and metadata types, leading to inconsistent and weakly grounded predictions. These findings highlight the limitations of current VLMs in structured cultural metadata inference beyond visual perception.
69d96d1926c932a7c2810106
Yuechen Jiang
claimed_verified
2026-04-11T10:41:19.203Z
67c07426a600107b41ed84fb
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/gI7Ok9-LvAjhg8eotX96g.png
Yuechen Jiang
CarolynJiang
user
CarolynJiang
69d96d1926c932a7c2810107
Enze Zhang
69d96d1926c932a7c2810108
Md Mohsinul Kabir
69d96d1926c932a7c2810109
Qianqian Xie
69d96d1926c932a7c281010a
Stavroula Golfomitsou
69d96d1926c932a7c281010b
Konstantinos Arvanitis
69d96d1926c932a7c281010c
Sophia Ananiadou
Appear2Meaning: A Cross-Cultural Benchmark for Structured Cultural Metadata Inference from Images
Recent advances in vision-language models (VLMs) have improved image captioning for cultural heritage. However, inferring structured cultural metadata (e.g., creator, origin, period) from visual input remains underexplored. We introduce a multi-category, cross-cultural benchmark for this task and evaluate VLMs using an LLM-as-Judge framework that measures semantic alignment with reference annotations. To assess cultural reasoning, we report exact-match, partial-match, and attribute-level accuracy across cultural regions. Results show that models capture fragmented signals and exhibit substantial performance variation across cultures and metadata types, leading to inconsistent and weakly grounded predictions. These findings highlight the limitations of current VLMs in structured cultural metadata inference beyond visual perception.
Vision-language models demonstrate limited capability in inferring structured cultural metadata from visual input, showing inconsistent performance across different cultures and metadata types.
vision-language models cultural heritage structured cultural metadata LLM-as-Judge framework semantic alignment exact-match accuracy partial-match accuracy attribute-level accuracy