No Shortcuts to Culture: Indonesian Multi-hop Question Answering for Complex Cultural Understanding
Understanding culture requires reasoning across context, tradition, and implicit social knowledge, far beyond recalling isolated facts. Yet most culturally focused question answering (QA) benchmarks rely on single-hop questions, which may allow models to exploit shallow cues rather than demonstrate genuine cultural reasoning. In this work, we introduce ID-MoCQA, the first large-scale multi-hop QA dataset for assessing the cultural understanding of large language models (LLMs), grounded in Indonesian traditions and available in both English and Indonesian. We present a new framework that systematically transforms single-hop cultural questions into multi-hop reasoning chains spanning six clue types (e.g., commonsense, temporal, geographical). Our multi-stage validation pipeline, combining expert review and LLM-as-a-judge filtering, ensures high-quality question-answer pairs. Our evaluation across state-of-the-art models reveals substantial gaps in cultural reasoning, particularly in tasks requiring nuanced inference. ID-MoCQA provides a challenging and essential benchmark for advancing the cultural competency of LLMs.
69830ec29084cb4f0ecb5943
Vynska Amalia Permadi
69830ec29084cb4f0ecb5944
Xingwei Tan
claimed_verified
2026-02-04T09:51:04.996Z
643d0a4d8a55b2bbf4f2a90e
/avatars/9534aaf81cbf12f015c6826b682fdb84.svg
Xingwei Tan
XingweiT
user
XingweiT
69830ec29084cb4f0ecb5945
Nafise Sadat Moosavi
69830ec29084cb4f0ecb5946
Nikos Aletras
No Shortcuts to Culture: Indonesian Multi-hop Question Answering for Complex Cultural Understanding
Understanding culture requires reasoning across context, tradition, and implicit social knowledge, far beyond recalling isolated facts. Yet most culturally focused question answering (QA) benchmarks rely on single-hop questions, which may allow models to exploit shallow cues rather than demonstrate genuine cultural reasoning. In this work, we introduce ID-MoCQA, the first large-scale multi-hop QA dataset for assessing the cultural understanding of large language models (LLMs), grounded in Indonesian traditions and available in both English and Indonesian. We present a new framework that systematically transforms single-hop cultural questions into multi-hop reasoning chains spanning six clue types (e.g., commonsense, temporal, geographical). Our multi-stage validation pipeline, combining expert review and LLM-as-a-judge filtering, ensures high-quality question-answer pairs. Our evaluation across state-of-the-art models reveals substantial gaps in cultural reasoning, particularly in tasks requiring nuanced inference. ID-MoCQA provides a challenging and essential benchmark for advancing the cultural competency of LLMs.
Multi-hop question answering dataset ID-MoCQA assesses cultural understanding in large language models through Indonesian traditions with diverse reasoning chains.
question answering large language models cultural understanding multi-hop reasoning dataset creation validation pipeline expert review LLM-as-a-judge filtering