WildGraphBench: Benchmarking GraphRAG with Wild-Source Corpora
Graph-based Retrieval-Augmented Generation (GraphRAG) organizes external knowledge as a hierarchical graph, enabling efficient retrieval and aggregation of scattered evidence across multiple documents. However, many existing benchmarks for GraphRAG rely on short, curated passages as external knowledge, failing to adequately evaluate systems in realistic settings involving long contexts and large-scale heterogeneous documents. To bridge this gap, we introduce WildGraphBench, a benchmark designed to assess GraphRAG performance in the wild. We leverage Wikipedia's unique structure, where cohesive narratives are grounded in long and heterogeneous external reference documents, to construct a benchmark reflecting real-word scenarios. Specifically, we sample articles across 12 top-level topics, using their external references as the retrieval corpus and citation-linked statements as ground truth, resulting in 1,100 questions spanning three levels of complexity: single-fact QA, multi-fact QA, and section-level summarization. Experiments across multiple baselines reveal that current GraphRAG pipelines help on multi-fact aggregation when evidence comes from a moderate number of sources, but this aggregation paradigm may overemphasize high-level statements at the expense of fine-grained details, leading to weaker performance on summarization tasks. Project page:https://github.com/BstWPY/WildGraphBench.
698175bdce18b18628096063
Pengyu Wang
claimed_verified
2026-02-03T10:07:18.739Z
68c8ccff2de8edcf8da2fe51
/avatars/e55e509d11049691fefe3750a4bed62a.svg
wangpengyu
Bstwpy
user
Bstwpy
698175bdce18b18628096064
Benfeng Xu
698175bdce18b18628096065
Licheng Zhang
698175bdce18b18628096066
Shaohan Wang
claimed_verified
2026-02-03T10:07:14.629Z
6440d49b7663594a126716f2
/avatars/bb04af9d1ae9c5bc058ebfbf08f4ebc8.svg
shaohanwang
WShao
user
WShao
698175bdce18b18628096067
Mingxuan Du
claimed_verified
2026-02-03T10:07:16.739Z
646dbba74ad7f907279dd486
https://cdn-avatars.huggingface.co/v1/production/uploads/646dbba74ad7f907279dd486/XGHJMFEIpWeDlh0cn9Slu.png
Mingxuan Du
Ayanami0730
user
Ayanami0730
698175bdce18b18628096068
Chiwei Zhu
claimed_verified
2026-02-03T10:07:12.194Z
663b22a80966eef8686aadaf
https://cdn-avatars.huggingface.co/v1/production/uploads/663b22a80966eef8686aadaf/iBzyQTyGZKf33RPVIFh9a.jpeg
Chiwei Zhu
IgnoraZ
user
IgnoraZ
698175bdce18b18628096069
Zhendong Mao
WildGraphBench: Benchmarking GraphRAG with Wild-Source Corpora
Graph-based Retrieval-Augmented Generation (GraphRAG) organizes external knowledge as a hierarchical graph, enabling efficient retrieval and aggregation of scattered evidence across multiple documents. However, many existing benchmarks for GraphRAG rely on short, curated passages as external knowledge, failing to adequately evaluate systems in realistic settings involving long contexts and large-scale heterogeneous documents. To bridge this gap, we introduce WildGraphBench, a benchmark designed to assess GraphRAG performance in the wild. We leverage Wikipedia's unique structure, where cohesive narratives are grounded in long and heterogeneous external reference documents, to construct a benchmark reflecting real-word scenarios. Specifically, we sample articles across 12 top-level topics, using their external references as the retrieval corpus and citation-linked statements as ground truth, resulting in 1,100 questions spanning three levels of complexity: single-fact QA, multi-fact QA, and section-level summarization. Experiments across multiple baselines reveal that current GraphRAG pipelines help on multi-fact aggregation when evidence comes from a moderate number of sources, but this aggregation paradigm may overemphasize high-level statements at the expense of fine-grained details, leading to weaker performance on summarization tasks. Project page:https://github.com/BstWPY/WildGraphBench.
WildGraphBench evaluates GraphRAG performance in realistic scenarios using Wikipedia's structured content to assess multi-fact aggregation and summarization capabilities across diverse document types.
GraphRAG retrieval-augmented generation hierarchical graph external knowledge long contexts heterogeneous documents Wikipedia retrieval corpus citation-linked statements multi-fact aggregation summarization