ChronoPlay: A Framework for Modeling Dual Dynamics and Authenticity in
  Game RAG Benchmarks
Retrieval Augmented Generation (RAG) systems are increasingly vital in
dynamic domains like online gaming, yet the lack of a dedicated benchmark has
impeded standardized evaluation in this area. The core difficulty lies in Dual
Dynamics: the constant interplay between game content updates and the shifting
focus of the player community. Furthermore, the necessity of automating such a
benchmark introduces a critical requirement for player-centric authenticity to
ensure generated questions are realistic. To address this integrated challenge,
we introduce ChronoPlay, a novel framework for the automated and continuous
generation of game RAG benchmarks. ChronoPlay utilizes a dual-dynamic update
mechanism to track both forms of change, and a dual-source synthesis engine
that draws from official sources and player community to ensure both factual
correctness and authentic query patterns. We instantiate our framework on three
distinct games to create the first dynamic RAG benchmark for the gaming domain,
offering new insights into model performance under these complex and realistic
conditions. Code is avaliable at: https://github.com/hly1998/ChronoPlay.
69008ddd646208eac0d1ef5c
Liyang He
claimed_verified
2025-10-29T12:49:35.567Z
661e6136a2bb874f6c283113
/avatars/fe28c25310af37be241955fe41e75557.svg
Liyang He
leoner24
user
leoner24
69008ddd646208eac0d1ef5d
Yuren Zhang
69008ddd646208eac0d1ef5e
Ziwei Zhu
69008ddd646208eac0d1ef5f
Zhenghui Li
69008ddd646208eac0d1ef60
Shiwei Tong
ChronoPlay: A Framework for Modeling Dual Dynamics and Authenticity in
  Game RAG Benchmarks
Retrieval Augmented Generation (RAG) systems are increasingly vital in
dynamic domains like online gaming, yet the lack of a dedicated benchmark has
impeded standardized evaluation in this area. The core difficulty lies in Dual
Dynamics: the constant interplay between game content updates and the shifting
focus of the player community. Furthermore, the necessity of automating such a
benchmark introduces a critical requirement for player-centric authenticity to
ensure generated questions are realistic. To address this integrated challenge,
we introduce ChronoPlay, a novel framework for the automated and continuous
generation of game RAG benchmarks. ChronoPlay utilizes a dual-dynamic update
mechanism to track both forms of change, and a dual-source synthesis engine
that draws from official sources and player community to ensure both factual
correctness and authentic query patterns. We instantiate our framework on three
distinct games to create the first dynamic RAG benchmark for the gaming domain,
offering new insights into model performance under these complex and realistic
conditions. Code is avaliable at: https://github.com/hly1998/ChronoPlay.
ChronoPlay is a framework for generating dynamic RAG benchmarks in gaming, addressing the challenges of game content updates and player focus shifts.
Retrieval Augmented Generation RAG Dual Dynamics ChronoPlay dual-dynamic update mechanism dual-source synthesis engine dynamic RAG benchmark