Tracing the Roots: A Multi-Agent Framework for Uncovering Data Lineage in Post-Training LLMs
Post-training data plays a pivotal role in shaping the capabilities of Large Language Models (LLMs), yet datasets are often treated as isolated artifacts, overlooking the systemic connections that underlie their evolution. To disentangle these complex relationships, we introduce the concept of data lineage to the LLM ecosystem and propose an automated multi-agent framework to reconstruct the evolutionary graph of dataset development. Through large-scale lineage analysis, we characterize domain-specific structural patterns, such as vertical refinement in math-oriented datasets and horizontal aggregation in general-domain corpora. Moreover, we uncover pervasive systemic issues, including structural redundancy induced by implicit dataset intersections and the propagation of benchmark contamination along lineage paths. To demonstrate the practical value of lineage analysis for data construction, we leverage the reconstructed lineage graph to create a lineage-aware diversity-oriented dataset. By anchoring instruction sampling at upstream root sources, this approach mitigates downstream homogenization and hidden redundancy, yielding a more diverse post-training corpus. We further highlight lineage-centric analysis as an efficient and robust topological alternative to sample-level dataset comparison for large-scale data ecosystems. By grounding data construction in explicit lineage structures, our work advances post-training data curation toward a more systematic and controllable paradigm.
69ddb9daaea31e227c22c59b
Yu Li
69ddb9daaea31e227c22c59c
Xiaoran Shang
69ddb9daaea31e227c22c59d
Qizhi Pei
claimed_verified
2026-04-14T08:15:09.048Z
6397f6081323f19c578f142e
https://cdn-avatars.huggingface.co/v1/production/uploads/6397f6081323f19c578f142e/it7FYYKjlLX8wSsMLm8EO.jpeg
QizhiPei
QizhiPei
user
QizhiPei
69ddb9daaea31e227c22c59e
Yun Zhu
69ddb9daaea31e227c22c59f
Xin Gao
69ddb9daaea31e227c22c5a0
Honglin Lin
69ddb9daaea31e227c22c5a1
Zhanping Zhong
69ddb9daaea31e227c22c5a2
Zhuoshi Pan
69ddb9daaea31e227c22c5a3
Zheng Liu
69ddb9daaea31e227c22c5a4
Xiaoyang Wang
69ddb9daaea31e227c22c5a5
Conghui He
69ddb9daaea31e227c22c5a6
Dahua Lin
69ddb9daaea31e227c22c5a7
Feng Zhao
69ddb9daaea31e227c22c5a8
Lijun Wu
Tracing the Roots: A Multi-Agent Framework for Uncovering Data Lineage in Post-Training LLMs
Post-training data plays a pivotal role in shaping the capabilities of Large Language Models (LLMs), yet datasets are often treated as isolated artifacts, overlooking the systemic connections that underlie their evolution. To disentangle these complex relationships, we introduce the concept of data lineage to the LLM ecosystem and propose an automated multi-agent framework to reconstruct the evolutionary graph of dataset development. Through large-scale lineage analysis, we characterize domain-specific structural patterns, such as vertical refinement in math-oriented datasets and horizontal aggregation in general-domain corpora. Moreover, we uncover pervasive systemic issues, including structural redundancy induced by implicit dataset intersections and the propagation of benchmark contamination along lineage paths. To demonstrate the practical value of lineage analysis for data construction, we leverage the reconstructed lineage graph to create a lineage-aware diversity-oriented dataset. By anchoring instruction sampling at upstream root sources, this approach mitigates downstream homogenization and hidden redundancy, yielding a more diverse post-training corpus. We further highlight lineage-centric analysis as an efficient and robust topological alternative to sample-level dataset comparison for large-scale data ecosystems. By grounding data construction in explicit lineage structures, our work advances post-training data curation toward a more systematic and controllable paradigm.
Lineage analysis reveals structural patterns and systemic issues in LLM dataset evolution, enabling more diverse and controlled data curation through lineage-aware sampling approaches.
data lineage multi-agent framework evolutionary graph structural patterns vertical refinement horizontal aggregation structural redundancy benchmark contamination lineage-aware diversity-oriented dataset topological alternative data curation