Data Science and Technology Towards AGI Part I: Tiered Data Management
The development of artificial intelligence can be viewed as an evolution of data-driven learning paradigms, with successive shifts in data organization and utilization continuously driving advances in model capability. Current LLM research is dominated by a paradigm that relies heavily on unidirectional scaling of data size, increasingly encountering bottlenecks in data availability, acquisition cost, and training efficiency. In this work, we argue that the development of AGI is entering a new phase of data-model co-evolution, in which models actively guide data management while high-quality data, in turn, amplifies model capabilities. To implement this vision, we propose a tiered data management framework, designed to support the full LLM training lifecycle across heterogeneous learning objectives and cost constraints. Specifically, we introduce an L0-L4 tiered data management framework, ranging from raw uncurated resources to organized and verifiable knowledge. Importantly, LLMs are fully used in data management processes, such as quality scoring and content editing, to refine data across tiers. Each tier is characterized by distinct data properties, management strategies, and training roles, enabling data to be strategically allocated across LLM training stages, including pre-training, mid-training, and alignment. The framework balances data quality, acquisition cost, and marginal training benefit, providing a systematic approach to scalable and sustainable data management. We validate the effectiveness of the proposed framework through empirical studies, in which tiered datasets are constructed from raw corpora and used across multiple training phases. Experimental results demonstrate that tier-aware data utilization significantly improves training efficiency and model performance. To facilitate further research, we release our tiered datasets and processing tools to the community.
698b18981b2dc6b37d61b404
Yudong Wang
claimed_verified
2026-02-11T11:17:07.915Z
63be286fb3b8c44f8cecc16f
https://cdn-avatars.huggingface.co/v1/production/uploads/63be286fb3b8c44f8cecc16f/1CIkfEKoTnBYdYDSuQ8AT.jpeg
Yudong Wang
BigDong
user
BigDong
698b18981b2dc6b37d61b405
Zixuan Fu
698b18981b2dc6b37d61b406
Hengyu Zhao
698b18981b2dc6b37d61b407
Chen Zhao
698b18981b2dc6b37d61b408
Chuyue Zhou
claimed_verified
2026-02-12T13:57:50.598Z
6419b96fb4adb0e101b13587
/avatars/42686176a092e8738ce43d48d6941160.svg
ZhouChuYue
chuyue
user
chuyue
698b18981b2dc6b37d61b409
Xinle Lin
698b18981b2dc6b37d61b40a
Hongya Lyu
698b18981b2dc6b37d61b40b
Shuaikang Xue
698b18981b2dc6b37d61b40c
Yi Yi
698b18981b2dc6b37d61b40d
Yingjiao Wang
698b18981b2dc6b37d61b40e
Zhi Zheng
claimed_verified
2026-02-12T13:57:52.673Z
62d0d2f1c1d35087f508fbea
https://cdn-avatars.huggingface.co/v1/production/uploads/1676873251372-62d0d2f1c1d35087f508fbea.jpeg
Zhi Zheng
neoz
user
neoz
698b18981b2dc6b37d61b40f
Yuzhou Zhang
698b18981b2dc6b37d61b410
Jie Zhou
698b18981b2dc6b37d61b411
Chaojun Xiao
698b18981b2dc6b37d61b412
Xu Han
698b18981b2dc6b37d61b413
Zhiyuan Liu
698b18981b2dc6b37d61b414
Maosong Sun
Data Science and Technology Towards AGI Part I: Tiered Data Management
The development of artificial intelligence can be viewed as an evolution of data-driven learning paradigms, with successive shifts in data organization and utilization continuously driving advances in model capability. Current LLM research is dominated by a paradigm that relies heavily on unidirectional scaling of data size, increasingly encountering bottlenecks in data availability, acquisition cost, and training efficiency. In this work, we argue that the development of AGI is entering a new phase of data-model co-evolution, in which models actively guide data management while high-quality data, in turn, amplifies model capabilities. To implement this vision, we propose a tiered data management framework, designed to support the full LLM training lifecycle across heterogeneous learning objectives and cost constraints. Specifically, we introduce an L0-L4 tiered data management framework, ranging from raw uncurated resources to organized and verifiable knowledge. Importantly, LLMs are fully used in data management processes, such as quality scoring and content editing, to refine data across tiers. Each tier is characterized by distinct data properties, management strategies, and training roles, enabling data to be strategically allocated across LLM training stages, including pre-training, mid-training, and alignment. The framework balances data quality, acquisition cost, and marginal training benefit, providing a systematic approach to scalable and sustainable data management. We validate the effectiveness of the proposed framework through empirical studies, in which tiered datasets are constructed from raw corpora and used across multiple training phases. Experimental results demonstrate that tier-aware data utilization significantly improves training efficiency and model performance. To facilitate further research, we release our tiered datasets and processing tools to the community.
Large language models are increasingly guiding data management processes through a tiered framework that optimizes data quality, cost, and training efficiency across different stages of model development.
large language models data management data quality training efficiency tiered data management data curation model-guided data refinement pre-training mid-training alignment