Less is Enough: Synthesizing Diverse Data in Feature Space of LLMs
The diversity of post-training data is critical for effective downstream performance in large language models (LLMs). Many existing approaches to constructing post-training data quantify diversity using text-based metrics that capture linguistic variation, but such metrics provide only weak signals for the task-relevant features that determine downstream performance. In this work, we introduce Feature Activation Coverage (FAC) which measures data diversity in an interpretable feature space. Building upon this metric, we further propose a diversity-driven data synthesis framework, named FAC Synthesis, that first uses a sparse autoencoder to identify missing features from a seed dataset, and then generates synthetic samples that explicitly reflect these features. Experiments show that our approach consistently improves both data diversity and downstream performance on various tasks, including instruction following, toxicity detection, reward modeling, and behavior steering. Interestingly, we identify a shared, interpretable feature space across model families (i.e., LLaMA, Mistral, and Qwen), enabling cross-model knowledge transfer. Our work provides a solid and practical methodology for exploring data-centric optimization of LLMs.
698d3bd265c0d15a6d16200e
Zhongzhi Li
claimed_verified
2026-02-12T13:57:05.580Z
6951c555b519522f565dfd0c
/avatars/9028d619483f359639ae7bfe4769da45.svg
ZhongzhiLi
Zhongzhi1228
user
Zhongzhi1228
698d3bd265c0d15a6d16200f
Xuansheng Wu
698d3bd265c0d15a6d162010
Yijiang Li
698d3bd265c0d15a6d162011
Lijie Hu
698d3bd265c0d15a6d162012
Ninghao Liu
Less is Enough: Synthesizing Diverse Data in Feature Space of LLMs
The diversity of post-training data is critical for effective downstream performance in large language models (LLMs). Many existing approaches to constructing post-training data quantify diversity using text-based metrics that capture linguistic variation, but such metrics provide only weak signals for the task-relevant features that determine downstream performance. In this work, we introduce Feature Activation Coverage (FAC) which measures data diversity in an interpretable feature space. Building upon this metric, we further propose a diversity-driven data synthesis framework, named FAC Synthesis, that first uses a sparse autoencoder to identify missing features from a seed dataset, and then generates synthetic samples that explicitly reflect these features. Experiments show that our approach consistently improves both data diversity and downstream performance on various tasks, including instruction following, toxicity detection, reward modeling, and behavior steering. Interestingly, we identify a shared, interpretable feature space across model families (i.e., LLaMA, Mistral, and Qwen), enabling cross-model knowledge transfer. Our work provides a solid and practical methodology for exploring data-centric optimization of LLMs.
Feature Activation Coverage measures data diversity in an interpretable feature space and enables diversity-driven data synthesis that improves downstream performance across multiple language model architectures.
Feature Activation Coverage sparse autoencoder data diversity downstream performance instruction following toxicity detection reward modeling behavior steering cross-model knowledge transfer data-centric optimization