Adam's Law: Textual Frequency Law on Large Language Models
While textual frequency has been validated as relevant to human cognition in reading speed, its relatedness to Large Language Models (LLMs) is seldom studied. We propose a novel research direction in terms of textual data frequency, which is an understudied topic, to the best of our knowledge. Our framework is composed of three units. First, this paper proposes Textual Frequency Law (TFL), which indicates that frequent textual data should be preferred for LLMs for both prompting and fine-tuning. Since many LLMs are closed-source in their training data, we propose using online resources to estimate the sentence-level frequency. We then utilize an input paraphraser to paraphrase the input into a more frequent textual expression. Next, we propose Textual Frequency Distillation (TFD) by querying LLMs to conduct story completion by further extending the sentences in the datasets, and the resulting corpora are used to adjust the initial estimation. Finally, we propose Curriculum Textual Frequency Training (CTFT) that fine-tunes LLMs in an increasing order of sentence-level frequency. Experiments are conducted on our curated dataset Textual Frequency Paired Dataset (TFPD) on math reasoning, machine translation, commonsense reasoning and agentic tool calling. Results show the effectiveness of our framework.
69cf92fee02fdd68b91b7c1b
Hongyuan Adam Lu
claimed_verified
2026-04-07T08:44:58.634Z
645b949b4093d5c140e12849
/avatars/b296f072f2578b846517517645bf9267.svg
lu
hongyyyyy
user
hongyyyyy
69cf92fee02fdd68b91b7c1c
Z. L.
69cf92fee02fdd68b91b7c1d
Victor Wei
69cf92fee02fdd68b91b7c1e
Zefan Zhang
69cf92fee02fdd68b91b7c1f
Zhao Hong
69cf92fee02fdd68b91b7c20
Qiqi Xiang
69cf92fee02fdd68b91b7c21
Bowen Cao
69cf92fee02fdd68b91b7c22
Wai Lam
Adam's Law: Textual Frequency Law on Large Language Models
While textual frequency has been validated as relevant to human cognition in reading speed, its relatedness to Large Language Models (LLMs) is seldom studied. We propose a novel research direction in terms of textual data frequency, which is an understudied topic, to the best of our knowledge. Our framework is composed of three units. First, this paper proposes Textual Frequency Law (TFL), which indicates that frequent textual data should be preferred for LLMs for both prompting and fine-tuning. Since many LLMs are closed-source in their training data, we propose using online resources to estimate the sentence-level frequency. We then utilize an input paraphraser to paraphrase the input into a more frequent textual expression. Next, we propose Textual Frequency Distillation (TFD) by querying LLMs to conduct story completion by further extending the sentences in the datasets, and the resulting corpora are used to adjust the initial estimation. Finally, we propose Curriculum Textual Frequency Training (CTFT) that fine-tunes LLMs in an increasing order of sentence-level frequency. Experiments are conducted on our curated dataset Textual Frequency Paired Dataset (TFPD) on math reasoning, machine translation, commonsense reasoning and agentic tool calling. Results show the effectiveness of our framework.
A novel framework for improving large language model performance through textual frequency analysis, including laws, distillation, and curriculum training approaches.
Textual Frequency Law Textual Frequency Distillation Curriculum Textual Frequency Training Large Language Models prompting fine-tuning sentence-level frequency input paraphraser story completion agentic tool calling math reasoning machine translation commonsense reasoning