FROST: Filtering Reasoning Outliers with Attention for Efficient Reasoning
We propose FROST, an attention-aware method for efficient reasoning. Unlike traditional approaches, FROST leverages attention weights to prune uncritical reasoning paths, yielding shorter and more reliable reasoning trajectories. Methodologically, we introduce the concept of reasoning outliers and design an attention-based mechanism to remove them. Theoretically, FROST preserves and enhances the model's reasoning capacity while eliminating outliers at the sentence level. Empirically, we validate FROST on four benchmarks using two strong reasoning models (Phi-4-Reasoning and GPT-OSS-20B), outperforming state-of-the-art methods such as TALE and ThinkLess. Notably, FROST achieves an average 69.68% reduction in token usage and a 26.70% improvement in accuracy over the base model. Furthermore, in evaluations of attention outlier metrics, FROST reduces the maximum infinity norm by 15.97% and the average kurtosis by 91.09% compared to the base model. Code is available at https://github.com/robinzixuan/FROST
697c6d70a67238fac88cc28e
Haozheng Luo
697c6d70a67238fac88cc28f
Zhuolin Jiang
697c6d70a67238fac88cc290
Md Zahid Hasan
claimed_verified
2026-03-23T11:01:46.311Z
62e8533ce69c50a7c5940730
/avatars/dc8402d88ce84bacebc20588eb3ca288.svg
Zahid Hasan
zahidh
user
zahidh
697c6d70a67238fac88cc291
Yan Chen
697c6d70a67238fac88cc292
Soumalya Sarkar
FROST: Filtering Reasoning Outliers with Attention for Efficient Reasoning
We propose FROST, an attention-aware method for efficient reasoning. Unlike traditional approaches, FROST leverages attention weights to prune uncritical reasoning paths, yielding shorter and more reliable reasoning trajectories. Methodologically, we introduce the concept of reasoning outliers and design an attention-based mechanism to remove them. Theoretically, FROST preserves and enhances the model's reasoning capacity while eliminating outliers at the sentence level. Empirically, we validate FROST on four benchmarks using two strong reasoning models (Phi-4-Reasoning and GPT-OSS-20B), outperforming state-of-the-art methods such as TALE and ThinkLess. Notably, FROST achieves an average 69.68% reduction in token usage and a 26.70% improvement in accuracy over the base model. Furthermore, in evaluations of attention outlier metrics, FROST reduces the maximum infinity norm by 15.97% and the average kurtosis by 91.09% compared to the base model. Code is available at https://github.com/robinzixuan/FROST
FROST is an attention-aware method that improves reasoning efficiency by pruning uncritical paths and removing reasoning outliers, leading to reduced token usage and improved accuracy.
attention weights reasoning paths attention-based mechanism reasoning outliers attention outlier metrics token usage kurtosis infinity norm