Graph-Based Chain-of-Thought Pruning for Reducing Redundant Reflections in Reasoning LLMs
Extending CoT through RL has been widely used to enhance the reasoning capabilities of LLMs. However, due to the sparsity of reward signals, it can also induce undesirable thinking patterns such as overthinking, i.e., generating redundant intermediate reasoning content. In this work, we argue that a major source of such redundancy is inefficient reflection, which often manifests in two problematic patterns: Indiscriminate Reflection, where the model performs broad, low-impact checks throughout reasoning, and Repetitive Reflection, where it repeatedly re-verifies an already established conclusion. To address this, we introduce a graph-based CoT optimization framework. Specifically, we convert each linear CoT into a directed acyclic graph (DAG) with explicit dependency edges, and design a dual pruning strategy: branch-level pruning removes weakly contributing reflection branches, while depth-level pruning eliminates late-stage re-verification. We distill this behavior via a three-stage pipeline: (1) SFT to initialize the policy on pruned concise traces, (2) DPO to prefer correct but less redundant trajectories, and (3) GRPO with length penalty to jointly optimize answer correctness and efficiency. Experiments show that our approach reduces the average reasoning tokens by 42\% while maintaining or improving accuracy.
69d79d5de33e601f8dfcd641
Hongyuan Yuan
69d79d5de33e601f8dfcd642
Xinran He
69d79d5de33e601f8dfcd643
Run Shao
69d79d5de33e601f8dfcd644
Bolei He
claimed_verified
2026-04-09T18:50:34.687Z
68aefd6482a3030535876c27
/avatars/8df724044e3087eea3628a8927b4da69.svg
HeBolei
HeBolei
user
HeBolei
69d79d5de33e601f8dfcd645
Xianwei Xue
69d79d5de33e601f8dfcd646
Mengke Chen
69d79d5de33e601f8dfcd647
Qiutong Pan
69d79d5de33e601f8dfcd648
Haiwei Wang
69d79d5de33e601f8dfcd649
Haifeng Li
Graph-Based Chain-of-Thought Pruning for Reducing Redundant Reflections in Reasoning LLMs
Extending CoT through RL has been widely used to enhance the reasoning capabilities of LLMs. However, due to the sparsity of reward signals, it can also induce undesirable thinking patterns such as overthinking, i.e., generating redundant intermediate reasoning content. In this work, we argue that a major source of such redundancy is inefficient reflection, which often manifests in two problematic patterns: Indiscriminate Reflection, where the model performs broad, low-impact checks throughout reasoning, and Repetitive Reflection, where it repeatedly re-verifies an already established conclusion. To address this, we introduce a graph-based CoT optimization framework. Specifically, we convert each linear CoT into a directed acyclic graph (DAG) with explicit dependency edges, and design a dual pruning strategy: branch-level pruning removes weakly contributing reflection branches, while depth-level pruning eliminates late-stage re-verification. We distill this behavior via a three-stage pipeline: (1) SFT to initialize the policy on pruned concise traces, (2) DPO to prefer correct but less redundant trajectories, and (3) GRPO with length penalty to jointly optimize answer correctness and efficiency. Experiments show that our approach reduces the average reasoning tokens by 42\% while maintaining or improving accuracy.
A graph-based framework optimizes chain-of-thought reasoning in large language models by identifying and eliminating redundant thinking patterns through structured pruning and reinforcement learning techniques.
chain-of-thought reinforcement learning reward signals overthinking reflection directed acyclic graph branch-level pruning depth-level pruning distillation supervised fine-tuning direct preference optimization generative reinforcement policy optimization length penalty