Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic
  Thought Reward
Large language models (LLMs) exhibit remarkable problem-solving abilities,
but struggle with complex tasks due to static internal knowledge.
Retrieval-Augmented Generation (RAG) enhances access to external information,
yet remains limited in multi-hop reasoning and strategic search due to rigid
workflows. Recent advancements in agentic deep research empower LLMs to
autonomously reason, search, and synthesize information. However, current
approaches relying on outcome-based reinforcement learning (RL) face critical
issues such as conflicting gradients and reward sparsity, limiting performance
gains and training efficiency. To address these, we first propose Atomic
Thought, a novel LLM thinking paradigm that decomposes reasoning into
fine-grained functional units. These units are supervised by Reasoning Reward
Models (RRMs), which provide Atomic Thought Rewards (ATR) for fine-grained
guidance. Building on this, we propose Atom-Searcher, a novel RL framework for
agentic deep research that integrates Atomic Thought and ATR. Atom-Searcher
uses a curriculum-inspired reward schedule, prioritizing process-level ATR
early and transitioning to outcome rewards, accelerating convergence on
effective reasoning paths. Experiments on seven benchmarks show consistent
improvements over the state-of-the-art. Key advantages include: (1)
Atom-Searcher scales computation at test-time. (2) Atomic Thought provides
supervision anchors for RRMs, bridging deep research tasks and RRMs. (3)
Atom-Searcher exhibits more interpretable, human-like reasoning patterns.
68a55dc46cf0bf898542ecf6
Yong Deng
claimed_verified
2025-08-20T08:51:11.768Z
63ea134a43d976de6e4e8ea7
https://cdn-avatars.huggingface.co/v1/production/uploads/63ea134a43d976de6e4e8ea7/OsyE7SYDW9foqsyWXtTiL.png
deng yong
dikw
user
dikw
68a55dc46cf0bf898542ecf7
Guoqing Wang
68a55dc46cf0bf898542ecf8
Zhenzhe Ying
68a55dc46cf0bf898542ecf9
Xiaofeng Wu
68a55dc46cf0bf898542ecfa
Jinzhen Lin
68a55dc46cf0bf898542ecfb
Wenwen Xiong
68a55dc46cf0bf898542ecfc
Yuqin Dai
claimed_verified
2026-02-09T08:38:43.666Z
668547e921e9f68a7a2a6d18
/avatars/5251e5c52993c7e5fb23cb4afba03f50.svg
Yuqin Dai
dayll
user
dayll
68a55dc46cf0bf898542ecfd
Shuo Yang
68a55dc46cf0bf898542ecfe
Zhanwei Zhang
68a55dc46cf0bf898542ecff
Qiwen Wang
68a55dc46cf0bf898542ed00
Yang Qin
68a55dc46cf0bf898542ed01
Changhua Meng
Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic
  Thought Reward
Large language models (LLMs) exhibit remarkable problem-solving abilities,
but struggle with complex tasks due to static internal knowledge.
Retrieval-Augmented Generation (RAG) enhances access to external information,
yet remains limited in multi-hop reasoning and strategic search due to rigid
workflows. Recent advancements in agentic deep research empower LLMs to
autonomously reason, search, and synthesize information. However, current
approaches relying on outcome-based reinforcement learning (RL) face critical
issues such as conflicting gradients and reward sparsity, limiting performance
gains and training efficiency. To address these, we first propose Atomic
Thought, a novel LLM thinking paradigm that decomposes reasoning into
fine-grained functional units. These units are supervised by Reasoning Reward
Models (RRMs), which provide Atomic Thought Rewards (ATR) for fine-grained
guidance. Building on this, we propose Atom-Searcher, a novel RL framework for
agentic deep research that integrates Atomic Thought and ATR. Atom-Searcher
uses a curriculum-inspired reward schedule, prioritizing process-level ATR
early and transitioning to outcome rewards, accelerating convergence on
effective reasoning paths. Experiments on seven benchmarks show consistent
improvements over the state-of-the-art. Key advantages include: (1)
Atom-Searcher scales computation at test-time. (2) Atomic Thought provides
supervision anchors for RRMs, bridging deep research tasks and RRMs. (3)
Atom-Searcher exhibits more interpretable, human-like reasoning patterns.
Atom-Searcher, an RL framework integrating Atomic Thought and Reasoning Reward Models, enhances LLMs' multi-hop reasoning and strategic search capabilities, improving performance and interpretability.
Retrieval-Augmented Generation RAG agentic deep research reinforcement learning RL Atomic Thought Reasoning Reward Models RRMs Atomic Thought Rewards ATR curriculum-inspired reward schedule multi-hop reasoning strategic search interpretable reasoning