IterResearch: Rethinking Long-Horizon Agents via Markovian State
  Reconstruction
Recent advances in deep-research agents have shown promise for autonomous
knowledge construction through dynamic reasoning over external sources.
However, existing approaches rely on a mono-contextual paradigm that
accumulates all information in a single, expanding context window, leading to
context suffocation and noise contamination that limit their effectiveness on
long-horizon tasks. We introduce IterResearch, a novel iterative deep-research
paradigm that reformulates long-horizon research as a Markov Decision Process
with strategic workspace reconstruction. By maintaining an evolving report as
memory and periodically synthesizing insights, our approach preserves
consistent reasoning capacity across arbitrary exploration depths. We further
develop Efficiency-Aware Policy Optimization (EAPO), a reinforcement learning
framework that incentivizes efficient exploration through geometric reward
discounting and enables stable distributed training via adaptive downsampling.
Extensive experiments demonstrate that IterResearch achieves substantial
improvements over existing open-source agents with average +14.5pp across six
benchmarks and narrows the gap with frontier proprietary systems. Remarkably,
our paradigm exhibits unprecedented interaction scaling, extending to 2048
interactions with dramatic performance gains (from 3.5\% to 42.5\%), and serves
as an effective prompting strategy, improving frontier models by up to 19.2pp
over ReAct on long-horizon tasks. These findings position IterResearch as a
versatile solution for long-horizon reasoning, effective both as a trained
agent and as a prompting paradigm for frontier models.
6912b5eda644ba07c499c740
Guoxin Chen
claimed_verified
2025-11-11T19:44:10.134Z
63f06116f1a47aaea5bd497b
/avatars/7d99ffa59c4579599e852a0ffb261268.svg
Guoxin Chen
GuoxinChen
user
GuoxinChen
6912b5eda644ba07c499c741
Zile Qiao
6912b5eda644ba07c499c742
Xuanzhong Chen
claimed_verified
2025-11-11T19:44:08.086Z
65e6970d135c27ea806526fe
/avatars/4aced113d9cab055ae06f3945869a280.svg
Xuanzhong Chen
chenxz
user
chenxz
6912b5eda644ba07c499c743
Donglei Yu
6912b5eda644ba07c499c744
Haotian Xu
6912b5eda644ba07c499c745
Wayne Xin Zhao
6912b5eda644ba07c499c746
Ruihua Song
6912b5eda644ba07c499c747
Wenbiao Yin
6912b5eda644ba07c499c748
Huifeng Yin
6912b5eda644ba07c499c749
Liwen Zhang
6912b5eda644ba07c499c74a
Kuan Li
6912b5eda644ba07c499c74b
Minpeng Liao
6912b5eda644ba07c499c74c
Yong Jiang
6912b5eda644ba07c499c74d
Pengjun Xie
6912b5eda644ba07c499c74e
Fei Huang
6912b5eda644ba07c499c74f
Jingren Zhou
IterResearch: Rethinking Long-Horizon Agents via Markovian State
  Reconstruction
Recent advances in deep-research agents have shown promise for autonomous
knowledge construction through dynamic reasoning over external sources.
However, existing approaches rely on a mono-contextual paradigm that
accumulates all information in a single, expanding context window, leading to
context suffocation and noise contamination that limit their effectiveness on
long-horizon tasks. We introduce IterResearch, a novel iterative deep-research
paradigm that reformulates long-horizon research as a Markov Decision Process
with strategic workspace reconstruction. By maintaining an evolving report as
memory and periodically synthesizing insights, our approach preserves
consistent reasoning capacity across arbitrary exploration depths. We further
develop Efficiency-Aware Policy Optimization (EAPO), a reinforcement learning
framework that incentivizes efficient exploration through geometric reward
discounting and enables stable distributed training via adaptive downsampling.
Extensive experiments demonstrate that IterResearch achieves substantial
improvements over existing open-source agents with average +14.5pp across six
benchmarks and narrows the gap with frontier proprietary systems. Remarkably,
our paradigm exhibits unprecedented interaction scaling, extending to 2048
interactions with dramatic performance gains (from 3.5\% to 42.5\%), and serves
as an effective prompting strategy, improving frontier models by up to 19.2pp
over ReAct on long-horizon tasks. These findings position IterResearch as a
versatile solution for long-horizon reasoning, effective both as a trained
agent and as a prompting paradigm for frontier models.
IterResearch, an iterative deep-research paradigm, improves long-horizon reasoning by reformulating it as a Markov Decision Process with strategic workspace reconstruction and Efficiency-Aware Policy Optimization, achieving better performance and interaction scaling compared to existing agents.
deep-research agents dynamic reasoning mono-contextual paradigm context suffocation noise contamination long-horizon tasks IterResearch Markov Decision Process strategic workspace reconstruction evolving report Efficiency-Aware Policy Optimization reinforcement learning geometric reward discounting adaptive downsampling prompting strategy ReAct