PACEvolve: Enabling Long-Horizon Progress-Aware Consistent Evolution
Large Language Models (LLMs) have emerged as powerful operators for evolutionary search, yet the design of efficient search scaffolds remains ad hoc. While promising, current LLM-in-the-loop systems lack a systematic approach to managing the evolutionary process. We identify three distinct failure modes: Context Pollution, where experiment history biases future candidate generation; Mode Collapse, where agents stagnate in local minima due to poor exploration-exploitation balance; and Weak Collaboration, where rigid crossover strategies fail to leverage parallel search trajectories effectively. We introduce Progress-Aware Consistent Evolution (PACEvolve), a framework designed to robustly govern the agent's context and search dynamics, to address these challenges. PACEvolve combines hierarchical context management (HCM) with pruning to address context pollution; momentum-based backtracking (MBB) to escape local minima; and a self-adaptive sampling policy that unifies backtracking and crossover for dynamic search coordination (CE), allowing agents to balance internal refinement with cross-trajectory collaboration. We demonstrate that PACEvolve provides a systematic path to consistent, long-horizon self-improvement, achieving state-of-the-art results on LLM-SR and KernelBench, while discovering solutions surpassing the record on Modded NanoGPT.
6969aaf232f0333869ff9430
Minghao Yan
claimed_verified
2026-01-16T10:29:43.551Z
64de3c145e192985054c74c3
/avatars/bc07670392fa7726412d48e8c8c2c8c0.svg
Minghao Yan
minghaoyan
user
minghaoyan
6969aaf232f0333869ff9431
Bo Peng
6969aaf232f0333869ff9432
Benjamin Coleman
admin_assigned
2026-01-16T15:49:04.553Z
665a131a08cba81fb74d745a
/avatars/fc7e216b5abb556762edf4fcdbd9aad0.svg
Benjamin Coleman
BenjaminColeman305
user
BenjaminColeman305
6969aaf232f0333869ff9433
Ziqi Chen
6969aaf232f0333869ff9434
Zhouhang Xie
admin_assigned
2026-01-16T15:49:15.754Z
6187199bb1085ab638324e82
/avatars/02281e4e5dc260fff62de34589f42b7d.svg
zhouhang xie
zhohanx
user
zhohanx
6969aaf232f0333869ff9435
Zhankui He
admin_assigned
2026-01-16T15:49:24.778Z
64daab70c38427829daf5958
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/cjL27oSvuJf1x0Zq3SrEJ.jpeg
Zhankui He
ZhankuiHe
user
ZhankuiHe
6969aaf232f0333869ff9436
Noveen Sachdeva
6969aaf232f0333869ff9437
Isabella Ye
admin_assigned
2026-01-16T15:49:30.522Z
683aa7ffb13aecd29bddac74
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/rFG39yrXagxE15Yqr2NKb.png
Isabella Ye
isye4
user
isye4
6969aaf232f0333869ff9438
Weili Wang
6969aaf232f0333869ff9439
Chi Wang
6969aaf232f0333869ff943a
Ed H. Chi
6969aaf232f0333869ff943b
Wang-Cheng Kang
6969aaf232f0333869ff943c
Derek Zhiyuan Cheng
6969aaf232f0333869ff943d
Beidou Wang
PACEvolve: Enabling Long-Horizon Progress-Aware Consistent Evolution
Large Language Models (LLMs) have emerged as powerful operators for evolutionary search, yet the design of efficient search scaffolds remains ad hoc. While promising, current LLM-in-the-loop systems lack a systematic approach to managing the evolutionary process. We identify three distinct failure modes: Context Pollution, where experiment history biases future candidate generation; Mode Collapse, where agents stagnate in local minima due to poor exploration-exploitation balance; and Weak Collaboration, where rigid crossover strategies fail to leverage parallel search trajectories effectively. We introduce Progress-Aware Consistent Evolution (PACEvolve), a framework designed to robustly govern the agent's context and search dynamics, to address these challenges. PACEvolve combines hierarchical context management (HCM) with pruning to address context pollution; momentum-based backtracking (MBB) to escape local minima; and a self-adaptive sampling policy that unifies backtracking and crossover for dynamic search coordination (CE), allowing agents to balance internal refinement with cross-trajectory collaboration. We demonstrate that PACEvolve provides a systematic path to consistent, long-horizon self-improvement, achieving state-of-the-art results on LLM-SR and KernelBench, while discovering solutions surpassing the record on Modded NanoGPT.
PACEvolve framework addresses key failure modes in LLM evolutionary search through hierarchical context management, momentum-based backtracking, and adaptive sampling policies for improved self-improvement and solution discovery.
evolutionary search large language models context pollution mode collapse weak collaboration Progress-Aware Consistent Evolution hierarchical context management momentum-based backtracking self-adaptive sampling policy cross-trajectory collaboration