Toward Ultra-Long-Horizon Agentic Science: Cognitive Accumulation for Machine Learning Engineering
The advancement of artificial intelligence toward agentic science is currently bottlenecked by the challenge of ultra-long-horizon autonomy, the ability to sustain strategic coherence and iterative correction over experimental cycles spanning days or weeks. While Large Language Models (LLMs) have demonstrated prowess in short-horizon reasoning, they are easily overwhelmed by execution details in the high-dimensional, delayed-feedback environments of real-world research, failing to consolidate sparse feedback into coherent long-term guidance. Here, we present ML-Master 2.0, an autonomous agent that masters ultra-long-horizon machine learning engineering (MLE) which is a representative microcosm of scientific discovery. By reframing context management as a process of cognitive accumulation, our approach introduces Hierarchical Cognitive Caching (HCC), a multi-tiered architecture inspired by computer systems that enables the structural differentiation of experience over time. By dynamically distilling transient execution traces into stable knowledge and cross-task wisdom, HCC allows agents to decouple immediate execution from long-term experimental strategy, effectively overcoming the scaling limits of static context windows. In evaluations on OpenAI's MLE-Bench under 24-hour budgets, ML-Master 2.0 achieves a state-of-the-art medal rate of 56.44%. Our findings demonstrate that ultra-long-horizon autonomy provides a scalable blueprint for AI capable of autonomous exploration beyond human-precedent complexities.
6969f54c844a787c4fdea404
Xinyu Zhu
claimed_verified
2026-01-16T10:28:31.884Z
65352517fca2c10e43035e32
/avatars/f404cef8a79f27435865ac4d85ef0927.svg
xinyuzhu
xinyuzhu
user
xinyuzhu
6969f54c844a787c4fdea405
Yuzhu Cai
claimed_verified
2026-01-16T14:36:59.231Z
6614ed4809c63bfbddc53ddf
/avatars/018bfc168bb4010cf6018e42148e0f51.svg
Yuzhu Cai
Ethical-Lens
user
Ethical-Lens
6969f54c844a787c4fdea406
Zexi Liu
claimed_verified
2026-01-16T10:28:33.861Z
651e1f5522890326303fb1f4
/avatars/4e47b8d7669c3aafcab4e1438ed386e7.svg
Zexi Liu
ZeroXLeo
user
ZeroXLeo
6969f54c844a787c4fdea407
Bingyang Zheng
admin_assigned
2026-01-16T15:38:51.672Z
690320c2eba76a9b99854b4d
/avatars/a612962c3627bdb7986a5ab5f14af2eb.svg
Bingyang Zheng
bulibuliyang
user
bulibuliyang
6969f54c844a787c4fdea408
Cheng Wang
6969f54c844a787c4fdea409
Rui Ye
claimed_verified
2026-03-17T08:31:40.234Z
63ecd42a60ff4b318ad1ef47
/avatars/c05001fa9c41f713d1fe31d11212faae.svg
RuiYe
ruiye-sjtu
user
ruiye-sjtu
6969f54c844a787c4fdea40a
Jiaao Chen
admin_assigned
2026-01-16T15:39:08.324Z
64f793d4dcd7b028c15bbe50
/avatars/fb5d7392736309dd5c80ac32750d164b.svg
Jiaao Chen
Jiaaoc
user
Jiaaoc
6969f54c844a787c4fdea40b
Hanrui Wang
admin_assigned
2026-01-16T15:39:14.281Z
677e237c70bd7b5431f88450
/avatars/cc64b0af06588e43c94c185900362751.svg
Hanrui Wang
azrealwang
user
azrealwang
6969f54c844a787c4fdea40c
Wei-Chen Wang
admin_assigned
2026-01-16T15:39:21.035Z
68c08c9da619a2f12eae4913
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/FYHEbHaFSOcWScBmeHQcF.png
Wei Chen Wang
bumbigby
user
bumbigby
6969f54c844a787c4fdea40d
Yuzhi Zhang
6969f54c844a787c4fdea40e
Linfeng Zhang
6969f54c844a787c4fdea40f
Weinan E
6969f54c844a787c4fdea410
Di Jin
6969f54c844a787c4fdea411
Siheng Chen
admin_assigned
2026-01-16T15:39:39.159Z
65257545b017be1fc1915364
/avatars/9bffd3fb567d2fa1e5c3546d77560b43.svg
Siheng Chen
sihengchen
user
sihengchen
Toward Ultra-Long-Horizon Agentic Science: Cognitive Accumulation for Machine Learning Engineering
The advancement of artificial intelligence toward agentic science is currently bottlenecked by the challenge of ultra-long-horizon autonomy, the ability to sustain strategic coherence and iterative correction over experimental cycles spanning days or weeks. While Large Language Models (LLMs) have demonstrated prowess in short-horizon reasoning, they are easily overwhelmed by execution details in the high-dimensional, delayed-feedback environments of real-world research, failing to consolidate sparse feedback into coherent long-term guidance. Here, we present ML-Master 2.0, an autonomous agent that masters ultra-long-horizon machine learning engineering (MLE) which is a representative microcosm of scientific discovery. By reframing context management as a process of cognitive accumulation, our approach introduces Hierarchical Cognitive Caching (HCC), a multi-tiered architecture inspired by computer systems that enables the structural differentiation of experience over time. By dynamically distilling transient execution traces into stable knowledge and cross-task wisdom, HCC allows agents to decouple immediate execution from long-term experimental strategy, effectively overcoming the scaling limits of static context windows. In evaluations on OpenAI's MLE-Bench under 24-hour budgets, ML-Master 2.0 achieves a state-of-the-art medal rate of 56.44%. Our findings demonstrate that ultra-long-horizon autonomy provides a scalable blueprint for AI capable of autonomous exploration beyond human-precedent complexities.
ML-Master 2.0 enables long-term autonomous machine learning engineering through hierarchical cognitive caching that manages extended context and learns from execution traces.
Large Language Models ultra-long-horizon autonomy machine learning engineering Hierarchical Cognitive Caching cognitive accumulation context management experimental strategy execution traces cross-task wisdom