Toward Autonomous Long-Horizon Engineering for ML Research
Autonomous AI research has advanced rapidly, but long-horizon ML research engineering remains difficult: agents must sustain coherent progress across task comprehension, environment setup, implementation, experimentation, and debugging over hours or days. We introduce AiScientist, a system for autonomous long-horizon engineering for ML research built on a simple principle: strong long-horizon performance requires both structured orchestration and durable state continuity. To this end, AiScientist combines hierarchical orchestration with a permission-scoped File-as-Bus workspace: a top-level Orchestrator maintains stage-level control through concise summaries and a workspace map, while specialized agents repeatedly re-ground on durable artifacts such as analyses, plans, code, and experimental evidence rather than relying primarily on conversational handoffs, yielding thin control over thick state. Across two complementary benchmarks, AiScientist improves PaperBench score by 10.54 points on average over the best matched baseline and achieves 81.82 Any Medal% on MLE-Bench Lite. Ablation studies further show that File-as-Bus protocol is a key driver of performance, reducing PaperBench by 6.41 points and MLE-Bench Lite by 31.82 points when removed. These results suggest that long-horizon ML research engineering is a systems problem of coordinating specialized work over durable project state, rather than a purely local reasoning problem.
69def7c2d3c84fcbf29215a8
Guoxin Chen
69def7c2d3c84fcbf29215a9
Jie Chen
claimed_verified
2026-04-16T08:03:26.894Z
651a29d566e78720a78317ec
https://cdn-avatars.huggingface.co/v1/production/uploads/651a29d566e78720a78317ec/WKPcw6Ziqjl44pkrHtCVa.jpeg
Jie Chen
survivi
user
survivi
69def7c2d3c84fcbf29215aa
Lei Chen
69def7c2d3c84fcbf29215ab
Jiale Zhao
69def7c2d3c84fcbf29215ac
Fanzhe Meng
69def7c2d3c84fcbf29215ad
Wayne Xin Zhao
69def7c2d3c84fcbf29215ae
Ruihua Song
69def7c2d3c84fcbf29215af
Cheng Chen
69def7c2d3c84fcbf29215b0
Ji-Rong Wen
69def7c2d3c84fcbf29215b1
Kai Jia
Toward Autonomous Long-Horizon Engineering for ML Research
Autonomous AI research has advanced rapidly, but long-horizon ML research engineering remains difficult: agents must sustain coherent progress across task comprehension, environment setup, implementation, experimentation, and debugging over hours or days. We introduce AiScientist, a system for autonomous long-horizon engineering for ML research built on a simple principle: strong long-horizon performance requires both structured orchestration and durable state continuity. To this end, AiScientist combines hierarchical orchestration with a permission-scoped File-as-Bus workspace: a top-level Orchestrator maintains stage-level control through concise summaries and a workspace map, while specialized agents repeatedly re-ground on durable artifacts such as analyses, plans, code, and experimental evidence rather than relying primarily on conversational handoffs, yielding thin control over thick state. Across two complementary benchmarks, AiScientist improves PaperBench score by 10.54 points on average over the best matched baseline and achieves 81.82 Any Medal% on MLE-Bench Lite. Ablation studies further show that File-as-Bus protocol is a key driver of performance, reducing PaperBench by 6.41 points and MLE-Bench Lite by 31.82 points when removed. These results suggest that long-horizon ML research engineering is a systems problem of coordinating specialized work over durable project state, rather than a purely local reasoning problem.
AiScientist enables autonomous long-horizon ML research engineering by combining hierarchical orchestration with durable state management, achieving superior performance on benchmark tasks through structured coordination and persistent project artifacts.
autonomous AI research long-horizon ML research engineering hierarchical orchestration File-as-Bus workspace durable state continuity specialized agents experimental evidence PaperBench MLE-Bench Lite