Synthetic Sandbox for Training Machine Learning Engineering Agents
As large language model agents advance beyond software engineering (SWE) tasks toward machine learning engineering (MLE), verifying agent behavior becomes orders of magnitude more expensive: while SWE tasks can be verified via fast-executing unit tests, MLE verification requires running full ML pipelines -- data preprocessing, model training, and metric evaluation -- on large datasets at each rollout step, rendering trajectory-wise on-policy reinforcement learning (RL) prohibitively slow. Existing approaches retreat to supervised fine-tuning (SFT) or offline proxy rewards, sacrificing the exploration and generalization benefits of on-policy RL. We observe that sandbox data size is the primary source of this bottleneck. Based on this insight, we introduce SandMLE, a multi-agent framework that generates diverse, verifiable synthetic MLE environments from a small number of seed tasks, preserving the structural and technical complexity of real-world problems while constraining datasets to micro-scale (each task is paired with only 50-200 training samples). Through extensive experiments, we show that SandMLE reduces execution time by over 13 times, enabling large-scale, on-policy trajectory-wise RL for the first time in the MLE domain. On MLE-bench-lite, SandMLE yields significant gains over SFT baselines across Qwen3-8B, 14B, and 30B-A3B, with relative medal rate improvements ranging from 20.3% to 66.9%. Furthermore, the trained policy generalizes across unseen agentic scaffolds, achieving up to 32.4% better HumanRank score on MLE-Dojo.
69d5149763f089495f06b238
Yuhang Zhou
claimed_verified
2026-04-08T08:06:15.874Z
64887eb15cf73a16e767b56a
/avatars/ada2b6a07346b1d61322ddd04d219318.svg
Yuhang Zhou
zyhang1998
user
zyhang1998
69d5149763f089495f06b239
Lizhu Zhang
69d5149763f089495f06b23a
Yifan Wu
69d5149763f089495f06b23b
Jiayi Liu
69d5149763f089495f06b23c
Xiangjun Fan
69d5149763f089495f06b23d
Zhuokai Zhao
claimed_verified
2026-04-08T08:06:13.601Z
63af25605fe9db73f67a0fb7
https://cdn-avatars.huggingface.co/v1/production/uploads/63af25605fe9db73f67a0fb7/mcQHbTdJVxi2GdHzRB9ad.jpeg
Zhuokai Zhao
zhuokai
user
zhuokai
69d5149763f089495f06b23e
Hong Yan
Synthetic Sandbox for Training Machine Learning Engineering Agents
As large language model agents advance beyond software engineering (SWE) tasks toward machine learning engineering (MLE), verifying agent behavior becomes orders of magnitude more expensive: while SWE tasks can be verified via fast-executing unit tests, MLE verification requires running full ML pipelines -- data preprocessing, model training, and metric evaluation -- on large datasets at each rollout step, rendering trajectory-wise on-policy reinforcement learning (RL) prohibitively slow. Existing approaches retreat to supervised fine-tuning (SFT) or offline proxy rewards, sacrificing the exploration and generalization benefits of on-policy RL. We observe that sandbox data size is the primary source of this bottleneck. Based on this insight, we introduce SandMLE, a multi-agent framework that generates diverse, verifiable synthetic MLE environments from a small number of seed tasks, preserving the structural and technical complexity of real-world problems while constraining datasets to micro-scale (each task is paired with only 50-200 training samples). Through extensive experiments, we show that SandMLE reduces execution time by over 13 times, enabling large-scale, on-policy trajectory-wise RL for the first time in the MLE domain. On MLE-bench-lite, SandMLE yields significant gains over SFT baselines across Qwen3-8B, 14B, and 30B-A3B, with relative medal rate improvements ranging from 20.3% to 66.9%. Furthermore, the trained policy generalizes across unseen agentic scaffolds, achieving up to 32.4% better HumanRank score on MLE-Dojo.
A multi-agent framework called SandMLE is introduced that generates synthetic machine learning engineering environments from limited seed tasks, enabling efficient on-policy reinforcement learning by reducing dataset sizes and computational costs while maintaining problem complexity.
large language model agents machine learning engineering reinforcement learning supervised fine-tuning on-policy reinforcement learning multi-agent framework synthetic environments MLE-bench-lite MLE-Dojo HumanRank score