TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Exploration
While Large Language Models (LLMs) have empowered AI research agents to perform isolated scientific tasks, automating complex, real-world workflows, such as LLM training, remains a significant challenge. In this paper, we introduce TREX, a multi-agent system that automates the entire LLM training life-cycle. By orchestrating collaboration between two core modules-the Researcher and the Executor-the system seamlessly performs requirement analysis, open-domain literature and data research, formulation of training strategies, preparation of data recipes, and model training and evaluation. The multi-round experimental process is modeled as a search tree, enabling the system to efficiently plan exploration paths, reuse historical results, and distill high-level insights from iterative trials. To evaluate the capability of automated LLM training, we construct FT-Bench, a benchmark comprising 10 tasks derived from real-world scenarios, ranging from optimizing fundamental model capabilities to enhancing performance on domain-specific tasks. Experimental results demonstrate that the TREX agent consistently optimizes model performance on target tasks.
69e03fa75b704445ba50b594
Zerun Ma
claimed_verified
2026-04-16T08:01:52.828Z
634fb993685e43e02d2544a2
https://cdn-avatars.huggingface.co/v1/production/uploads/634fb993685e43e02d2544a2/L9MuXYlXMBuQgnwXd4lCU.jpeg
mzr1996
mzr1996
user
mzr1996
69e03fa75b704445ba50b595
Guoqiang Wang
69e03fa75b704445ba50b596
Xinchen Xie
69e03fa75b704445ba50b597
Yicheng Chen
69e03fa75b704445ba50b598
He Du
69e03fa75b704445ba50b599
Bowen Li
69e03fa75b704445ba50b59a
Yanan Sun
69e03fa75b704445ba50b59b
Wenran Liu
69e03fa75b704445ba50b59c
Kai Chen
69e03fa75b704445ba50b59d
Yining Li
claimed_verified
2026-04-16T20:21:41.707Z
649988726677f66c2b486392
/avatars/b649a77370660e129726e29504daba34.svg
Yining Li
ly015
user
ly015
TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Exploration
While Large Language Models (LLMs) have empowered AI research agents to perform isolated scientific tasks, automating complex, real-world workflows, such as LLM training, remains a significant challenge. In this paper, we introduce TREX, a multi-agent system that automates the entire LLM training life-cycle. By orchestrating collaboration between two core modules-the Researcher and the Executor-the system seamlessly performs requirement analysis, open-domain literature and data research, formulation of training strategies, preparation of data recipes, and model training and evaluation. The multi-round experimental process is modeled as a search tree, enabling the system to efficiently plan exploration paths, reuse historical results, and distill high-level insights from iterative trials. To evaluate the capability of automated LLM training, we construct FT-Bench, a benchmark comprising 10 tasks derived from real-world scenarios, ranging from optimizing fundamental model capabilities to enhancing performance on domain-specific tasks. Experimental results demonstrate that the TREX agent consistently optimizes model performance on target tasks.
A multi-agent system automates the complete lifecycle of large language model training by coordinating research and execution modules through iterative planning and experimentation.
multi-agent system large language models automated training research agent execution agent search tree iterative trials benchmark FT-Bench