Language-based Trial and Error Falls Behind in the Era of Experience
While Large Language Models (LLMs) excel in language-based agentic tasks, their applicability to unseen, nonlinguistic environments (e.g., symbolic or spatial tasks) remains limited. Previous work attributes this performance gap to the mismatch between the pretraining distribution and the testing distribution. In this work, we demonstrate the primary bottleneck is the prohibitive cost of exploration: mastering these tasks requires extensive trial-and-error, which is computationally unsustainable for parameter-heavy LLMs operating in a high dimensional semantic space. To address this, we propose SCOUT (Sub-Scale Collaboration On Unseen Tasks), a novel framework that decouples exploration from exploitation. We employ lightweight "scouts" (e.g., small MLPs) to probe environmental dynamics at a speed and scale far exceeding LLMs. The collected trajectories are utilized to bootstrap the LLM via Supervised Fine-Tuning (SFT), followed by multi-turn Reinforcement Learning (RL) to activate its latent world knowledge. Empirically, SCOUT enables a Qwen2.5-3B-Instruct model to achieve an average score of 0.86, significantly outperforming proprietary models, including Gemini-2.5-Pro (0.60), while saving about 60% GPU hours consumption.
697c1b0da67238fac88cc08d
Haoyu Wang
claimed_verified
2026-01-30T09:36:27.856Z
645503e102912fad3f2202fb
/avatars/87d8c80f4ff7920d990be3c3b94dfc64.svg
Haoyu Wang
Harryis
user
Harryis
697c1b0da67238fac88cc08e
Guozheng Ma
697c1b0da67238fac88cc08f
Shugang Cui
697c1b0da67238fac88cc090
Yilun Kong
697c1b0da67238fac88cc091
Haotian Luo
697c1b0da67238fac88cc092
Li Shen
697c1b0da67238fac88cc093
Mengya Gao
697c1b0da67238fac88cc094
Yichao Wu
697c1b0da67238fac88cc095
Xiaogang Wang
697c1b0da67238fac88cc096
Dacheng Tao
Language-based Trial and Error Falls Behind in the Era of Experience
While Large Language Models (LLMs) excel in language-based agentic tasks, their applicability to unseen, nonlinguistic environments (e.g., symbolic or spatial tasks) remains limited. Previous work attributes this performance gap to the mismatch between the pretraining distribution and the testing distribution. In this work, we demonstrate the primary bottleneck is the prohibitive cost of exploration: mastering these tasks requires extensive trial-and-error, which is computationally unsustainable for parameter-heavy LLMs operating in a high dimensional semantic space. To address this, we propose SCOUT (Sub-Scale Collaboration On Unseen Tasks), a novel framework that decouples exploration from exploitation. We employ lightweight "scouts" (e.g., small MLPs) to probe environmental dynamics at a speed and scale far exceeding LLMs. The collected trajectories are utilized to bootstrap the LLM via Supervised Fine-Tuning (SFT), followed by multi-turn Reinforcement Learning (RL) to activate its latent world knowledge. Empirically, SCOUT enables a Qwen2.5-3B-Instruct model to achieve an average score of 0.86, significantly outperforming proprietary models, including Gemini-2.5-Pro (0.60), while saving about 60% GPU hours consumption.
A novel framework called SCOUT is introduced that uses lightweight scouts to reduce exploration costs for large language models in nonlinguistic environments, enabling improved performance through supervised fine-tuning and reinforcement learning.
Large Language Models agentic tasks pretraining distribution testing distribution exploration cost trial-and-error parameter-heavy LLMs high dimensional semantic space lightweight scouts MLPs environmental dynamics Supervised Fine-Tuning reinforcement learning Qwen2.5-3B-Instruct Gemini-2.5-Pro