SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agents
Scientific reasoning inherently demands integrating sophisticated toolkits to navigate domain-specific knowledge. Yet, current benchmarks largely overlook agents' ability to orchestrate tools for such rigorous workflows. To bridge this gap, we introduce SciAgentGym, a scalable interactive environment featuring 1,780 domain-specific tools across four natural science disciplines, supported by a robust execution infrastructure. Complementing this, we present SciAgentBench, a tiered evaluation suite designed to stress-test agentic capabilities from elementary actions to long-horizon workflows. Our evaluation identifies a critical bottleneck: state-of-the-art models struggle with complex scientific tool-use. Even for a leading model like GPT-5, success rates drop sharply from 60.6% to 30.9% as interaction horizons extend, primarily due to failures in multi-step workflow execution. To address this, we propose SciForge, a data synthesis method that models the tool action space as a dependency graph to generate logic-aware training trajectories. By fine-tuning on these trajectories, our SciAgent-8B outperforms the significantly larger Qwen3-VL-235B-Instruct while exhibiting positive cross-domain transfer of scientific tool-use capabilities. These results underscore the promising potential of next-generation autonomous scientific agents.
699288af50fb2c0be47838e7
Yujiong Shen
699288af50fb2c0be47838e8
Yajie Yang
admin_assigned
2026-02-18T13:46:45.748Z
6938cef76607e405562be877
/avatars/9a9a13a5811db39ca6383667dc105875.svg
Yajie Yang
Yooojay
user
Yooojay
699288af50fb2c0be47838e9
Zhiheng Xi
admin_assigned
2026-02-18T13:46:52.968Z
653a6e5cae155b92bae77b74
https://cdn-avatars.huggingface.co/v1/production/uploads/653a6e5cae155b92bae77b74/TA5FWKAUsB249ux4MzD_R.jpeg
Zhiheng Xi
WooooDyy
user
WooooDyy
699288af50fb2c0be47838ea
Binze Hu
699288af50fb2c0be47838eb
Huayu Sha
699288af50fb2c0be47838ec
Jiazheng Zhang
699288af50fb2c0be47838ed
Qiyuan Peng
699288af50fb2c0be47838ee
Junlin Shang
699288af50fb2c0be47838ef
Jixuan Huang
699288af50fb2c0be47838f0
Yutao Fan
699288af50fb2c0be47838f1
Jingqi Tong
claimed_verified
2026-03-19T13:04:55.029Z
64a7e7b2ef22f9c793e01454
/avatars/cd3706ffedbf68f58a8e53046008b7fb.svg
tongjingqi(SII)
tongjingqi
user
tongjingqi
699288af50fb2c0be47838f2
Shihan Dou
699288af50fb2c0be47838f3
Ming Zhang
claimed_verified
2026-03-18T08:46:46.953Z
65b71c0582d38451342f7334
/avatars/f9763a0ac361c350e6c6732e23564567.svg
Ming Zhang
konglongge
user
konglongge
699288af50fb2c0be47838f4
Lei Bai
699288af50fb2c0be47838f5
Zhenfei Yin
699288af50fb2c0be47838f6
Tao Gui
699288af50fb2c0be47838f7
Xingjun Ma
699288af50fb2c0be47838f8
Qi Zhang
699288af50fb2c0be47838f9
Xuanjing Huang
699288af50fb2c0be47838fa
Yu-Gang Jiang
SciAgentGym: Benchmarking Multi-Step Scientific Tool-use in LLM Agents
Scientific reasoning inherently demands integrating sophisticated toolkits to navigate domain-specific knowledge. Yet, current benchmarks largely overlook agents' ability to orchestrate tools for such rigorous workflows. To bridge this gap, we introduce SciAgentGym, a scalable interactive environment featuring 1,780 domain-specific tools across four natural science disciplines, supported by a robust execution infrastructure. Complementing this, we present SciAgentBench, a tiered evaluation suite designed to stress-test agentic capabilities from elementary actions to long-horizon workflows. Our evaluation identifies a critical bottleneck: state-of-the-art models struggle with complex scientific tool-use. Even for a leading model like GPT-5, success rates drop sharply from 60.6% to 30.9% as interaction horizons extend, primarily due to failures in multi-step workflow execution. To address this, we propose SciForge, a data synthesis method that models the tool action space as a dependency graph to generate logic-aware training trajectories. By fine-tuning on these trajectories, our SciAgent-8B outperforms the significantly larger Qwen3-VL-235B-Instruct while exhibiting positive cross-domain transfer of scientific tool-use capabilities. These results underscore the promising potential of next-generation autonomous scientific agents.
SciAgentGym and SciAgentBench enable evaluation of scientific tool-use capabilities, while SciForge improves agent performance through dependency graph modeling of tool interactions.
SciAgentGym SciAgentBench SciForge tool action space dependency graph fine-tuning cross-domain transfer