FinToolBench: Evaluating LLM Agents for Real-World Financial Tool Use
The integration of Large Language Models (LLMs) into the financial domain is driving a paradigm shift from passive information retrieval to dynamic, agentic interaction. While general-purpose tool learning has witnessed a surge in benchmarks, the financial sector, characterized by high stakes, strict compliance, and rapid data volatility, remains critically underserved. Existing financial evaluations predominantly focus on static textual analysis or document-based QA, ignoring the complex reality of tool execution. Conversely, general tool benchmarks lack the domain-specific rigor required for finance, often relying on toy environments or a negligible number of financial APIs. To bridge this gap, we introduce FinToolBench, the first real-world, runnable benchmark dedicated to evaluating financial tool learning agents. Unlike prior works limited to a handful of mock tools, FinToolBench establishes a realistic ecosystem coupling 760 executable financial tools with 295 rigorous, tool-required queries. We propose a novel evaluation framework that goes beyond binary execution success, assessing agents on finance-critical dimensions: timeliness, intent type, and regulatory domain alignment. Furthermore, we present FATR, a finance-aware tool retrieval and reasoning baseline that enhances stability and compliance. By providing the first testbed for auditable, agentic financial execution, FinToolBench sets a new standard for trustworthy AI in finance. The tool manifest, execution environment, and evaluation code will be open-sourced to facilitate future research.
69b0dbc01f7902d47f6dfa23
Jiaxuan Lu
admin_assigned
2026-03-18T10:01:29.270Z
67e655f7d6b8333a8f78eadf
/avatars/11cc80d3c03747fd869e4dc1dbdd031a.svg
Jiaxuan Lu
Blue-Giant
user
Blue-Giant
69b0dbc01f7902d47f6dfa24
Kong Wang
69b0dbc01f7902d47f6dfa25
Yemin Wang
admin_assigned
2026-03-18T10:01:35.254Z
6971caaa5a5aadea5d915daf
/avatars/697d706b64355d1c44e48230545c0f43.svg
Yemin Wang
YeminWang
user
YeminWang
69b0dbc01f7902d47f6dfa26
Qingmei Tang
69b0dbc01f7902d47f6dfa27
Hongwei Zeng
69b0dbc01f7902d47f6dfa28
Xiang Chen
69b0dbc01f7902d47f6dfa29
Jiahao Pi
69b0dbc01f7902d47f6dfa2a
Shujian Deng
admin_assigned
2026-03-18T10:01:53.900Z
67220a19688998c4ea48844c
/avatars/dab2203c212e6f7bd1171d9d7a1352b9.svg
shujian deng
emergencyd
user
emergencyd
69b0dbc01f7902d47f6dfa2b
Lingzhi Chen
admin_assigned
2026-03-18T10:01:59.911Z
692950a8f39ea343dd3f99f6
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/apGtoTpWu69FnUNMtN8Ja.png
lingzhi chen
chenvc
user
chenvc
69b0dbc01f7902d47f6dfa2c
Yi Fu
69b0dbc01f7902d47f6dfa2d
Kehua Yang
69b0dbc01f7902d47f6dfa2e
Xiao Sun
FinToolBench: Evaluating LLM Agents for Real-World Financial Tool Use
The integration of Large Language Models (LLMs) into the financial domain is driving a paradigm shift from passive information retrieval to dynamic, agentic interaction. While general-purpose tool learning has witnessed a surge in benchmarks, the financial sector, characterized by high stakes, strict compliance, and rapid data volatility, remains critically underserved. Existing financial evaluations predominantly focus on static textual analysis or document-based QA, ignoring the complex reality of tool execution. Conversely, general tool benchmarks lack the domain-specific rigor required for finance, often relying on toy environments or a negligible number of financial APIs. To bridge this gap, we introduce FinToolBench, the first real-world, runnable benchmark dedicated to evaluating financial tool learning agents. Unlike prior works limited to a handful of mock tools, FinToolBench establishes a realistic ecosystem coupling 760 executable financial tools with 295 rigorous, tool-required queries. We propose a novel evaluation framework that goes beyond binary execution success, assessing agents on finance-critical dimensions: timeliness, intent type, and regulatory domain alignment. Furthermore, we present FATR, a finance-aware tool retrieval and reasoning baseline that enhances stability and compliance. By providing the first testbed for auditable, agentic financial execution, FinToolBench sets a new standard for trustworthy AI in finance. The tool manifest, execution environment, and evaluation code will be open-sourced to facilitate future research.
FinToolBench presents the first real-world benchmark for evaluating financial tool learning agents, featuring 760 executable tools and comprehensive evaluation criteria beyond simple execution success.
Large Language Models tool learning financial domain agentic interaction FinToolBench executable financial tools tool retrieval reasoning baseline auditability trustworthy AI