\$OneMillion-Bench: How Far are Language Agents from Human Experts?
As language models (LMs) evolve from chat assistants to long-horizon agents capable of multi-step reasoning and tool use, existing benchmarks remain largely confined to structured or exam-style tasks that fall short of real-world professional demands. To this end, we introduce \OneMillion-Bench OneMillion-Bench, a benchmark of 400 expert-curated tasks spanning Law, Finance, Industry, Healthcare, and Natural Science, built to evaluate agents across economically consequential scenarios. Unlike prior work, the benchmark requires retrieving authoritative sources, resolving conflicting evidence, applying domain-specific rules, and making constraint decisions, where correctness depends as much on the reasoning process as the final answer. We adopt a rubric-based evaluation protocol scoring factual accuracy, logical coherence, practical feasibility, and professional compliance, focused on expert-level problems to ensure meaningful differentiation across agents. Together, \$OneMillion-Bench provides a unified testbed for assessing agentic reliability, professional depth, and practical readiness in domain-intensive scenarios.
69af873bb3ab31dee1281c70
Qianyu Yang
69af873bb3ab31dee1281c71
Yang Liu
claimed_verified
2026-03-10T09:45:03.924Z
6191cc9e6d34e827404cebab
https://cdn-avatars.huggingface.co/v1/production/uploads/1674119843175-6191cc9e6d34e827404cebab.jpeg
Yang
jacklanda
user
jacklanda
69af873bb3ab31dee1281c72
Jiaqi Li
69af873bb3ab31dee1281c73
Jun Bai
69af873bb3ab31dee1281c74
Hao Chen
69af873bb3ab31dee1281c75
Kaiyuan Chen
69af873bb3ab31dee1281c76
Tiliang Duan
69af873bb3ab31dee1281c77
Jiayun Dong
69af873bb3ab31dee1281c78
Xiaobo Hu
69af873bb3ab31dee1281c79
Zixia Jia
69af873bb3ab31dee1281c7a
Yang Liu
69af873bb3ab31dee1281c7b
Tao Peng
69af873bb3ab31dee1281c7c
Yixin Ren
69af873bb3ab31dee1281c7d
Ran Tian
69af873bb3ab31dee1281c7e
Zaiyuan Wang
69af873bb3ab31dee1281c7f
Yanglihong Xiao
69af873bb3ab31dee1281c80
Gang Yao
69af873bb3ab31dee1281c81
Lingyue Yin
69af873bb3ab31dee1281c82
Ge Zhang
claimed_verified
2026-03-16T10:44:35.091Z
638efcf4c67af472d316d424
/avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
Ge Zhang
zhangysk
user
zhangysk
69af873bb3ab31dee1281c83
Chun Zhang
69af873bb3ab31dee1281c84
Jianpeng Jiao
69af873bb3ab31dee1281c85
Zilong Zheng
69af873bb3ab31dee1281c86
Yuan Gong
\$OneMillion-Bench: How Far are Language Agents from Human Experts?
As language models (LMs) evolve from chat assistants to long-horizon agents capable of multi-step reasoning and tool use, existing benchmarks remain largely confined to structured or exam-style tasks that fall short of real-world professional demands. To this end, we introduce \OneMillion-Bench OneMillion-Bench, a benchmark of 400 expert-curated tasks spanning Law, Finance, Industry, Healthcare, and Natural Science, built to evaluate agents across economically consequential scenarios. Unlike prior work, the benchmark requires retrieving authoritative sources, resolving conflicting evidence, applying domain-specific rules, and making constraint decisions, where correctness depends as much on the reasoning process as the final answer. We adopt a rubric-based evaluation protocol scoring factual accuracy, logical coherence, practical feasibility, and professional compliance, focused on expert-level problems to ensure meaningful differentiation across agents. Together, \$OneMillion-Bench provides a unified testbed for assessing agentic reliability, professional depth, and practical readiness in domain-intensive scenarios.
A new benchmark evaluates language models on complex, real-world professional tasks requiring multi-step reasoning, evidence resolution, and domain-specific decision-making across multiple industries.