EcomBench: Towards Holistic Evaluation of Foundation Agents in E-commerce
Foundation agents have rapidly advanced in their ability to reason and interact with real environments, making the evaluation of their core capabilities increasingly important. While many benchmarks have been developed to assess agent performance, most concentrate on academic settings or artificially designed scenarios while overlooking the challenges that arise in real applications. To address this issue, we focus on a highly practical real-world setting, the e-commerce domain, which involves a large volume of diverse user interactions, dynamic market conditions, and tasks directly tied to real decision-making processes. To this end, we introduce EcomBench, a holistic E-commerce Benchmark designed to evaluate agent performance in realistic e-commerce environments. EcomBench is built from genuine user demands embedded in leading global e-commerce ecosystems and is carefully curated and annotated through human experts to ensure clarity, accuracy, and domain relevance. It covers multiple task categories within e-commerce scenarios and defines three difficulty levels that evaluate agents on key capabilities such as deep information retrieval, multi-step reasoning, and cross-source knowledge integration. By grounding evaluation in real e-commerce contexts, EcomBench provides a rigorous and dynamic testbed for measuring the practical capabilities of agents in modern e-commerce.
6938e481dfc35938ba129f9c
Rui Min
6938e481dfc35938ba129f9d
Zile Qiao
6938e481dfc35938ba129f9e
Ze Xu
6938e481dfc35938ba129f9f
Jiawen Zhai
6938e481dfc35938ba129fa0
Wenyu Gao
6938e481dfc35938ba129fa1
Xuanzhong Chen
claimed_verified
2025-12-10T08:56:17.089Z
65e6970d135c27ea806526fe
/avatars/4aced113d9cab055ae06f3945869a280.svg
Xuanzhong Chen
chenxz
user
chenxz
6938e481dfc35938ba129fa2
Haozhen Sun
6938e481dfc35938ba129fa3
Zhen Zhang
6938e481dfc35938ba129fa4
Xinyu Wang
6938e481dfc35938ba129fa5
Hong Zhou
6938e481dfc35938ba129fa6
Wenbiao Yin
6938e481dfc35938ba129fa7
Xuan Zhou
6938e481dfc35938ba129fa8
Yong Jiang
6938e481dfc35938ba129fa9
Haicheng Liu
6938e481dfc35938ba129faa
Liang Ding
claimed_verified
2025-12-12T09:22:28.049Z
641123b4230ce11b1be68fa1
https://cdn-avatars.huggingface.co/v1/production/uploads/641123b4230ce11b1be68fa1/Xm1gZgK4MRq-LN20ZxxHE.jpeg
Liang Ding (Hiring https://liamding.cc/hiring.htm)
alphadl
user
alphadl
6938e481dfc35938ba129fab
Ling Zou
6938e481dfc35938ba129fac
Yi R.
6938e481dfc35938ba129fad
Fung
6938e481dfc35938ba129fae
Yalong Li
6938e481dfc35938ba129faf
Pengjun Xie
EcomBench: Towards Holistic Evaluation of Foundation Agents in E-commerce
Foundation agents have rapidly advanced in their ability to reason and interact with real environments, making the evaluation of their core capabilities increasingly important. While many benchmarks have been developed to assess agent performance, most concentrate on academic settings or artificially designed scenarios while overlooking the challenges that arise in real applications. To address this issue, we focus on a highly practical real-world setting, the e-commerce domain, which involves a large volume of diverse user interactions, dynamic market conditions, and tasks directly tied to real decision-making processes. To this end, we introduce EcomBench, a holistic E-commerce Benchmark designed to evaluate agent performance in realistic e-commerce environments. EcomBench is built from genuine user demands embedded in leading global e-commerce ecosystems and is carefully curated and annotated through human experts to ensure clarity, accuracy, and domain relevance. It covers multiple task categories within e-commerce scenarios and defines three difficulty levels that evaluate agents on key capabilities such as deep information retrieval, multi-step reasoning, and cross-source knowledge integration. By grounding evaluation in real e-commerce contexts, EcomBench provides a rigorous and dynamic testbed for measuring the practical capabilities of agents in modern e-commerce.
EcomBench is a benchmark that evaluates agent performance in real-world e-commerce environments through deep information retrieval, multi-step reasoning, and cross-source knowledge integration.
deep information retrieval multi-step reasoning cross-source knowledge integration