CocoaBench: Evaluating Unified Digital Agents in the Wild
LLM agents now perform strongly in software engineering, deep research, GUI automation, and various other applications, while recent agent scaffolds and models are increasingly integrating these capabilities into unified systems. Yet, most evaluations still test these capabilities in isolation, which leaves a gap for more diverse use cases that require agents to combine different capabilities. We introduce CocoaBench, a benchmark for unified digital agents built from human-designed, long-horizon tasks that require flexible composition of vision, search, and coding. Tasks are specified only by an instruction and an automatic evaluation function over the final output, enabling reliable and scalable evaluation across diverse agent infrastructures. We also present CocoaAgent, a lightweight shared scaffold for controlled comparison across model backbones. Experiments show that current agents remain far from reliable on CocoaBench, with the best evaluated system achieving only 45.1% success rate. Our analysis further points to substantial room for improvement in reasoning and planning, tool use and execution, and visual grounding.
69ddb372aea31e227c22c524
CocoaBench Team
69ddb372aea31e227c22c525
Shibo Hao
69ddb372aea31e227c22c526
Zhining Zhang
69ddb372aea31e227c22c527
Zhiqi Liang
69ddb372aea31e227c22c528
Tianyang Liu
claimed_verified
2026-04-14T08:15:23.761Z
629e2bcc46b4826be2c57fe3
https://cdn-avatars.huggingface.co/v1/production/uploads/629e2bcc46b4826be2c57fe3/41BiA52XlZi31ABsljFiq.jpeg
Tianyang Liu
tianyang
user
tianyang
69ddb372aea31e227c22c529
Yuheng Zha
69ddb372aea31e227c22c52a
Qiyue Gao
69ddb372aea31e227c22c52b
Jixuan Chen
claimed_verified
2026-04-16T08:08:33.663Z
6465941d0e6c7618f615675b
https://cdn-avatars.huggingface.co/v1/production/uploads/6465941d0e6c7618f615675b/W4EHqlCucz_bojFLFEeV_.jpeg
Jixuan Chen
Mayome
user
Mayome
69ddb372aea31e227c22c52c
Zilong Wang
69ddb372aea31e227c22c52d
Zhoujun Cheng
69ddb372aea31e227c22c52e
Haoxiang Zhang
claimed_verified
2026-04-14T08:15:25.863Z
64913f1b24d9bc9bb8ff407e
https://cdn-avatars.huggingface.co/v1/production/uploads/64913f1b24d9bc9bb8ff407e/N1cdMd9_DJb5GymdKJ3Mb.jpeg
Haoxiang Zhang
IPF
user
IPF
69ddb372aea31e227c22c52f
Junli Wang
69ddb372aea31e227c22c530
Hexi Jin
69ddb372aea31e227c22c531
Boyuan Zheng
69ddb372aea31e227c22c532
Kun Zhou
69ddb372aea31e227c22c533
Yu Wang
69ddb372aea31e227c22c534
Feng Yao
69ddb372aea31e227c22c535
Licheng Liu
claimed_verified
2026-04-15T12:07:47.878Z
661caf2f6158c41091b3954e
https://cdn-avatars.huggingface.co/v1/production/uploads/661caf2f6158c41091b3954e/zrAqbiL4u1l6_Loea00_k.jpeg
Licheng Liu
LichengLiu03
user
LichengLiu03
69ddb372aea31e227c22c536
Yijiang Li
69ddb372aea31e227c22c537
Zhifei Li
69ddb372aea31e227c22c538
Zhengtao Han
69ddb372aea31e227c22c539
Pracha Promthaw
69ddb372aea31e227c22c53a
Tommaso Cerruti
69ddb372aea31e227c22c53b
Xiaohan Fu
69ddb372aea31e227c22c53c
Ziqiao Ma
69ddb372aea31e227c22c53d
Jingbo Shang
69ddb372aea31e227c22c53e
Lianhui Qin
69ddb372aea31e227c22c53f
Julian McAuley
69ddb372aea31e227c22c540
Eric P. Xing
69ddb372aea31e227c22c541
Zhengzhong Liu
69ddb372aea31e227c22c542
Rupesh Kumar Srivastava
69ddb372aea31e227c22c543
Zhiting Hu
CocoaBench: Evaluating Unified Digital Agents in the Wild
LLM agents now perform strongly in software engineering, deep research, GUI automation, and various other applications, while recent agent scaffolds and models are increasingly integrating these capabilities into unified systems. Yet, most evaluations still test these capabilities in isolation, which leaves a gap for more diverse use cases that require agents to combine different capabilities. We introduce CocoaBench, a benchmark for unified digital agents built from human-designed, long-horizon tasks that require flexible composition of vision, search, and coding. Tasks are specified only by an instruction and an automatic evaluation function over the final output, enabling reliable and scalable evaluation across diverse agent infrastructures. We also present CocoaAgent, a lightweight shared scaffold for controlled comparison across model backbones. Experiments show that current agents remain far from reliable on CocoaBench, with the best evaluated system achieving only 45.1% success rate. Our analysis further points to substantial room for improvement in reasoning and planning, tool use and execution, and visual grounding.
A new benchmark called CocoaBench evaluates unified digital agents on complex, multi-capability tasks requiring vision, search, and coding integration, revealing significant room for improvement in current agent systems.
LLM agents software engineering deep research GUI automation agent scaffolds unified systems digital agents long-horizon tasks vision search coding automatic evaluation controlled comparison model backbones reasoning planning tool use execution visual grounding