CUA-Suite: Massive Human-annotated Video Demonstrations for Computer-Use Agents
Computer-use agents (CUAs) hold great promise for automating complex desktop workflows, yet progress toward general-purpose agents is bottlenecked by the scarcity of continuous, high-quality human demonstration videos. Recent work emphasizes that continuous video, not sparse screenshots, is the critical missing ingredient for scaling these agents. However, the largest existing open dataset, ScaleCUA, contains only 2 million screenshots, equating to less than 20 hours of video. To address this bottleneck, we introduce CUA-Suite, a large-scale ecosystem of expert video demonstrations and dense annotations for professional desktop computer-use agents. At its core is VideoCUA, which provides approximately 10,000 human-demonstrated tasks across 87 diverse applications with continuous 30 fps screen recordings, kinematic cursor traces, and multi-layerfed reasoning annotations, totaling approximately 55 hours and 6 million frames of expert video. Unlike sparse datasets that capture only final click coordinates, these continuous video streams preserve the full temporal dynamics of human interaction, forming a superset of information that can be losslessly transformed into the formats required by existing agent frameworks. CUA-Suite further provides two complementary resources: UI-Vision, a rigorous benchmark for evaluating grounding and planning capabilities in CUAs, and GroundCUA, a large-scale grounding dataset with 56K annotated screenshots and over 3.6 million UI element annotations. Preliminary evaluation reveals that current foundation action models struggle substantially with professional desktop applications (~60% task failure rate). Beyond evaluation, CUA-Suite's rich multimodal corpus supports emerging research directions including generalist screen parsing, continuous spatial control, video-based reward modeling, and visual world models. All data and models are publicly released.
69c49aee4ae7eef815b9d9a3
Xiangru Jian
claimed_verified
2026-03-26T13:42:49.162Z
636865b8cca0a0a962c21f3f
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Mja7cpws4gb2Jmdj_foPA.png
Xiangru (Edward) Jian
HideOnBush
user
HideOnBush
69c49aee4ae7eef815b9d9a4
Shravan Nayak
claimed_verified
2026-03-26T13:42:52.473Z
643ee7606d5be535d28034f6
https://cdn-avatars.huggingface.co/v1/production/uploads/643ee7606d5be535d28034f6/1F241mP_buW74byC2GAWH.jpeg
Shravan Nayak
BAJUKA
user
BAJUKA
69c49aee4ae7eef815b9d9a5
Kevin Qinghong Lin
claimed_verified
2026-03-26T14:58:28.570Z
64440be5af034cdfd69ca3a7
https://cdn-avatars.huggingface.co/v1/production/uploads/64440be5af034cdfd69ca3a7/qmx24QiDFT29vleCxL9TX.jpeg
Qinghong (Kevin) Lin
KevinQHLin
user
KevinQHLin
69c49aee4ae7eef815b9d9a6
Aarash Feizi
69c49aee4ae7eef815b9d9a7
Kaixin Li
claimed_verified
2026-03-27T11:35:08.913Z
6346be8f7fb9f11870c63998
https://cdn-avatars.huggingface.co/v1/production/uploads/6346be8f7fb9f11870c63998/tFWawSkXL6bv1zgvzFWQd.png
Kaixin Li
likaixin
user
likaixin
69c49aee4ae7eef815b9d9a8
Patrice Bechard
claimed_verified
2026-03-26T13:42:47.120Z
607f060442beb4da0f990182
https://cdn-avatars.huggingface.co/v1/production/uploads/607f060442beb4da0f990182/j5W2tLyU6JqkaTf3kv66s.jpeg
Patrice Bechard
patricebechard
user
patricebechard
69c49aee4ae7eef815b9d9a9
Spandana Gella
claimed_verified
2026-03-27T11:35:12.344Z
65ca8745d64d82c92fa7c71f
https://cdn-avatars.huggingface.co/v1/production/uploads/65ca8745d64d82c92fa7c71f/xayDYjs_LoAsa8BEgDlhb.jpeg
G
spandanagella
user
spandanagella
69c49aee4ae7eef815b9d9aa
Sai Rajeswar
claimed_verified
2026-03-27T11:35:14.518Z
642f99079b2484d7d857341b
/avatars/01965cc5a5dbe9c08025a51973462a6a.svg
Sai Rajeswar
rajeswarsai
user
rajeswarsai
CUA-Suite: Massive Human-annotated Video Demonstrations for Computer-Use Agents
Computer-use agents (CUAs) hold great promise for automating complex desktop workflows, yet progress toward general-purpose agents is bottlenecked by the scarcity of continuous, high-quality human demonstration videos. Recent work emphasizes that continuous video, not sparse screenshots, is the critical missing ingredient for scaling these agents. However, the largest existing open dataset, ScaleCUA, contains only 2 million screenshots, equating to less than 20 hours of video. To address this bottleneck, we introduce CUA-Suite, a large-scale ecosystem of expert video demonstrations and dense annotations for professional desktop computer-use agents. At its core is VideoCUA, which provides approximately 10,000 human-demonstrated tasks across 87 diverse applications with continuous 30 fps screen recordings, kinematic cursor traces, and multi-layerfed reasoning annotations, totaling approximately 55 hours and 6 million frames of expert video. Unlike sparse datasets that capture only final click coordinates, these continuous video streams preserve the full temporal dynamics of human interaction, forming a superset of information that can be losslessly transformed into the formats required by existing agent frameworks. CUA-Suite further provides two complementary resources: UI-Vision, a rigorous benchmark for evaluating grounding and planning capabilities in CUAs, and GroundCUA, a large-scale grounding dataset with 56K annotated screenshots and over 3.6 million UI element annotations. Preliminary evaluation reveals that current foundation action models struggle substantially with professional desktop applications (~60% task failure rate). Beyond evaluation, CUA-Suite's rich multimodal corpus supports emerging research directions including generalist screen parsing, continuous spatial control, video-based reward modeling, and visual world models. All data and models are publicly released.
CUA-Suite introduces a large-scale ecosystem of expert video demonstrations and annotations for computer-use agents, providing continuous screen recordings and detailed reasoning annotations to advance desktop automation capabilities.
computer-use agents expert video demonstrations continuous screen recordings multimodal corpus visual world models