OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent
While Vision-Language Models (VLMs) have significantly advanced Computer-Using Agents (CUAs), current frameworks struggle with robustness in long-horizon workflows and generalization in novel domains. These limitations stem from a lack of granular control over historical visual context curation and the absence of visual-aware tutorial retrieval. To bridge these gaps, we introduce OS-Symphony, a holistic framework that comprises an Orchestrator coordinating two key innovations for robust automation: (1) a Reflection-Memory Agent that utilizes milestone-driven long-term memory to enable trajectory-level self-correction, effectively mitigating visual context loss in long-horizon tasks; (2) Versatile Tool Agents featuring a Multimodal Searcher that adopts a SeeAct paradigm to navigate a browser-based sandbox to synthesize live, visually aligned tutorials, thereby resolving fidelity issues in unseen scenarios. Experimental results demonstrate that OS-Symphony delivers substantial performance gains across varying model scales, establishing new state-of-the-art results on three online benchmarks, notably achieving 65.84% on OSWorld.
6965f217fc8c4ecc02c7fa6b
Bowen Yang
claimed_verified
2026-01-14T14:21:49.683Z
66abb1d1930106b4b433f295
/avatars/12e3375fb6aa04fc24d5092bf40cdecd.svg
ybw
YYangzzzz
user
YYangzzzz
6965f217fc8c4ecc02c7fa6c
Kaiming Jin
claimed_verified
2026-01-16T10:34:22.055Z
6763dadd396e4cbdd6ab9a54
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/ver55upgXTsPn7_dFJaIY.png
Jin Kaiming
jkm1104
user
jkm1104
6965f217fc8c4ecc02c7fa6d
Zhenyu Wu
6965f217fc8c4ecc02c7fa6e
Zhaoyang Liu
6965f217fc8c4ecc02c7fa6f
Qiushi Sun
claimed_verified
2026-01-13T08:23:13.865Z
6064a0eeb1703ddba0d458b9
https://cdn-avatars.huggingface.co/v1/production/uploads/1617207525789-noauth.png
Qiushi
QiushiSun
user
QiushiSun
6965f217fc8c4ecc02c7fa70
Zehao Li
6965f217fc8c4ecc02c7fa71
JingJing Xie
6965f217fc8c4ecc02c7fa72
Zhoumianze Liu
6965f217fc8c4ecc02c7fa73
Fangzhi Xu
claimed_verified
2026-02-06T18:57:05.183Z
64e6cf78ecce34cb442dc889
https://cdn-avatars.huggingface.co/v1/production/uploads/64e6cf78ecce34cb442dc889/qVZFiUEpBpSkmH8SQeinm.jpeg
Fangzhi Xu
xufangzhi
user
xufangzhi
6965f217fc8c4ecc02c7fa74
Kanzhi Cheng
6965f217fc8c4ecc02c7fa75
Qingyun Li
6965f217fc8c4ecc02c7fa76
Yian Wang
6965f217fc8c4ecc02c7fa77
Yu Qiao
6965f217fc8c4ecc02c7fa78
Zun Wang
6965f217fc8c4ecc02c7fa79
Zichen Ding
claimed_verified
2026-01-13T08:23:16.476Z
642b9861bb77f8456634b048
https://cdn-avatars.huggingface.co/v1/production/uploads/642b9861bb77f8456634b048/VrNmmcdgX7FufQmdP5YaG.jpeg
Zichen Ding
heroding77
user
heroding77
OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent
While Vision-Language Models (VLMs) have significantly advanced Computer-Using Agents (CUAs), current frameworks struggle with robustness in long-horizon workflows and generalization in novel domains. These limitations stem from a lack of granular control over historical visual context curation and the absence of visual-aware tutorial retrieval. To bridge these gaps, we introduce OS-Symphony, a holistic framework that comprises an Orchestrator coordinating two key innovations for robust automation: (1) a Reflection-Memory Agent that utilizes milestone-driven long-term memory to enable trajectory-level self-correction, effectively mitigating visual context loss in long-horizon tasks; (2) Versatile Tool Agents featuring a Multimodal Searcher that adopts a SeeAct paradigm to navigate a browser-based sandbox to synthesize live, visually aligned tutorials, thereby resolving fidelity issues in unseen scenarios. Experimental results demonstrate that OS-Symphony delivers substantial performance gains across varying model scales, establishing new state-of-the-art results on three online benchmarks, notably achieving 65.84% on OSWorld.
OS-Symphony presents a comprehensive framework for computer-using agents that enhances robustness in long-horizon tasks through reflection-memory and multimodal search capabilities.
Vision-Language Models Computer-Using Agents long-horizon workflows visual context curation visual-aware tutorial retrieval Orchestrator Reflection-Memory Agent milestone-driven long-term memory trajectory-level self-correction Versatile Tool Agents Multimodal Searcher SeeAct paradigm browser-based sandbox live visually aligned tutorials