MolmoWeb: Open Visual Web Agent and Open Data for the Open Web
Web agents--autonomous systems that navigate and execute tasks on the web on behalf of users--have the potential to transform how people interact with the digital world. However, the most capable web agents today rely on proprietary models with undisclosed training data and recipes, limiting scientific understanding, reproducibility, and community-driven progress.
  We believe agents for the open web should be built in the open. To this end, we introduce (1) MolmoWebMix, a large and diverse mixture of browser task demonstrations and web-GUI perception data and (2) MolmoWeb, a family of fully open multimodal web agents. Specifically, MolmoWebMix combines over 100K synthetic task trajectories from multiple complementary generation pipelines with 30K+ human demonstrations, atomic web-skill trajectories, and GUI perception data, including referring expression grounding and screenshot question answering. MolmoWeb agents operate as instruction-conditioned visual-language action policies: given a task instruction and a webpage screenshot, they predict the next browser action, requiring no access to HTML, accessibility trees, or specialized APIs.
  Available in 4B and 8B size, on browser-use benchmarks like WebVoyager, Online-Mind2Web, and DeepShop, MolmoWeb agents achieve state-of-the-art results outperforming similar scale open-weight-only models such as Fara-7B, UI-Tars-1.5-7B, and Holo1-7B. MolmoWeb-8B also surpasses set-of-marks (SoM) agents built on much larger closed frontier models like GPT-4o. We further demonstrate consistent gains through test-time scaling via parallel rollouts with best-of-N selection, achieving 94.7% and 60.5% pass@4 (compared to 78.2% and 35.3% pass@1) on WebVoyager and Online-Mind2Web respectively. We will release model checkpoints, training data, code, and a unified evaluation harness to enable reproducibility and accelerate open research on web agents.
69d859fa79548974da946494
Tanmay Gupta
69d859fa79548974da946495
Piper Wolters
69d859fa79548974da946496
Zixian Ma
69d859fa79548974da946497
Peter Sushko
69d859fa79548974da946498
Rock Yuren Pang
69d859fa79548974da946499
Diego Llanes
69d859fa79548974da94649a
Yue Yang
claimed_verified
2026-04-13T07:34:00.285Z
62f6c68904e5e02f82b04690
https://cdn-avatars.huggingface.co/v1/production/uploads/62f6c68904e5e02f82b04690/kK2-PkeAGzAOLhkfajswf.jpeg
Yue Yang
yyupenn
user
yyupenn
69d859fa79548974da94649b
Taira Anderson
69d859fa79548974da94649c
Boyuan Zheng
69d859fa79548974da94649d
Zhongzheng Ren
69d859fa79548974da94649e
Harsh Trivedi
69d859fa79548974da94649f
Taylor Blanton
69d859fa79548974da9464a0
Caleb Ouellette
69d859fa79548974da9464a1
Winson Han
69d859fa79548974da9464a2
Ali Farhadi
69d859fa79548974da9464a3
Ranjay Krishna
MolmoWeb: Open Visual Web Agent and Open Data for the Open Web
Web agents--autonomous systems that navigate and execute tasks on the web on behalf of users--have the potential to transform how people interact with the digital world. However, the most capable web agents today rely on proprietary models with undisclosed training data and recipes, limiting scientific understanding, reproducibility, and community-driven progress.
  We believe agents for the open web should be built in the open. To this end, we introduce (1) MolmoWebMix, a large and diverse mixture of browser task demonstrations and web-GUI perception data and (2) MolmoWeb, a family of fully open multimodal web agents. Specifically, MolmoWebMix combines over 100K synthetic task trajectories from multiple complementary generation pipelines with 30K+ human demonstrations, atomic web-skill trajectories, and GUI perception data, including referring expression grounding and screenshot question answering. MolmoWeb agents operate as instruction-conditioned visual-language action policies: given a task instruction and a webpage screenshot, they predict the next browser action, requiring no access to HTML, accessibility trees, or specialized APIs.
  Available in 4B and 8B size, on browser-use benchmarks like WebVoyager, Online-Mind2Web, and DeepShop, MolmoWeb agents achieve state-of-the-art results outperforming similar scale open-weight-only models such as Fara-7B, UI-Tars-1.5-7B, and Holo1-7B. MolmoWeb-8B also surpasses set-of-marks (SoM) agents built on much larger closed frontier models like GPT-4o. We further demonstrate consistent gains through test-time scaling via parallel rollouts with best-of-N selection, achieving 94.7% and 60.5% pass@4 (compared to 78.2% and 35.3% pass@1) on WebVoyager and Online-Mind2Web respectively. We will release model checkpoints, training data, code, and a unified evaluation harness to enable reproducibility and accelerate open research on web agents.
Open-source web agents leveraging diverse mixed datasets achieve state-of-the-art performance on browser-based tasks while operating without access to HTML or accessibility tree information.
web agents browser task demonstrations web-GUI perception multimodal web agents synthetic task trajectories human demonstrations atomic web-skill trajectories referring expression grounding screenshot question answering instruction-conditioned visual-language action policies pass@4 pass@1 test-time scaling parallel rollouts best-of-N selection