VectorGym: A Multitask Benchmark for SVG Code Generation, Sketching, and Editing
We introduce VectorGym, a comprehensive benchmark suite for Scalable Vector Graphics (SVG) that spans generation from text and sketches, complex editing, and visual understanding. VectorGym addresses the lack of realistic, challenging benchmarks aligned with professional design workflows. Our benchmark comprises four tasks with expert human-authored annotations: the novel Sketch2SVG task (VG-Sketch); a new SVG editing dataset (VG-Edit) featuring complex, multi-step edits with higher-order primitives; Text2SVG generation (VG-Text); and SVG captioning (VG-Cap). Unlike prior benchmarks that rely on synthetic edits, VectorGym provides gold-standard human annotations that require semantic understanding and design intent. We also propose a multi-task reinforcement learning approach that jointly optimizes across all four tasks using rendering-based rewards. Our method, built on GRPO with curriculum learning, trains a Qwen3-VL 8B model that achieves state-of-the-art performance among open-source models, surpassing much larger models including Qwen3-VL 235B and matching GPT-4o. We also introduce a VLM-as-a-Judge metric for SVG generation, validated through human correlation studies. Our evaluation of frontier VLMs reveals significant performance gaps, positioning VectorGym as a rigorous framework for advancing visual code generation. VectorGym is publicly available on huggingface.co/datasets/ServiceNow/VectorGym.
69cc7ecb3f84a0971be66a93
Juan Rodriguez
69cc7ecb3f84a0971be66a94
Haotian Zhang
69cc7ecb3f84a0971be66a95
Abhay Puri
69cc7ecb3f84a0971be66a96
Tianyang Zhang
69cc7ecb3f84a0971be66a97
Rishav Pramanik
claimed_verified
2026-04-07T08:47:04.493Z
62bb84f82ada492aa5775709
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/ltN5gU-f9NojwgrQq2G-s.png
Rishav Pramanik
rishavpramanik
user
rishavpramanik
69cc7ecb3f84a0971be66a98
Meng Lin
69cc7ecb3f84a0971be66a99
Xiaoqing Xie
69cc7ecb3f84a0971be66a9a
Marco Terral
69cc7ecb3f84a0971be66a9b
Darsh Kaushik
69cc7ecb3f84a0971be66a9c
Aly Shariff
69cc7ecb3f84a0971be66a9d
Perouz Taslakian
69cc7ecb3f84a0971be66a9e
Spandana Gella
69cc7ecb3f84a0971be66a9f
Sai Rajeswar
69cc7ecb3f84a0971be66aa0
David Vazquez
69cc7ecb3f84a0971be66aa1
Christopher Pal
69cc7ecb3f84a0971be66aa2
Marco Pedersoli
VectorGym: A Multitask Benchmark for SVG Code Generation, Sketching, and Editing
We introduce VectorGym, a comprehensive benchmark suite for Scalable Vector Graphics (SVG) that spans generation from text and sketches, complex editing, and visual understanding. VectorGym addresses the lack of realistic, challenging benchmarks aligned with professional design workflows. Our benchmark comprises four tasks with expert human-authored annotations: the novel Sketch2SVG task (VG-Sketch); a new SVG editing dataset (VG-Edit) featuring complex, multi-step edits with higher-order primitives; Text2SVG generation (VG-Text); and SVG captioning (VG-Cap). Unlike prior benchmarks that rely on synthetic edits, VectorGym provides gold-standard human annotations that require semantic understanding and design intent. We also propose a multi-task reinforcement learning approach that jointly optimizes across all four tasks using rendering-based rewards. Our method, built on GRPO with curriculum learning, trains a Qwen3-VL 8B model that achieves state-of-the-art performance among open-source models, surpassing much larger models including Qwen3-VL 235B and matching GPT-4o. We also introduce a VLM-as-a-Judge metric for SVG generation, validated through human correlation studies. Our evaluation of frontier VLMs reveals significant performance gaps, positioning VectorGym as a rigorous framework for advancing visual code generation. VectorGym is publicly available on huggingface.co/datasets/ServiceNow/VectorGym.
VectorGym presents a comprehensive benchmark suite for scalable vector graphics encompassing text-to-svg generation, sketch-to-svg conversion, complex svg editing, and visual understanding tasks with human-annotated datasets and a multi-task reinforcement learning approach.
Scalable Vector Graphics Sketch2SVG SVG editing Text2SVG SVG captioning multi-task reinforcement learning GRPO curriculum learning VLM-as-a-Judge visual code generation