RoboVIP: Multi-View Video Generation with Visual Identity Prompting Augments Robot Manipulation
The diversity, quantity, and quality of manipulation data are critical for training effective robot policies. However, due to hardware and physical setup constraints, collecting large-scale real-world manipulation data remains difficult to scale across diverse environments. Recent work uses text-prompt conditioned image diffusion models to augment manipulation data by altering the backgrounds and tabletop objects in the visual observations. However, these approaches often overlook the practical need for multi-view and temporally coherent observations required by state-of-the-art policy models. Further, text prompts alone cannot reliably specify the scene setup. To provide the diffusion model with explicit visual guidance, we introduce visual identity prompting, which supplies exemplar images as conditioning inputs to guide the generation of the desired scene setup. To this end, we also build a scalable pipeline to curate a visual identity pool from large robotics datasets. Using our augmented manipulation data to train downstream vision-language-action and visuomotor policy models yields consistent performance gains in both simulation and real-robot settings.
6960775a5b7998385e6394ff
Boyang Wang
claimed_verified
2026-01-09T08:35:11.481Z
64ed876a74d9b58eabc769a4
https://cdn-avatars.huggingface.co/v1/production/uploads/64ed876a74d9b58eabc769a4/K4bJVW0FlqRtAAxJBJifR.jpeg
Boyang Wang
HikariDawn
user
HikariDawn
6960775a5b7998385e639500
Haoran Zhang
6960775a5b7998385e639501
Shujie Zhang
6960775a5b7998385e639502
Jinkun Hao
claimed_verified
2026-01-09T08:35:09.450Z
64edb581067fbb625f893628
/avatars/d59893d6f1f752bb73255bd78b325fe9.svg
hao
wuzhi-hao
user
wuzhi-hao
6960775a5b7998385e639503
Mingda Jia
6960775a5b7998385e639504
Qi Lv
6960775a5b7998385e639505
Yucheng Mao
admin_assigned
2026-01-09T15:52:47.293Z
65de9c6cf68c3d3bac330509
/avatars/150858545ea1a9e7c96d6f227093ac54.svg
Yucheng Mao
matthewmao
user
matthewmao
6960775a5b7998385e639506
Zhaoyang Lyu
admin_assigned
2026-01-09T15:52:53.025Z
63f2ec797ddf724fbcc75aee
/avatars/e93432ad11da703d46fe5e594d69f8c0.svg
Zhaoyang Lyu
ZhaoyangLyu
user
ZhaoyangLyu
6960775a5b7998385e639507
Jia Zeng
claimed_verified
2026-01-09T15:46:06.251Z
685d08b9fc7a0ff2f338dbd0
https://cdn-avatars.huggingface.co/v1/production/uploads/685d08b9fc7a0ff2f338dbd0/bW3G1ZM-tAXIIbCbUdwC9.png
Jia Zeng
Jia-Zeng
user
Jia-Zeng
6960775a5b7998385e639508
Xudong Xu
6960775a5b7998385e639509
Jiangmiao Pang
admin_assigned
2026-01-09T15:52:00.637Z
65783ee6ee33d547aecc3ffc
https://cdn-avatars.huggingface.co/v1/production/uploads/65783ee6ee33d547aecc3ffc/lWZX88c-0dCsN-yB9Jhlf.jpeg
Jiangmiao Pang
Jiangmiao
user
Jiangmiao
RoboVIP: Multi-View Video Generation with Visual Identity Prompting Augments Robot Manipulation
The diversity, quantity, and quality of manipulation data are critical for training effective robot policies. However, due to hardware and physical setup constraints, collecting large-scale real-world manipulation data remains difficult to scale across diverse environments. Recent work uses text-prompt conditioned image diffusion models to augment manipulation data by altering the backgrounds and tabletop objects in the visual observations. However, these approaches often overlook the practical need for multi-view and temporally coherent observations required by state-of-the-art policy models. Further, text prompts alone cannot reliably specify the scene setup. To provide the diffusion model with explicit visual guidance, we introduce visual identity prompting, which supplies exemplar images as conditioning inputs to guide the generation of the desired scene setup. To this end, we also build a scalable pipeline to curate a visual identity pool from large robotics datasets. Using our augmented manipulation data to train downstream vision-language-action and visuomotor policy models yields consistent performance gains in both simulation and real-robot settings.
Visual identity prompting enhances manipulation data augmentation for robot policies by providing explicit visual guidance to diffusion models, improving policy performance in both simulation and real-world settings.
image diffusion models visual identity prompting manipulation data vision-language-action models visuomotor policy models visual identity pool