Scaling Zero-Shot Reference-to-Video Generation
Reference-to-video (R2V) generation aims to synthesize videos that align with a text prompt while preserving the subject identity from reference images. However, current R2V methods are hindered by the reliance on explicit reference image-video-text triplets, whose construction is highly expensive and difficult to scale. We bypass this bottleneck by introducing Saber, a scalable zero-shot framework that requires no explicit R2V data. Trained exclusively on video-text pairs, Saber employs a masked training strategy and a tailored attention-based model design to learn identity-consistent and reference-aware representations. Mask augmentation techniques are further integrated to mitigate copy-paste artifacts common in reference-to-video generation. Moreover, Saber demonstrates remarkable generalization capabilities across a varying number of references and achieves superior performance on the OpenS2V-Eval benchmark compared to methods trained with R2V data.
69379d8b19d912300c34a2de
Zijian Zhou
claimed_verified
2025-12-09T10:34:32.358Z
62e0fcfe4db2175cd2710063
/avatars/0f8d1b5ba3e3eba2af18f98f65f0b180.svg
Zijian Zhou
franciszzj
user
franciszzj
69379d8b19d912300c34a2df
Shikun Liu
69379d8b19d912300c34a2e0
Haozhe Liu
69379d8b19d912300c34a2e1
Haonan Qiu
69379d8b19d912300c34a2e2
Zhaochong An
claimed_verified
2026-03-30T09:12:32.891Z
65e5eae6958b39864e8b683e
/avatars/b6a857e7b725767197dd95bc876f8ad1.svg
Zhaochong An
ZhaochongAn
user
ZhaochongAn
69379d8b19d912300c34a2e3
Weiming Ren
claimed_verified
2026-01-20T09:41:32.936Z
64405a9d518271b0d1beea38
/avatars/b702474588fd7090773320422417a582.svg
Weiming Ren
wren93
user
wren93
69379d8b19d912300c34a2e4
Zhiheng Liu
69379d8b19d912300c34a2e5
Xiaoke Huang
claimed_verified
2026-02-26T10:04:26.909Z
63318b2349a9563915469f3b
https://cdn-avatars.huggingface.co/v1/production/uploads/63318b2349a9563915469f3b/zlbeB2997i8YkoyOTb9FL.jpeg
Xiaoke Huang
xk-huang
user
xk-huang
69379d8b19d912300c34a2e6
Kam Woh Ng
69379d8b19d912300c34a2e7
Tian Xie
69379d8b19d912300c34a2e8
Xiao Han
69379d8b19d912300c34a2e9
Yuren Cong
69379d8b19d912300c34a2ea
Hang Li
69379d8b19d912300c34a2eb
Chuyan Zhu
69379d8b19d912300c34a2ec
Aditya Patel
69379d8b19d912300c34a2ed
Tao Xiang
69379d8b19d912300c34a2ee
Sen He
Scaling Zero-Shot Reference-to-Video Generation
Reference-to-video (R2V) generation aims to synthesize videos that align with a text prompt while preserving the subject identity from reference images. However, current R2V methods are hindered by the reliance on explicit reference image-video-text triplets, whose construction is highly expensive and difficult to scale. We bypass this bottleneck by introducing Saber, a scalable zero-shot framework that requires no explicit R2V data. Trained exclusively on video-text pairs, Saber employs a masked training strategy and a tailored attention-based model design to learn identity-consistent and reference-aware representations. Mask augmentation techniques are further integrated to mitigate copy-paste artifacts common in reference-to-video generation. Moreover, Saber demonstrates remarkable generalization capabilities across a varying number of references and achieves superior performance on the OpenS2V-Eval benchmark compared to methods trained with R2V data.
Saber is a scalable zero-shot framework for reference-to-video generation that uses video-text pairs to learn identity-consistent representations and outperforms models trained with explicit reference data.
reference-to-video (R2V) generation masked training strategy attention-based model design mask augmentation OpenS2V-Eval benchmark