OpenSubject: Leveraging Video-Derived Identity and Diversity Priors for Subject-driven Image Generation and Manipulation
Despite the promising progress in subject-driven image generation, current models often deviate from the reference identities and struggle in complex scenes with multiple subjects. To address this challenge, we introduce OpenSubject, a video-derived large-scale corpus with 2.5M samples and 4.35M images for subject-driven generation and manipulation. The dataset is built with a four-stage pipeline that exploits cross-frame identity priors. (i) Video Curation. We apply resolution and aesthetic filtering to obtain high-quality clips. (ii) Cross-Frame Subject Mining and Pairing. We utilize vision-language model (VLM)-based category consensus, local grounding, and diversity-aware pairing to select image pairs. (iii) Identity-Preserving Reference Image Synthesis. We introduce segmentation map-guided outpainting to synthesize the input images for subject-driven generation and box-guided inpainting to generate input images for subject-driven manipulation, together with geometry-aware augmentations and irregular boundary erosion. (iv) Verification and Captioning. We utilize a VLM to validate synthesized samples, re-synthesize failed samples based on stage (iii), and then construct short and long captions. In addition, we introduce a benchmark covering subject-driven generation and manipulation, and then evaluate identity fidelity, prompt adherence, manipulation consistency, and background consistency with a VLM judge. Extensive experiments show that training with OpenSubject improves generation and manipulation performance, particularly in complex scenes.
6938da07dfc35938ba129f2f
Yexin Liu
claimed_verified
2025-12-19T09:07:57.368Z
6497ff395b5d43c1c77d2a11
/avatars/e3704701ed3ea0c1c165001521f40086.svg
Yexin Liu
AIPeanutman
user
AIPeanutman
6938da07dfc35938ba129f30
Manyuan Zhang
6938da07dfc35938ba129f31
Yueze Wang
claimed_verified
2025-12-12T09:17:48.423Z
6458b59c7a7e192202df8fa0
/avatars/33ee716477e5686da8723d01e199cd27.svg
Yueze Wang
yzwang
user
yzwang
6938da07dfc35938ba129f32
Hongyu Li
6938da07dfc35938ba129f33
Dian Zheng
claimed_verified
2025-12-18T08:21:00.481Z
67e60ae6ac37824273d74389
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/YvPKZ_0gyJnvNwM1zK3JS.png
Dian Zheng
zhengli1013
user
zhengli1013
6938da07dfc35938ba129f34
Weiming Zhang
6938da07dfc35938ba129f35
Changsheng Lu
6938da07dfc35938ba129f36
Xunliang Cai
6938da07dfc35938ba129f37
Yan Feng
6938da07dfc35938ba129f38
Peng Pei
6938da07dfc35938ba129f39
Harry Yang
OpenSubject: Leveraging Video-Derived Identity and Diversity Priors for Subject-driven Image Generation and Manipulation
Despite the promising progress in subject-driven image generation, current models often deviate from the reference identities and struggle in complex scenes with multiple subjects. To address this challenge, we introduce OpenSubject, a video-derived large-scale corpus with 2.5M samples and 4.35M images for subject-driven generation and manipulation. The dataset is built with a four-stage pipeline that exploits cross-frame identity priors. (i) Video Curation. We apply resolution and aesthetic filtering to obtain high-quality clips. (ii) Cross-Frame Subject Mining and Pairing. We utilize vision-language model (VLM)-based category consensus, local grounding, and diversity-aware pairing to select image pairs. (iii) Identity-Preserving Reference Image Synthesis. We introduce segmentation map-guided outpainting to synthesize the input images for subject-driven generation and box-guided inpainting to generate input images for subject-driven manipulation, together with geometry-aware augmentations and irregular boundary erosion. (iv) Verification and Captioning. We utilize a VLM to validate synthesized samples, re-synthesize failed samples based on stage (iii), and then construct short and long captions. In addition, we introduce a benchmark covering subject-driven generation and manipulation, and then evaluate identity fidelity, prompt adherence, manipulation consistency, and background consistency with a VLM judge. Extensive experiments show that training with OpenSubject improves generation and manipulation performance, particularly in complex scenes.
OpenSubject, a large-scale video-derived dataset, enhances subject-driven image generation and manipulation through a four-stage pipeline that maintains identity fidelity and handles complex scenes.
vision-language model category consensus local grounding diversity-aware pairing segmentation map-guided outpainting box-guided inpainting geometry-aware augmentations irregular boundary erosion identity fidelity prompt adherence manipulation consistency background consistency