Reg-DPO: SFT-Regularized Direct Preference Optimization with GT-Pair for
  Improving Video Generation
Recent studies have identified Direct Preference Optimization (DPO) as an
efficient and reward-free approach to improving video generation quality.
However, existing methods largely follow image-domain paradigms and are mainly
developed on small-scale models (approximately 2B parameters), limiting their
ability to address the unique challenges of video tasks, such as costly data
construction, unstable training, and heavy memory consumption. To overcome
these limitations, we introduce a GT-Pair that automatically builds
high-quality preference pairs by using real videos as positives and
model-generated videos as negatives, eliminating the need for any external
annotation. We further present Reg-DPO, which incorporates the SFT loss as a
regularization term into the DPO objective to enhance training stability and
generation fidelity. Additionally, by combining the FSDP framework with
multiple memory optimization techniques, our approach achieves nearly three
times higher training capacity than using FSDP alone. Extensive experiments on
both I2V and T2V tasks across multiple datasets demonstrate that our method
consistently outperforms existing approaches, delivering superior video
generation quality.
690b7e0a60494e4fa7675560
Jie Du
claimed_verified
2025-11-05T16:46:05.265Z
674ec1c465d9c6670de9fee1
/avatars/564338e32bd475dd0265f417fd5c7632.svg
dujie
dujielvtqs
user
dujielvtqs
690b7e0a60494e4fa7675561
Xinyu Gong
690b7e0a60494e4fa7675562
Qingshan Tan
690b7e0a60494e4fa7675563
Wen Li
690b7e0a60494e4fa7675564
Yangming Cheng
690b7e0a60494e4fa7675565
Weitao Wang
690b7e0a60494e4fa7675566
Chenlu Zhan
690b7e0a60494e4fa7675567
Suhui Wu
690b7e0a60494e4fa7675568
Hao Zhang
690b7e0a60494e4fa7675569
Jun Zhang
Reg-DPO: SFT-Regularized Direct Preference Optimization with GT-Pair for
  Improving Video Generation
Recent studies have identified Direct Preference Optimization (DPO) as an
efficient and reward-free approach to improving video generation quality.
However, existing methods largely follow image-domain paradigms and are mainly
developed on small-scale models (approximately 2B parameters), limiting their
ability to address the unique challenges of video tasks, such as costly data
construction, unstable training, and heavy memory consumption. To overcome
these limitations, we introduce a GT-Pair that automatically builds
high-quality preference pairs by using real videos as positives and
model-generated videos as negatives, eliminating the need for any external
annotation. We further present Reg-DPO, which incorporates the SFT loss as a
regularization term into the DPO objective to enhance training stability and
generation fidelity. Additionally, by combining the FSDP framework with
multiple memory optimization techniques, our approach achieves nearly three
times higher training capacity than using FSDP alone. Extensive experiments on
both I2V and T2V tasks across multiple datasets demonstrate that our method
consistently outperforms existing approaches, delivering superior video
generation quality.
A novel approach combining GT-Pair and Reg-DPO enhances video generation quality by addressing data construction, training stability, and memory consumption challenges.
Direct Preference Optimization DPO GT-Pair Reg-DPO SFT loss FSDP framework I2V tasks T2V tasks