VGGRPO: Towards World-Consistent Video Generation with 4D Latent Reward
Large-scale video diffusion models achieve impressive visual quality, yet often fail to preserve geometric consistency. Prior approaches improve consistency either by augmenting the generator with additional modules or applying geometry-aware alignment. However, architectural modifications can compromise the generalization of internet-scale pretrained models, while existing alignment methods are limited to static scenes and rely on RGB-space rewards that require repeated VAE decoding, incurring substantial compute overhead and failing to generalize to highly dynamic real-world scenes. To preserve the pretrained capacity while improving geometric consistency, we propose VGGRPO (Visual Geometry GRPO), a latent geometry-guided framework for geometry-aware video post-training. VGGRPO introduces a Latent Geometry Model (LGM) that stitches video diffusion latents to geometry foundation models, enabling direct decoding of scene geometry from the latent space. By constructing LGM from a geometry model with 4D reconstruction capability, VGGRPO naturally extends to dynamic scenes, overcoming the static-scene limitations of prior methods. Building on this, we perform latent-space Group Relative Policy Optimization with two complementary rewards: a camera motion smoothness reward that penalizes jittery trajectories, and a geometry reprojection consistency reward that enforces cross-view geometric coherence. Experiments on both static and dynamic benchmarks show that VGGRPO improves camera stability, geometry consistency, and overall quality while eliminating costly VAE decoding, making latent-space geometry-guided reinforcement an efficient and flexible approach to world-consistent video generation.
69cc01f13f84a0971be66971
Zhaochong An
claimed_verified
2026-04-07T08:47:08.203Z
65e5eae6958b39864e8b683e
/avatars/b6a857e7b725767197dd95bc876f8ad1.svg
Zhaochong An
ZhaochongAn
user
ZhaochongAn
69cc01f13f84a0971be66972
Orest Kupyn
claimed_verified
2026-04-01T12:43:43.773Z
64b7ae216ab5d14ca7febde0
/avatars/bb18ece0b50eb72dd6df6e32e5051d52.svg
Orest Kupyn
okupyn
user
okupyn
69cc01f13f84a0971be66973
Théo Uscidda
69cc01f13f84a0971be66974
Andrea Colaco
69cc01f13f84a0971be66975
Karan Ahuja
69cc01f13f84a0971be66976
Serge Belongie
69cc01f13f84a0971be66977
Mar Gonzalez-Franco
69cc01f13f84a0971be66978
Marta Tintore Gazulla
VGGRPO: Towards World-Consistent Video Generation with 4D Latent Reward
Large-scale video diffusion models achieve impressive visual quality, yet often fail to preserve geometric consistency. Prior approaches improve consistency either by augmenting the generator with additional modules or applying geometry-aware alignment. However, architectural modifications can compromise the generalization of internet-scale pretrained models, while existing alignment methods are limited to static scenes and rely on RGB-space rewards that require repeated VAE decoding, incurring substantial compute overhead and failing to generalize to highly dynamic real-world scenes. To preserve the pretrained capacity while improving geometric consistency, we propose VGGRPO (Visual Geometry GRPO), a latent geometry-guided framework for geometry-aware video post-training. VGGRPO introduces a Latent Geometry Model (LGM) that stitches video diffusion latents to geometry foundation models, enabling direct decoding of scene geometry from the latent space. By constructing LGM from a geometry model with 4D reconstruction capability, VGGRPO naturally extends to dynamic scenes, overcoming the static-scene limitations of prior methods. Building on this, we perform latent-space Group Relative Policy Optimization with two complementary rewards: a camera motion smoothness reward that penalizes jittery trajectories, and a geometry reprojection consistency reward that enforces cross-view geometric coherence. Experiments on both static and dynamic benchmarks show that VGGRPO improves camera stability, geometry consistency, and overall quality while eliminating costly VAE decoding, making latent-space geometry-guided reinforcement an efficient and flexible approach to world-consistent video generation.
VGGRPO enhances video diffusion models with latent geometry guidance to improve geometric consistency and camera stability while avoiding expensive VAE decoding.
video diffusion models latent space geometry foundation models Latent Geometry Model 4D reconstruction Group Relative Policy Optimization camera motion smoothness geometry reprojection consistency world-consistent video generation