Generative Neural Video Compression via Video Diffusion Prior
We present GNVC-VD, the first DiT-based generative neural video compression framework built upon an advanced video generation foundation model, where spatio-temporal latent compression and sequence-level generative refinement are unified within a single codec. Existing perceptual codecs primarily rely on pre-trained image generative priors to restore high-frequency details, but their frame-wise nature lacks temporal modeling and inevitably leads to perceptual flickering. To address this, GNVC-VD introduces a unified flow-matching latent refinement module that leverages a video diffusion transformer to jointly enhance intra- and inter-frame latents through sequence-level denoising, ensuring consistent spatio-temporal details. Instead of denoising from pure Gaussian noise as in video generation, GNVC-VD initializes refinement from decoded spatio-temporal latents and learns a correction term that adapts the diffusion prior to compression-induced degradation. A conditioning adaptor further injects compression-aware cues into intermediate DiT layers, enabling effective artifact removal while maintaining temporal coherence under extreme bitrate constraints. Extensive experiments show that GNVC-VD surpasses both traditional and learned codecs in perceptual quality and significantly reduces the flickering artifacts that persist in prior generative approaches, even below 0.01 bpp, highlighting the promise of integrating video-native generative priors into neural codecs for next-generation perceptual video compression.
693287186d1060ca587a27f2
Qi Mao
claimed_verified
2025-12-05T08:28:52.549Z
6388a7e98a5dbe2f3dc61faa
https://cdn-avatars.huggingface.co/v1/production/uploads/6388a7e98a5dbe2f3dc61faa/Zj8pR7BG_wOQjsYzosb5p.jpeg
Qi Mao
HelenMao
user
HelenMao
693287186d1060ca587a27f3
Hao Cheng
claimed_verified
2025-12-05T08:28:55.977Z
67c8d9d1a0518b43770cb894
/avatars/278150f028036178627b3cb5c3a0b46f.svg
Cheng Hao
Catastrophe12138
user
Catastrophe12138
693287186d1060ca587a27f4
Tinghan Yang
693287186d1060ca587a27f5
Libiao Jin
693287186d1060ca587a27f6
Siwei Ma
Generative Neural Video Compression via Video Diffusion Prior
We present GNVC-VD, the first DiT-based generative neural video compression framework built upon an advanced video generation foundation model, where spatio-temporal latent compression and sequence-level generative refinement are unified within a single codec. Existing perceptual codecs primarily rely on pre-trained image generative priors to restore high-frequency details, but their frame-wise nature lacks temporal modeling and inevitably leads to perceptual flickering. To address this, GNVC-VD introduces a unified flow-matching latent refinement module that leverages a video diffusion transformer to jointly enhance intra- and inter-frame latents through sequence-level denoising, ensuring consistent spatio-temporal details. Instead of denoising from pure Gaussian noise as in video generation, GNVC-VD initializes refinement from decoded spatio-temporal latents and learns a correction term that adapts the diffusion prior to compression-induced degradation. A conditioning adaptor further injects compression-aware cues into intermediate DiT layers, enabling effective artifact removal while maintaining temporal coherence under extreme bitrate constraints. Extensive experiments show that GNVC-VD surpasses both traditional and learned codecs in perceptual quality and significantly reduces the flickering artifacts that persist in prior generative approaches, even below 0.01 bpp, highlighting the promise of integrating video-native generative priors into neural codecs for next-generation perceptual video compression.
GNVC-VD, a DiT-based generative neural video compression framework, integrates spatio-temporal latent compression and sequence-level generative refinement to improve perceptual quality and reduce flickering artifacts.
DiT-based generative neural video compression spatio-temporal latent compression sequence-level generative refinement unified codec perceptual codecs pre-trained image generative priors frame-wise nature temporal modeling perceptual flickering unified flow-matching latent refinement module video diffusion transformer sequence-level denoising intra-frame latents inter-frame latents Gaussian noise decoded spatio-temporal latents correction term diffusion prior compression-induced degradation conditioning adaptor intermediate DiT layers artifact removal temporal coherence bitrate constraints next-generation perceptual video compression