Unified Latents (UL): How to train your latents
We present Unified Latents (UL), a framework for learning latent representations that are jointly regularized by a diffusion prior and decoded by a diffusion model. By linking the encoder's output noise to the prior's minimum noise level, we obtain a simple training objective that provides a tight upper bound on the latent bitrate. On ImageNet-512, our approach achieves competitive FID of 1.4, with high reconstruction quality (PSNR) while requiring fewer training FLOPs than models trained on Stable Diffusion latents. On Kinetics-600, we set a new state-of-the-art FVD of 1.3.
6997ce527a658569d5a10165
Jonathan Heek
6997ce527a658569d5a10166
Emiel Hoogeboom
admin_assigned
2026-02-20T09:03:14.323Z
671feaa3e269c38584cb065f
/avatars/d7bc35ab98b208ef1d009aa9b629682a.svg
Emiel Hoogeboom
Emiel2
user
Emiel2
6997ce527a658569d5a10167
Thomas Mensink
admin_assigned
2026-02-20T09:03:21.125Z
642ad5bc508b7246f9d27ab9
/avatars/6cdf6f855536896575733e273388d7fb.svg
Thomas Mensink
tmensink
user
tmensink
6997ce527a658569d5a10168
Tim Salimans
Unified Latents (UL): How to train your latents
We present Unified Latents (UL), a framework for learning latent representations that are jointly regularized by a diffusion prior and decoded by a diffusion model. By linking the encoder's output noise to the prior's minimum noise level, we obtain a simple training objective that provides a tight upper bound on the latent bitrate. On ImageNet-512, our approach achieves competitive FID of 1.4, with high reconstruction quality (PSNR) while requiring fewer training FLOPs than models trained on Stable Diffusion latents. On Kinetics-600, we set a new state-of-the-art FVD of 1.3.
Unified Latents framework learns joint latent representations using diffusion prior regularization and diffusion model decoding, achieving competitive FID scores with reduced training compute.
diffusion prior diffusion model latent representations training objective latent bitrate FID PSNR FLOPs FVD