Adaptive 1D Video Diffusion Autoencoder
Recent video generation models largely rely on video autoencoders that compress pixel-space videos into latent representations. However, existing video autoencoders suffer from three major limitations: (1) fixed-rate compression that wastes tokens on simple videos, (2) inflexible CNN architectures that prevent variable-length latent modeling, and (3) deterministic decoders that struggle to recover appropriate details from compressed latents. To address these issues, we propose One-Dimensional Diffusion Video Autoencoder (One-DVA), a transformer-based framework for adaptive 1D encoding and diffusion-based decoding. The encoder employs query-based vision transformers to extract spatiotemporal features and produce latent representations, while a variable-length dropout mechanism dynamically adjusts the latent length. The decoder is a pixel-space diffusion transformer that reconstructs videos with the latents as input conditions. With a two-stage training strategy, One-DVA achieves performance comparable to 3D-CNN VAEs on reconstruction metrics at identical compression ratios. More importantly, it supports adaptive compression and thus can achieve higher compression ratios. To better support downstream latent generation, we further regularize the One-DVA latent distribution for generative modeling and fine-tune its decoder to mitigate artifacts caused by the generation process.
6985abed4ad556f294b7ed41
Yao Teng
6985abed4ad556f294b7ed42
Minxuan Lin
claimed_verified
2026-02-12T20:28:49.742Z
62fcbf8c085065dcf8667ffd
https://cdn-avatars.huggingface.co/v1/production/uploads/62fcbf8c085065dcf8667ffd/yWn5ASwtptr3Rz_omEKXj.png
Minxuan Lin
kingofprank
user
kingofprank
6985abed4ad556f294b7ed43
Xian Liu
6985abed4ad556f294b7ed44
Shuai Wang
claimed_verified
2026-02-11T11:20:12.493Z
66615c855fd9d736e670e0a9
/avatars/0ff3127b513552432a7c651e21d7f283.svg
wangshuai
wangsssssss
user
wangsssssss
6985abed4ad556f294b7ed45
Xiao Yang
claimed_verified
2026-03-05T09:58:54.684Z
62b3d2490385d6048985b42b
/avatars/1c4e534d086d53ef1057fbc00334350a.svg
Xiao Yang
seravee008
user
seravee008
6985abed4ad556f294b7ed46
Xihui Liu
Adaptive 1D Video Diffusion Autoencoder
Recent video generation models largely rely on video autoencoders that compress pixel-space videos into latent representations. However, existing video autoencoders suffer from three major limitations: (1) fixed-rate compression that wastes tokens on simple videos, (2) inflexible CNN architectures that prevent variable-length latent modeling, and (3) deterministic decoders that struggle to recover appropriate details from compressed latents. To address these issues, we propose One-Dimensional Diffusion Video Autoencoder (One-DVA), a transformer-based framework for adaptive 1D encoding and diffusion-based decoding. The encoder employs query-based vision transformers to extract spatiotemporal features and produce latent representations, while a variable-length dropout mechanism dynamically adjusts the latent length. The decoder is a pixel-space diffusion transformer that reconstructs videos with the latents as input conditions. With a two-stage training strategy, One-DVA achieves performance comparable to 3D-CNN VAEs on reconstruction metrics at identical compression ratios. More importantly, it supports adaptive compression and thus can achieve higher compression ratios. To better support downstream latent generation, we further regularize the One-DVA latent distribution for generative modeling and fine-tune its decoder to mitigate artifacts caused by the generation process.
A transformer-based video autoencoder with adaptive 1D encoding and diffusion-based decoding addresses limitations of fixed-rate compression and deterministic reconstruction in video compression.
video autoencoders pixel-space videos latent representations vision transformers spatiotemporal features variable-length dropout diffusion transformer reconstruction metrics compression ratios generative modeling