Pretraining Frame Preservation in Autoregressive Video Memory Compression
We present PFP, a neural network structure to compress long videos into short contexts, with an explicit pretraining objective to preserve the high-frequency details of single frames at arbitrary temporal positions. The baseline model can compress a 20-second video into a context at about 5k length, where random frames can be retrieved with perceptually preserved appearances. Such pretrained models can be directly fine-tuned as memory encoders for autoregressive video models, enabling long history memory with low context cost and relatively low fidelity loss. We evaluate the framework with ablative settings and discuss the trade-offs of possible neural architecture designs.
6955e31a832867f2535255ba
Lvmin Zhang
6955e31a832867f2535255bb
Shengqu Cai
6955e31a832867f2535255bc
Muyang Li
6955e31a832867f2535255bd
Chong Zeng
claimed_verified
2026-01-28T11:32:34.661Z
60747cbf3ea03830676542b5
https://cdn-avatars.huggingface.co/v1/production/uploads/60747cbf3ea03830676542b5/wGr1Jzz520JM9nZ-UcLyb.png
Chong Zeng
NCJ
user
NCJ
6955e31a832867f2535255be
Beijia Lu
6955e31a832867f2535255bf
Anyi Rao
claimed_verified
2026-01-02T15:39:54.196Z
63f8130749569335b679af62
https://cdn-avatars.huggingface.co/v1/production/uploads/63f8130749569335b679af62/vgTu23-y0UKocwAGqNMwT.jpeg
Anyi Rao
anyirao
user
anyirao
6955e31a832867f2535255c0
Song Han
6955e31a832867f2535255c1
Gordon Wetzstein
6955e31a832867f2535255c2
Maneesh Agrawala
Pretraining Frame Preservation in Autoregressive Video Memory Compression
We present PFP, a neural network structure to compress long videos into short contexts, with an explicit pretraining objective to preserve the high-frequency details of single frames at arbitrary temporal positions. The baseline model can compress a 20-second video into a context at about 5k length, where random frames can be retrieved with perceptually preserved appearances. Such pretrained models can be directly fine-tuned as memory encoders for autoregressive video models, enabling long history memory with low context cost and relatively low fidelity loss. We evaluate the framework with ablative settings and discuss the trade-offs of possible neural architecture designs.