A Frame is Worth One Token: Efficient Generative World Modeling with Delta Tokens
Anticipating diverse future states is a central challenge in video world modeling. Discriminative world models produce a deterministic prediction that implicitly averages over possible futures, while existing generative world models remain computationally expensive. Recent work demonstrates that predicting the future in the feature space of a vision foundation model (VFM), rather than a latent space optimized for pixel reconstruction, requires significantly fewer world model parameters. However, most such approaches remain discriminative. In this work, we introduce DeltaTok, a tokenizer that encodes the VFM feature difference between consecutive frames into a single continuous "delta" token, and DeltaWorld, a generative world model operating on these tokens to efficiently generate diverse plausible futures. Delta tokens reduce video from a three-dimensional spatio-temporal representation to a one-dimensional temporal sequence, for example yielding a 1,024x token reduction with 512x512 frames. This compact representation enables tractable multi-hypothesis training, where many futures are generated in parallel and only the best is supervised. At inference, this leads to diverse predictions in a single forward pass. Experiments on dense forecasting tasks demonstrate that DeltaWorld forecasts futures that more closely align with real-world outcomes, while having over 35x fewer parameters and using 2,000x fewer FLOPs than existing generative world models. Code and weights: https://deltatok.github.io.
69d53ebf63f089495f06b2eb
Tommie Kerssies
claimed_verified
2026-04-16T20:22:25.002Z
6368212544e19ccad212bbf2
/avatars/3ab0d1c6273a95f16b7b82c72db4a5e2.svg
Tommie Kerssies
tommiekerssies
user
tommiekerssies
69d53ebf63f089495f06b2ec
Gabriele Berton
69d53ebf63f089495f06b2ed
Ju He
69d53ebf63f089495f06b2ee
Qihang Yu
69d53ebf63f089495f06b2ef
Wufei Ma
69d53ebf63f089495f06b2f0
Daan de Geus
claimed_verified
2026-04-16T20:22:23.053Z
6392408ae836c1426b776472
/avatars/380b66f142197572e4d8516046c82508.svg
Daan de Geus
ddegeus
user
ddegeus
69d53ebf63f089495f06b2f1
Gijs Dubbelman
69d53ebf63f089495f06b2f2
Liang-Chieh Chen
A Frame is Worth One Token: Efficient Generative World Modeling with Delta Tokens
Anticipating diverse future states is a central challenge in video world modeling. Discriminative world models produce a deterministic prediction that implicitly averages over possible futures, while existing generative world models remain computationally expensive. Recent work demonstrates that predicting the future in the feature space of a vision foundation model (VFM), rather than a latent space optimized for pixel reconstruction, requires significantly fewer world model parameters. However, most such approaches remain discriminative. In this work, we introduce DeltaTok, a tokenizer that encodes the VFM feature difference between consecutive frames into a single continuous "delta" token, and DeltaWorld, a generative world model operating on these tokens to efficiently generate diverse plausible futures. Delta tokens reduce video from a three-dimensional spatio-temporal representation to a one-dimensional temporal sequence, for example yielding a 1,024x token reduction with 512x512 frames. This compact representation enables tractable multi-hypothesis training, where many futures are generated in parallel and only the best is supervised. At inference, this leads to diverse predictions in a single forward pass. Experiments on dense forecasting tasks demonstrate that DeltaWorld forecasts futures that more closely align with real-world outcomes, while having over 35x fewer parameters and using 2,000x fewer FLOPs than existing generative world models. Code and weights: https://deltatok.github.io.
DeltaTok encodes visual feature differences as delta tokens and DeltaWorld generates diverse video futures with reduced parameters and computational cost through multi-hypothesis training.
vision foundation model generative world model feature space latent space tokenizer delta tokens multi-hypothesis training dense forecasting video world modeling