TokenTrim: Inference-Time Token Pruning for Autoregressive Long Video Generation
Auto-regressive video generation enables long video synthesis by iteratively conditioning each new batch of frames on previously generated content. However, recent work has shown that such pipelines suffer from severe temporal drift, where errors accumulate and amplify over long horizons. We hypothesize that this drift does not primarily stem from insufficient model capacity, but rather from inference-time error propagation. Specifically, we contend that drift arises from the uncontrolled reuse of corrupted latent conditioning tokens during auto-regressive inference. To correct this accumulation of errors, we propose a simple, inference-time method that mitigates temporal drift by identifying and removing unstable latent tokens before they are reused for conditioning. For this purpose, we define unstable tokens as latent tokens whose representations deviate significantly from those of the previously generated batch, indicating potential corruption or semantic drift. By explicitly removing corrupted latent tokens from the auto-regressive context, rather than modifying entire spatial regions or model parameters, our method prevents unreliable latent information from influencing future generation steps. As a result, it significantly improves long-horizon temporal consistency without modifying the model architecture, training procedure, or leaving latent space.
698c6b52eb12ea7453916823
Ariel Shaulov
claimed_verified
2026-02-11T12:34:28.330Z
65d4985d4e358ce02a949f8c
/avatars/3eda6f50d17802b1ce94349c89637e3c.svg
Ariel Shaulov
shaulov
user
shaulov
698c6b52eb12ea7453916824
Eitan Shaar
698c6b52eb12ea7453916825
Amit Edenzon
698c6b52eb12ea7453916826
Lior Wolf
TokenTrim: Inference-Time Token Pruning for Autoregressive Long Video Generation
Auto-regressive video generation enables long video synthesis by iteratively conditioning each new batch of frames on previously generated content. However, recent work has shown that such pipelines suffer from severe temporal drift, where errors accumulate and amplify over long horizons. We hypothesize that this drift does not primarily stem from insufficient model capacity, but rather from inference-time error propagation. Specifically, we contend that drift arises from the uncontrolled reuse of corrupted latent conditioning tokens during auto-regressive inference. To correct this accumulation of errors, we propose a simple, inference-time method that mitigates temporal drift by identifying and removing unstable latent tokens before they are reused for conditioning. For this purpose, we define unstable tokens as latent tokens whose representations deviate significantly from those of the previously generated batch, indicating potential corruption or semantic drift. By explicitly removing corrupted latent tokens from the auto-regressive context, rather than modifying entire spatial regions or model parameters, our method prevents unreliable latent information from influencing future generation steps. As a result, it significantly improves long-horizon temporal consistency without modifying the model architecture, training procedure, or leaving latent space.
Auto-regressive video generation suffers from temporal drift due to error accumulation in latent conditioning tokens, which is addressed by identifying and removing unstable tokens during inference to improve long-horizon consistency.
auto-regressive video generation temporal drift latent conditioning tokens inference-time error propagation unstable tokens latent space