CAST: Modeling Visual State Transitions for Consistent Video Retrieval
As video content creation shifts toward long-form narratives, composing short clips into coherent storylines becomes increasingly important. However, prevailing retrieval formulations remain context-agnostic at inference time, prioritizing local semantic alignment while neglecting state and identity consistency. To address this structural limitation, we formalize the task of Consistent Video Retrieval (CVR) and introduce a diagnostic benchmark spanning YouCook2, COIN, and CrossTask. We propose CAST (Context-Aware State Transition), a lightweight, plug-and-play adapter compatible with diverse frozen vision-language embedding spaces. By predicting a state-conditioned residual update (Δ) from visual history, CAST introduces an explicit inductive bias for latent state evolution. Extensive experiments show that CAST improves performance on YouCook2 and CrossTask, remains competitive on COIN, and consistently outperforms zero-shot baselines across diverse foundation backbones. Furthermore, CAST provides a useful reranking signal for black-box video generation candidates (e.g., from Veo), promoting more temporally coherent continuations.
69afa5b4b3ab31dee1281e6f
Yanqing Liu
69afa5b4b3ab31dee1281e70
Yingcheng Liu
69afa5b4b3ab31dee1281e71
Fanghong Dong
69afa5b4b3ab31dee1281e72
Budianto Budianto
69afa5b4b3ab31dee1281e73
Cihang Xie
69afa5b4b3ab31dee1281e74
Yan Jiao
CAST: Modeling Visual State Transitions for Consistent Video Retrieval
As video content creation shifts toward long-form narratives, composing short clips into coherent storylines becomes increasingly important. However, prevailing retrieval formulations remain context-agnostic at inference time, prioritizing local semantic alignment while neglecting state and identity consistency. To address this structural limitation, we formalize the task of Consistent Video Retrieval (CVR) and introduce a diagnostic benchmark spanning YouCook2, COIN, and CrossTask. We propose CAST (Context-Aware State Transition), a lightweight, plug-and-play adapter compatible with diverse frozen vision-language embedding spaces. By predicting a state-conditioned residual update (Δ) from visual history, CAST introduces an explicit inductive bias for latent state evolution. Extensive experiments show that CAST improves performance on YouCook2 and CrossTask, remains competitive on COIN, and consistently outperforms zero-shot baselines across diverse foundation backbones. Furthermore, CAST provides a useful reranking signal for black-box video generation candidates (e.g., from Veo), promoting more temporally coherent continuations.
Consistent Video Retrieval addresses context-agnostic limitations in video content composition by introducing CAST, a context-aware state transition method that improves narrative coherence through state-conditioned visual history prediction.
Consistent Video Retrieval CVR CAST context-aware state transition visual history state-conditioned residual update latent state evolution video retrieval reranking signal video generation