Towards Universal Video Retrieval: Generalizing Video Embedding via
  Synthesized Multimodal Pyramid Curriculum
The prevailing video retrieval paradigm is structurally misaligned, as narrow
benchmarks incentivize correspondingly limited data and single-task training.
Therefore, universal capability is suppressed due to the absence of a
diagnostic evaluation that defines and demands multi-dimensional
generalization. To break this cycle, we introduce a framework built on the
co-design of evaluation, data, and modeling. First, we establish the Universal
Video Retrieval Benchmark (UVRB), a suite of 16 datasets designed not only to
measure performance but also to diagnose critical capability gaps across tasks
and domains. Second, guided by UVRB's diagnostics, we introduce a scalable
synthesis workflow that generates 1.55 million high-quality pairs to populate
the semantic space required for universality. Finally, we devise the Modality
Pyramid, a curriculum that trains our General Video Embedder (GVE) by
explicitly leveraging the latent interconnections within our diverse data.
Extensive experiments show GVE achieves state-of-the-art zero-shot
generalization on UVRB. In particular, our analysis reveals that popular
benchmarks are poor predictors of general ability and that partially relevant
retrieval is a dominant but overlooked scenario. Overall, our co-designed
framework provides a practical path to escape the limited scope and advance
toward truly universal video retrieval.
6908681a812eca10f9cc5f5a
Zhuoning Guo
claimed_verified
2025-11-04T20:27:05.194Z
6746b1e2224b22ef67fbff11
https://cdn-avatars.huggingface.co/v1/production/uploads/6746b1e2224b22ef67fbff11/q6sN8lQjtLGCgen_41VxU.jpeg
Zhuoning Guo
Zhuoning
user
Zhuoning
6908681a812eca10f9cc5f5b
Mingxin Li
6908681a812eca10f9cc5f5c
Yanzhao Zhang
6908681a812eca10f9cc5f5d
Dingkun Long
6908681a812eca10f9cc5f5e
Pengjun Xie
6908681a812eca10f9cc5f5f
Xiaowen Chu
Towards Universal Video Retrieval: Generalizing Video Embedding via
  Synthesized Multimodal Pyramid Curriculum
The prevailing video retrieval paradigm is structurally misaligned, as narrow
benchmarks incentivize correspondingly limited data and single-task training.
Therefore, universal capability is suppressed due to the absence of a
diagnostic evaluation that defines and demands multi-dimensional
generalization. To break this cycle, we introduce a framework built on the
co-design of evaluation, data, and modeling. First, we establish the Universal
Video Retrieval Benchmark (UVRB), a suite of 16 datasets designed not only to
measure performance but also to diagnose critical capability gaps across tasks
and domains. Second, guided by UVRB's diagnostics, we introduce a scalable
synthesis workflow that generates 1.55 million high-quality pairs to populate
the semantic space required for universality. Finally, we devise the Modality
Pyramid, a curriculum that trains our General Video Embedder (GVE) by
explicitly leveraging the latent interconnections within our diverse data.
Extensive experiments show GVE achieves state-of-the-art zero-shot
generalization on UVRB. In particular, our analysis reveals that popular
benchmarks are poor predictors of general ability and that partially relevant
retrieval is a dominant but overlooked scenario. Overall, our co-designed
framework provides a practical path to escape the limited scope and advance
toward truly universal video retrieval.
A framework combining a diagnostic benchmark, data synthesis, and a modality pyramid curriculum achieves state-of-the-art zero-shot generalization in video retrieval.
Universal Video Retrieval Benchmark UVRB scalable synthesis workflow Modality Pyramid General Video Embedder GVE zero-shot generalization