RANKVIDEO: Reasoning Reranking for Text-to-Video Retrieval
Reranking is a critical component of modern retrieval systems, which typically pair an efficient first-stage retriever with a more expressive model to refine results. While large reasoning models have driven rapid progress in text-centric reranking, reasoning-based reranking for video retrieval remains underexplored. To address this gap, we introduce RANKVIDEO, a reasoning-based reranker for video retrieval that explicitly reasons over query-video pairs using video content to assess relevance. RANKVIDEO is trained using a two-stage curriculum consisting of perception-grounded supervised fine-tuning followed by reranking training that combines pointwise, pairwise, and teacher confidence distillation objectives, and is supported by a data synthesis pipeline for constructing reasoning-intensive query-video pairs. Experiments on the large-scale MultiVENT 2.0 benchmark demonstrate that RANKVIDEO consistently improves retrieval performance within a two-stage framework, yielding an average improvement of 31% on nDCG@10 and outperforming text-only and vision-language reranking alternatives, while more efficient.
69835a0237bca54cd0587d6f
Tyler Skow
claimed_verified
2026-02-04T19:03:29.499Z
67277f6a8baabd079b017d5f
/avatars/58fd8d2be0f953e22e332923cbf69c41.svg
Tyler Skow
tskow21
user
tskow21
69835a0237bca54cd0587d70
Alexander Martin
claimed_verified
2026-03-03T10:44:20.900Z
644ffb154f731658826945c8
/avatars/46fa3a15c2b974692bee5b6ed6468be0.svg
Alex Martin
alexmartin1722
user
alexmartin1722
69835a0237bca54cd0587d71
Benjamin Van Durme
69835a0237bca54cd0587d72
Rama Chellappa
69835a0237bca54cd0587d73
Reno Kriz
RANKVIDEO: Reasoning Reranking for Text-to-Video Retrieval
Reranking is a critical component of modern retrieval systems, which typically pair an efficient first-stage retriever with a more expressive model to refine results. While large reasoning models have driven rapid progress in text-centric reranking, reasoning-based reranking for video retrieval remains underexplored. To address this gap, we introduce RANKVIDEO, a reasoning-based reranker for video retrieval that explicitly reasons over query-video pairs using video content to assess relevance. RANKVIDEO is trained using a two-stage curriculum consisting of perception-grounded supervised fine-tuning followed by reranking training that combines pointwise, pairwise, and teacher confidence distillation objectives, and is supported by a data synthesis pipeline for constructing reasoning-intensive query-video pairs. Experiments on the large-scale MultiVENT 2.0 benchmark demonstrate that RANKVIDEO consistently improves retrieval performance within a two-stage framework, yielding an average improvement of 31% on nDCG@10 and outperforming text-only and vision-language reranking alternatives, while more efficient.
RANKVIDEO is a reasoning-based video retrieval system that improves upon traditional two-stage frameworks through explicit query-video pair analysis and a multi-objective training approach.
reranking video retrieval reasoning-based reranker curriculum training supervised fine-tuning pointwise pairwise teacher confidence distillation data synthesis pipeline MultiVENT 2.0 benchmark