SAGE: Training Smart Any-Horizon Agents for Long Video Reasoning with Reinforcement Learning
As humans, we are natural any-horizon reasoners, i.e., we can decide whether to iteratively skim long videos or watch short ones in full when necessary for a given task. With this in mind, one would expect video reasoning models to reason flexibly across different durations. However, SOTA models are still trained to predict answers in a single turn while processing a large number of frames, akin to watching an entire long video, requiring significant resources. This raises the question: Is it possible to develop performant any-horizon video reasoning systems? Inspired by human behavior, we first propose SAGE, an agent system that performs multi-turn reasoning on long videos while handling simpler problems in a single turn. Secondly, we introduce an easy synthetic data generation pipeline using Gemini-2.5-Flash to train the orchestrator, SAGE-MM, which lies at the core of SAGE. We further propose an effective RL post-training recipe essential for instilling any-horizon reasoning ability in SAGE-MM. Thirdly, we curate SAGE-Bench with an average duration of greater than 700 seconds for evaluating video reasoning ability in real-world entertainment use cases. Lastly, we empirically validate the effectiveness of our system, data, and RL recipe, observing notable improvements of up to 6.1% on open-ended video reasoning tasks, as well as an impressive 8.2% improvement on videos longer than 10 minutes.
694376f5542d62d58a7bf73e
Jitesh Jain
claimed_verified
2025-12-18T07:59:47.204Z
623dfe96dcda6a715304cbca
https://cdn-avatars.huggingface.co/v1/production/uploads/623dfe96dcda6a715304cbca/B7V_IbQNXAcotOKE_mJQ4.jpeg
Jitesh Jain
praeclarumjj3
user
praeclarumjj3
694376f5542d62d58a7bf73f
Jialuo Li
claimed_verified
2025-12-19T09:07:07.678Z
6555a56f7cca2ba493f5331a
https://cdn-avatars.huggingface.co/v1/production/uploads/6555a56f7cca2ba493f5331a/d1_JZic4rFi80xe7m7VWs.jpeg
Jialuo Li
Jialuo21
user
Jialuo21
694376f5542d62d58a7bf740
Zixian Ma
claimed_verified
2026-03-30T09:12:21.665Z
6442cce0e255a338677e59ff
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VheGtt37p7aYIodlt9N3n.jpeg
Zixian Ma
zixianma
user
zixianma
694376f5542d62d58a7bf741
Jieyu Zhang
694376f5542d62d58a7bf742
Chris Dongjoo Kim
694376f5542d62d58a7bf743
Sangho Lee
694376f5542d62d58a7bf744
Rohun Tripathi
694376f5542d62d58a7bf745
Tanmay Gupta
694376f5542d62d58a7bf746
Christopher Clark
694376f5542d62d58a7bf747
Humphrey Shi
SAGE: Training Smart Any-Horizon Agents for Long Video Reasoning with Reinforcement Learning
As humans, we are natural any-horizon reasoners, i.e., we can decide whether to iteratively skim long videos or watch short ones in full when necessary for a given task. With this in mind, one would expect video reasoning models to reason flexibly across different durations. However, SOTA models are still trained to predict answers in a single turn while processing a large number of frames, akin to watching an entire long video, requiring significant resources. This raises the question: Is it possible to develop performant any-horizon video reasoning systems? Inspired by human behavior, we first propose SAGE, an agent system that performs multi-turn reasoning on long videos while handling simpler problems in a single turn. Secondly, we introduce an easy synthetic data generation pipeline using Gemini-2.5-Flash to train the orchestrator, SAGE-MM, which lies at the core of SAGE. We further propose an effective RL post-training recipe essential for instilling any-horizon reasoning ability in SAGE-MM. Thirdly, we curate SAGE-Bench with an average duration of greater than 700 seconds for evaluating video reasoning ability in real-world entertainment use cases. Lastly, we empirically validate the effectiveness of our system, data, and RL recipe, observing notable improvements of up to 6.1% on open-ended video reasoning tasks, as well as an impressive 8.2% improvement on videos longer than 10 minutes.
Researchers developed SAGE, an agent system that enables flexible video reasoning across different time scales by using multi-turn processing and reinforcement learning, achieving improved performance on long-form video analysis tasks.
SAGE SAGE-MM multi-turn reasoning reinforcement learning video reasoning SAGE-Bench