LongVideoAgent: Multi-Agent Reasoning with Long Videos
Recent advances in multimodal LLMs and systems that use tools for long-video QA point to the promise of reasoning over hour-long episodes. However, many methods still compress content into lossy summaries or rely on limited toolsets, weakening temporal grounding and missing fine-grained cues. We propose a multi-agent framework in which a master LLM coordinates a grounding agent to localize question-relevant segments and a vision agent to extract targeted textual observations. The master agent plans with a step limit, and is trained with reinforcement learning to encourage concise, correct, and efficient multi-agent cooperation. This design helps the master agent focus on relevant clips via grounding, complements subtitles with visual detail, and yields interpretable trajectories. On our proposed LongTVQA and LongTVQA+ which are episode-level datasets aggregated from TVQA/TVQA+, our multi-agent system significantly outperforms strong non-agent baselines. Experiments also show reinforcement learning further strengthens reasoning and planning for the trained agent. Code and data will be shared at https://longvideoagent.github.io/.
694ba02a746a34b55dd53e8b
Runtao Liu
claimed_verified
2025-12-25T20:44:49.001Z
67433c5ffc499ac0b61b7b8b
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/UmWQHO2kgkGEXt1qP9zXS.png
Rt Liu
rtliu
user
rtliu
694ba02a746a34b55dd53e8c
Ziyi Liu
694ba02a746a34b55dd53e8d
Jiaqi Tang
claimed_verified
2025-12-25T20:44:45.529Z
642e7a12ccdcf5da7f9657a0
https://cdn-avatars.huggingface.co/v1/production/uploads/642e7a12ccdcf5da7f9657a0/w8jW5BagTuTp6EvC6KEyR.png
Jiaqi Tang
Jiaqi-hkust
user
Jiaqi-hkust
694ba02a746a34b55dd53e8e
Yue Ma
694ba02a746a34b55dd53e8f
Renjie Pi
694ba02a746a34b55dd53e90
Jipeng Zhang
694ba02a746a34b55dd53e91
Qifeng Chen
LongVideoAgent: Multi-Agent Reasoning with Long Videos
Recent advances in multimodal LLMs and systems that use tools for long-video QA point to the promise of reasoning over hour-long episodes. However, many methods still compress content into lossy summaries or rely on limited toolsets, weakening temporal grounding and missing fine-grained cues. We propose a multi-agent framework in which a master LLM coordinates a grounding agent to localize question-relevant segments and a vision agent to extract targeted textual observations. The master agent plans with a step limit, and is trained with reinforcement learning to encourage concise, correct, and efficient multi-agent cooperation. This design helps the master agent focus on relevant clips via grounding, complements subtitles with visual detail, and yields interpretable trajectories. On our proposed LongTVQA and LongTVQA+ which are episode-level datasets aggregated from TVQA/TVQA+, our multi-agent system significantly outperforms strong non-agent baselines. Experiments also show reinforcement learning further strengthens reasoning and planning for the trained agent. Code and data will be shared at https://longvideoagent.github.io/.
A multi-agent framework, involving a master LLM, grounding agent, and vision agent, enhances long-video QA by improving temporal grounding and leveraging visual and textual data.
multimodal LLMs long-video QA multi-agent framework grounding agent vision agent reinforcement learning temporal grounding LongTVQA LongTVQA+