WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoning
Recent advances in video large language models have demonstrated strong capabilities in understanding short clips. However, scaling them to hours- or days-long videos remains highly challenging due to limited context capacity and the loss of critical visual details during abstraction. Existing memory-augmented methods mitigate this by leveraging textual summaries of video segments, yet they heavily rely on text and fail to utilize visual evidence when reasoning over complex scenes. Moreover, retrieving from fixed temporal scales further limits their flexibility in capturing events that span variable durations. To address this, we introduce WorldMM, a novel multimodal memory agent that constructs and retrieves from multiple complementary memories, encompassing both textual and visual representations. WorldMM comprises three types of memory: episodic memory indexes factual events across multiple temporal scales, semantic memory continuously updates high-level conceptual knowledge, and visual memory preserves detailed information about scenes. During inference, an adaptive retrieval agent iteratively selects the most relevant memory source and leverages multiple temporal granularities based on the query, continuing until it determines that sufficient information has been gathered. WorldMM significantly outperforms existing baselines across five long video question-answering benchmarks, achieving an average 8.4% performance gain over previous state-of-the-art methods, showing its effectiveness on long video reasoning.
692faddf26742347f61dac4c
Woongyeong Yeo
claimed_verified
2025-12-03T09:15:47.690Z
66d30f5fad293ffc4b7672bc
/avatars/6f164d813b947940a088820f8fd4dbe8.svg
Woongyeong Yeo
wgcyeo
user
wgcyeo
692faddf26742347f61dac4d
Kangsan Kim
692faddf26742347f61dac4e
Jaehong Yoon
claimed_verified
2026-01-05T08:28:05.469Z
652066649004117947e46ed6
/avatars/972c97df6f26d2c3d6ce71ec579984bb.svg
Jaehong Yoon
jaehong31
user
jaehong31
692faddf26742347f61dac4f
Sung Ju Hwang
WorldMM: Dynamic Multimodal Memory Agent for Long Video Reasoning
Recent advances in video large language models have demonstrated strong capabilities in understanding short clips. However, scaling them to hours- or days-long videos remains highly challenging due to limited context capacity and the loss of critical visual details during abstraction. Existing memory-augmented methods mitigate this by leveraging textual summaries of video segments, yet they heavily rely on text and fail to utilize visual evidence when reasoning over complex scenes. Moreover, retrieving from fixed temporal scales further limits their flexibility in capturing events that span variable durations. To address this, we introduce WorldMM, a novel multimodal memory agent that constructs and retrieves from multiple complementary memories, encompassing both textual and visual representations. WorldMM comprises three types of memory: episodic memory indexes factual events across multiple temporal scales, semantic memory continuously updates high-level conceptual knowledge, and visual memory preserves detailed information about scenes. During inference, an adaptive retrieval agent iteratively selects the most relevant memory source and leverages multiple temporal granularities based on the query, continuing until it determines that sufficient information has been gathered. WorldMM significantly outperforms existing baselines across five long video question-answering benchmarks, achieving an average 8.4% performance gain over previous state-of-the-art methods, showing its effectiveness on long video reasoning.
WorldMM, a novel multimodal memory agent with episodic, semantic, and visual memory, outperforms existing methods in long video question-answering by adaptively retrieving from multiple temporal scales and memory sources.
video large language models memory-augmented methods textual summaries visual evidence multimodal memory agent episodic memory semantic memory visual memory adaptive retrieval agent long video question-answering benchmarks