Structured Causal Video Reasoning via Multi-Objective Alignment
Human understanding of video dynamics is typically grounded in a structured mental representation of entities, actions, and temporal relations, rather than relying solely on immediate deductive reasoning. In contrast, existing Video-LLMs largely depend on unstructured video reasoning, where critical visual evidence is embedded in verbose textual descriptions and temporal causality is often weakly modeled. This leads to inefficient processes and fragile causal inference. To bridge this cognitive gap, we propose constructing a compact representation of salient events and their causal relationships, which we name Structured Event Facts, prior to the reasoning stage. This structured prior serves as an explicit constraint to promote concise and causally grounded reasoning, while also making intermediate evidence easier to verify. To effectively train models on such structured facts, we introduce CausalFact-60K and a four-stage training pipeline comprising facts alignment, format warm-start, thinking warm-start, and reinforcement learning-based post-training. During RL stage, we find that this framework introduces competing objectives, as structural completeness and causal fidelity must be balanced against reasoning length, making it difficult to optimize. We address this challenge by formulating the optimization as a Multi-Objective Reinforcement Learning (MORL) problem and explicitly optimizing toward the Pareto-Frontier to balance these trade-offs. As a result, we introduce Factum-4B, which yields more reliable reasoning and delivers stronger performance on challenging video understanding tasks requiring fine-grained temporal inference.
69dc5e972b6b8dc66ffeebec
Zinuo Li
69dc5e972b6b8dc66ffeebed
Yongxin Guo
claimed_verified
2026-04-13T07:32:09.172Z
652c9bdf4d0eef4a2b3c577f
/avatars/6e92eee19ae2c43e46a8b294c02f9915.svg
Yongxin Guo
Yongxin-Guo
user
Yongxin-Guo
69dc5e972b6b8dc66ffeebee
Jun Liu
69dc5e972b6b8dc66ffeebef
Jiawei Zhan
69dc5e972b6b8dc66ffeebf0
Xi Jiang
69dc5e972b6b8dc66ffeebf1
Chengjie Wang
69dc5e972b6b8dc66ffeebf2
Mohammed Bennamoun
69dc5e972b6b8dc66ffeebf3
Farid Boussaid
69dc5e972b6b8dc66ffeebf4
Feng Zheng
69dc5e972b6b8dc66ffeebf5
Qiuhong Ke
Structured Causal Video Reasoning via Multi-Objective Alignment
Human understanding of video dynamics is typically grounded in a structured mental representation of entities, actions, and temporal relations, rather than relying solely on immediate deductive reasoning. In contrast, existing Video-LLMs largely depend on unstructured video reasoning, where critical visual evidence is embedded in verbose textual descriptions and temporal causality is often weakly modeled. This leads to inefficient processes and fragile causal inference. To bridge this cognitive gap, we propose constructing a compact representation of salient events and their causal relationships, which we name Structured Event Facts, prior to the reasoning stage. This structured prior serves as an explicit constraint to promote concise and causally grounded reasoning, while also making intermediate evidence easier to verify. To effectively train models on such structured facts, we introduce CausalFact-60K and a four-stage training pipeline comprising facts alignment, format warm-start, thinking warm-start, and reinforcement learning-based post-training. During RL stage, we find that this framework introduces competing objectives, as structural completeness and causal fidelity must be balanced against reasoning length, making it difficult to optimize. We address this challenge by formulating the optimization as a Multi-Objective Reinforcement Learning (MORL) problem and explicitly optimizing toward the Pareto-Frontier to balance these trade-offs. As a result, we introduce Factum-4B, which yields more reliable reasoning and delivers stronger performance on challenging video understanding tasks requiring fine-grained temporal inference.
Video-LLMs trained on structured event facts with causal relationships outperform existing methods on complex video understanding tasks requiring precise temporal inference.
Video-LLMs structured event facts causal relationships temporal inference CausalFact-60K four-stage training pipeline facts alignment format warm-start thinking warm-start reinforcement learning Multi-Objective Reinforcement Learning Pareto-Frontier Factum-4B