MIND: Benchmarking Memory Consistency and Action Control in World Models
World models aim to understand, remember, and predict dynamic visual environments, yet a unified benchmark for evaluating their fundamental abilities remains lacking. To address this gap, we introduce MIND, the first open-domain closed-loop revisited benchmark for evaluating Memory consIstency and action coNtrol in worlD models. MIND contains 250 high-quality videos at 1080p and 24 FPS, including 100 (first-person) + 100 (third-person) video clips under a shared action space and 25 + 25 clips across varied action spaces covering eight diverse scenes. We design an efficient evaluation framework to measure two core abilities: memory consistency and action control, capturing temporal stability and contextual coherence across viewpoints. Furthermore, we design various action spaces, including different character movement speeds and camera rotation angles, to evaluate the action generalization capability across different action spaces under shared scenes. To facilitate future performance benchmarking on MIND, we introduce MIND-World, a novel interactive Video-to-World baseline. Extensive experiments demonstrate the completeness of MIND and reveal key challenges in current world models, including the difficulty of maintaining long-term memory consistency and generalizing across action spaces. Project page: https://csu-jpg.github.io/MIND.github.io/
698c7597eb12ea7453916837
Yixuan Ye
claimed_verified
2026-02-12T13:57:38.731Z
68fc3ddcdc9e5cbf49cbc716
https://cdn-avatars.huggingface.co/v1/production/uploads/68fc3ddcdc9e5cbf49cbc716/gzvksq-XgWnekB6Xl25pw.jpeg
EasonYe
EasonUwU
user
EasonUwU
698c7597eb12ea7453916838
Xuanyu Lu
698c7597eb12ea7453916839
Yuxin Jiang
698c7597eb12ea745391683a
Yuchao Gu
698c7597eb12ea745391683b
Rui Zhao
698c7597eb12ea745391683c
Qiwei Liang
698c7597eb12ea745391683d
Jiachun Pan
698c7597eb12ea745391683e
Fengda Zhang
698c7597eb12ea745391683f
Weijia Wu
698c7597eb12ea7453916840
Alex Jinpeng Wang
MIND: Benchmarking Memory Consistency and Action Control in World Models
World models aim to understand, remember, and predict dynamic visual environments, yet a unified benchmark for evaluating their fundamental abilities remains lacking. To address this gap, we introduce MIND, the first open-domain closed-loop revisited benchmark for evaluating Memory consIstency and action coNtrol in worlD models. MIND contains 250 high-quality videos at 1080p and 24 FPS, including 100 (first-person) + 100 (third-person) video clips under a shared action space and 25 + 25 clips across varied action spaces covering eight diverse scenes. We design an efficient evaluation framework to measure two core abilities: memory consistency and action control, capturing temporal stability and contextual coherence across viewpoints. Furthermore, we design various action spaces, including different character movement speeds and camera rotation angles, to evaluate the action generalization capability across different action spaces under shared scenes. To facilitate future performance benchmarking on MIND, we introduce MIND-World, a novel interactive Video-to-World baseline. Extensive experiments demonstrate the completeness of MIND and reveal key challenges in current world models, including the difficulty of maintaining long-term memory consistency and generalizing across action spaces. Project page: https://csu-jpg.github.io/MIND.github.io/
MIND is introduced as the first open-domain closed-loop benchmark for evaluating memory consistency and action control in world models, featuring high-quality videos and diverse action spaces to assess temporal stability and contextual coherence.
world models memory consistency action control closed-loop benchmark interactive Video-to-World baseline temporal stability contextual coherence action generalization