DreamWorld: Unified World Modeling in Video Generation
Despite impressive progress in video generation, existing models remain limited to surface-level plausibility, lacking a coherent and unified understanding of the world. Prior approaches typically incorporate only a single form of world-related knowledge or rely on rigid alignment strategies to introduce additional knowledge. However, aligning the single world knowledge is insufficient to constitute a world model that requires jointly modeling multiple heterogeneous dimensions (e.g., physical commonsense, 3D and temporal consistency). To address this limitation, we introduce DreamWorld, a unified framework that integrates complementary world knowledge into video generators via a Joint World Modeling Paradigm, jointly predicting video pixels and features from foundation models to capture temporal dynamics, spatial geometry, and semantic consistency. However, naively optimizing these heterogeneous objectives can lead to visual instability and temporal flickering. To mitigate this issue, we propose Consistent Constraint Annealing (CCA) to progressively regulate world-level constraints during training, and Multi-Source Inner-Guidance to enforce learned world priors at inference. Extensive evaluations show that DreamWorld improves world consistency, outperforming Wan2.1 by 2.26 points on VBench. Code will be made publicly available at https://github.com/ABU121111/DreamWorld{mypink{Github}}.
69aa4fb8c4f6edddcc6c86c9
Boming Tan
69aa4fb8c4f6edddcc6c86ca
Xiangdong Zhang
69aa4fb8c4f6edddcc6c86cb
Ning Liao
claimed_verified
2026-03-13T08:15:04.784Z
6281ebe63e7734836f2f7724
https://cdn-avatars.huggingface.co/v1/production/uploads/6281ebe63e7734836f2f7724/lf5by3U79jW_mEKmJgVh2.jpeg
NingLiao
NingLiao
user
NingLiao
69aa4fb8c4f6edddcc6c86cc
Yuqing Zhang
69aa4fb8c4f6edddcc6c86cd
Shaofeng Zhang
69aa4fb8c4f6edddcc6c86ce
Xue Yang
69aa4fb8c4f6edddcc6c86cf
Qi Fan
69aa4fb8c4f6edddcc6c86d0
Yanyong Zhang
DreamWorld: Unified World Modeling in Video Generation
Despite impressive progress in video generation, existing models remain limited to surface-level plausibility, lacking a coherent and unified understanding of the world. Prior approaches typically incorporate only a single form of world-related knowledge or rely on rigid alignment strategies to introduce additional knowledge. However, aligning the single world knowledge is insufficient to constitute a world model that requires jointly modeling multiple heterogeneous dimensions (e.g., physical commonsense, 3D and temporal consistency). To address this limitation, we introduce DreamWorld, a unified framework that integrates complementary world knowledge into video generators via a Joint World Modeling Paradigm, jointly predicting video pixels and features from foundation models to capture temporal dynamics, spatial geometry, and semantic consistency. However, naively optimizing these heterogeneous objectives can lead to visual instability and temporal flickering. To mitigate this issue, we propose Consistent Constraint Annealing (CCA) to progressively regulate world-level constraints during training, and Multi-Source Inner-Guidance to enforce learned world priors at inference. Extensive evaluations show that DreamWorld improves world consistency, outperforming Wan2.1 by 2.26 points on VBench. Code will be made publicly available at https://github.com/ABU121111/DreamWorld{mypink{Github}}.
DreamWorld introduces a unified framework for video generation that integrates multiple types of world knowledge through joint modeling of temporal dynamics, spatial geometry, and semantic consistency, addressing limitations in existing models' understanding of physical and temporal relationships.
video generation world model joint world modeling paradigm temporal dynamics spatial geometry semantic consistency visual stability temporal flickering consistent constraint annealing multi-source inner-guidance