DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planning
Recently, world-action models (WAM) have emerged to bridge vision-language-action (VLA) models and world models, unifying their reasoning and instruction-following capabilities and spatio-temporal world modeling. However, existing WAM approaches often focus on modeling 2D appearance or latent representations, with limited geometric grounding-an essential element for embodied systems operating in the physical world. We present DriveDreamer-Policy, a unified driving world-action model that integrates depth generation, future video generation, and motion planning within a single modular architecture. The model employs a large language model to process language instructions, multi-view images, and actions, followed by three lightweight generators that produce depth, future video, and actions. By learning a geometry-aware world representation and using it to guide both future prediction and planning within a unified framework, the proposed model produces more coherent imagined futures and more informed driving actions, while maintaining modularity and controllable latency. Experiments on the Navsim v1 and v2 benchmarks demonstrate that DriveDreamer-Policy achieves strong performance on both closed-loop planning and world generation tasks. In particular, our model reaches 89.2 PDMS on Navsim v1 and 88.7 EPDMS on Navsim v2, outperforming existing world-model-based approaches while producing higher-quality future video and depth predictions. Ablation studies further show that explicit depth learning provides complementary benefits to video imagination and improves planning robustness.
69d45d962c523d36431fafdc
Yang Zhou
69d45d962c523d36431fafdd
Xiaofeng Wang
claimed_verified
2026-04-07T08:34:59.710Z
6426616ea5ec4a5cbc535634
https://cdn-avatars.huggingface.co/v1/production/uploads/6426616ea5ec4a5cbc535634/5IfSFYd9QOxz8K9QmBCst.png
JeffWang
Jeff-Wang
user
Jeff-Wang
69d45d962c523d36431fafde
Hao Shao
69d45d962c523d36431fafdf
Letian Wang
69d45d962c523d36431fafe0
Guosheng Zhao
69d45d962c523d36431fafe1
Jiangnan Shao
69d45d962c523d36431fafe2
Jiagang Zhu
69d45d962c523d36431fafe3
Tingdong Yu
69d45d962c523d36431fafe4
Zheng Zhu
69d45d962c523d36431fafe5
Guan Huang
69d45d962c523d36431fafe6
Steven L. Waslander
DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planning
Recently, world-action models (WAM) have emerged to bridge vision-language-action (VLA) models and world models, unifying their reasoning and instruction-following capabilities and spatio-temporal world modeling. However, existing WAM approaches often focus on modeling 2D appearance or latent representations, with limited geometric grounding-an essential element for embodied systems operating in the physical world. We present DriveDreamer-Policy, a unified driving world-action model that integrates depth generation, future video generation, and motion planning within a single modular architecture. The model employs a large language model to process language instructions, multi-view images, and actions, followed by three lightweight generators that produce depth, future video, and actions. By learning a geometry-aware world representation and using it to guide both future prediction and planning within a unified framework, the proposed model produces more coherent imagined futures and more informed driving actions, while maintaining modularity and controllable latency. Experiments on the Navsim v1 and v2 benchmarks demonstrate that DriveDreamer-Policy achieves strong performance on both closed-loop planning and world generation tasks. In particular, our model reaches 89.2 PDMS on Navsim v1 and 88.7 EPDMS on Navsim v2, outperforming existing world-model-based approaches while producing higher-quality future video and depth predictions. Ablation studies further show that explicit depth learning provides complementary benefits to video imagination and improves planning robustness.
DriveDreamer-Policy integrates depth generation, future video prediction, and motion planning in a unified modular framework for driving applications, achieving superior performance on navigation benchmarks through geometry-aware world representation learning.
world-action models vision-language-action models world models embodied systems depth generation future video generation motion planning large language model multi-view images lightweight generators geometry-aware world representation closed-loop planning world generation tasks Navsim v1 Navsim v2 PDMS EPDMS