Causal World Modeling for Robot Control
This work highlights that video world modeling, alongside vision-language pre-training, establishes a fresh and independent foundation for robot learning. Intuitively, video world models provide the ability to imagine the near future by understanding the causality between actions and visual dynamics. Inspired by this, we introduce LingBot-VA, an autoregressive diffusion framework that learns frame prediction and policy execution simultaneously. Our model features three carefully crafted designs: (1) a shared latent space, integrating vision and action tokens, driven by a Mixture-of-Transformers (MoT) architecture, (2) a closed-loop rollout mechanism, allowing for ongoing acquisition of environmental feedback with ground-truth observations, (3) an asynchronous inference pipeline, parallelizing action prediction and motor execution to support efficient control. We evaluate our model on both simulation benchmarks and real-world scenarios, where it shows significant promise in long-horizon manipulation, data efficiency in post-training, and strong generalizability to novel configurations. The code and model are made publicly available to facilitate the community.
6980d46983fdbbe1963c2ce6
Lin Li
claimed_verified
2026-02-03T10:08:25.627Z
660a6a412e58edd19e348eb3
/avatars/08aafe53ca81910621db6b6f99d1e835.svg
Lin Li
lilinhitcrt
user
lilinhitcrt
6980d46983fdbbe1963c2ce7
Qihang Zhang
6980d46983fdbbe1963c2ce8
Yiming Luo
6980d46983fdbbe1963c2ce9
Shuai Yang
claimed_verified
2026-02-03T10:08:27.809Z
64548f6c363bb3aaf9cba136
https://cdn-avatars.huggingface.co/v1/production/uploads/64548f6c363bb3aaf9cba136/HqJL9HQ5CWVOJCsHyuMOm.jpeg
Shuai Yang
ShuaiYang03
user
ShuaiYang03
6980d46983fdbbe1963c2cea
Ruilin Wang
6980d46983fdbbe1963c2ceb
Fei Han
6980d46983fdbbe1963c2cec
Mingrui Yu
6980d46983fdbbe1963c2ced
Zelin Gao
6980d46983fdbbe1963c2cee
Nan Xue
6980d46983fdbbe1963c2cef
Xing Zhu
6980d46983fdbbe1963c2cf0
Yujun Shen
6980d46983fdbbe1963c2cf1
Yinghao Xu
claimed_verified
2026-02-04T12:33:21.040Z
6555da9adcf410fd0753569c
/avatars/ac58a796bb54f334fdb475a4b75c4d27.svg
Yinghao Xu
justimyhxu
user
justimyhxu
Causal World Modeling for Robot Control
This work highlights that video world modeling, alongside vision-language pre-training, establishes a fresh and independent foundation for robot learning. Intuitively, video world models provide the ability to imagine the near future by understanding the causality between actions and visual dynamics. Inspired by this, we introduce LingBot-VA, an autoregressive diffusion framework that learns frame prediction and policy execution simultaneously. Our model features three carefully crafted designs: (1) a shared latent space, integrating vision and action tokens, driven by a Mixture-of-Transformers (MoT) architecture, (2) a closed-loop rollout mechanism, allowing for ongoing acquisition of environmental feedback with ground-truth observations, (3) an asynchronous inference pipeline, parallelizing action prediction and motor execution to support efficient control. We evaluate our model on both simulation benchmarks and real-world scenarios, where it shows significant promise in long-horizon manipulation, data efficiency in post-training, and strong generalizability to novel configurations. The code and model are made publicly available to facilitate the community.
Video world modeling enables robot learning through a unified framework that predicts frames and executes policies simultaneously using a shared latent space and closed-loop feedback mechanisms.
video world modeling autoregressive diffusion framework frame prediction policy execution shared latent space Mixture-of-Transformers closed-loop rollout mechanism asynchronous inference pipeline long-horizon manipulation data efficiency generalizability