Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution
In this report, we introduce Xiaomi-Robotics-0, an advanced vision-language-action (VLA) model optimized for high performance and fast and smooth real-time execution. The key to our method lies in a carefully designed training recipe and deployment strategy. Xiaomi-Robotics-0 is first pre-trained on large-scale cross-embodiment robot trajectories and vision-language data, endowing it with broad and generalizable action-generation capabilities while avoiding catastrophic forgetting of the visual-semantic knowledge of the underlying pre-trained VLM. During post-training, we propose several techniques for training the VLA model for asynchronous execution to address the inference latency during real-robot rollouts. During deployment, we carefully align the timesteps of consecutive predicted action chunks to ensure continuous and seamless real-time rollouts. We evaluate Xiaomi-Robotics-0 extensively in simulation benchmarks and on two challenging real-robot tasks that require precise and dexterous bimanual manipulation. Results show that our method achieves state-of-the-art performance across all simulation benchmarks. Moreover, Xiaomi-Robotics-0 can roll out fast and smoothly on real robots using a consumer-grade GPU, achieving high success rates and throughput on both real-robot tasks. To facilitate future research, code and model checkpoints are open-sourced at https://xiaomi-robotics-0.github.io
69928a9e50fb2c0be4783917
Rui Cai
69928a9e50fb2c0be4783918
Jun Guo
69928a9e50fb2c0be4783919
Xinze He
69928a9e50fb2c0be478391a
Piaopiao Jin
69928a9e50fb2c0be478391b
Jie Li
69928a9e50fb2c0be478391c
Bingxuan Lin
69928a9e50fb2c0be478391d
Futeng Liu
69928a9e50fb2c0be478391e
Wei Liu
69928a9e50fb2c0be478391f
Fei Ma
69928a9e50fb2c0be4783920
Kun Ma
69928a9e50fb2c0be4783921
Feng Qiu
69928a9e50fb2c0be4783922
Heng Qu
69928a9e50fb2c0be4783923
Yifei Su
69928a9e50fb2c0be4783924
Qiao Sun
69928a9e50fb2c0be4783925
Dong Wang
69928a9e50fb2c0be4783926
Donghao Wang
69928a9e50fb2c0be4783927
Yunhong Wang
69928a9e50fb2c0be4783928
Rujie Wu
69928a9e50fb2c0be4783929
Diyun Xiang
69928a9e50fb2c0be478392a
Yu Yang
69928a9e50fb2c0be478392b
Hangjun Ye
69928a9e50fb2c0be478392c
Yuan Zhang
69928a9e50fb2c0be478392d
Quanyun Zhou
Xiaomi-Robotics-0: An Open-Sourced Vision-Language-Action Model with Real-Time Execution
In this report, we introduce Xiaomi-Robotics-0, an advanced vision-language-action (VLA) model optimized for high performance and fast and smooth real-time execution. The key to our method lies in a carefully designed training recipe and deployment strategy. Xiaomi-Robotics-0 is first pre-trained on large-scale cross-embodiment robot trajectories and vision-language data, endowing it with broad and generalizable action-generation capabilities while avoiding catastrophic forgetting of the visual-semantic knowledge of the underlying pre-trained VLM. During post-training, we propose several techniques for training the VLA model for asynchronous execution to address the inference latency during real-robot rollouts. During deployment, we carefully align the timesteps of consecutive predicted action chunks to ensure continuous and seamless real-time rollouts. We evaluate Xiaomi-Robotics-0 extensively in simulation benchmarks and on two challenging real-robot tasks that require precise and dexterous bimanual manipulation. Results show that our method achieves state-of-the-art performance across all simulation benchmarks. Moreover, Xiaomi-Robotics-0 can roll out fast and smoothly on real robots using a consumer-grade GPU, achieving high success rates and throughput on both real-robot tasks. To facilitate future research, code and model checkpoints are open-sourced at https://xiaomi-robotics-0.github.io
A vision-language-action model for robotics combines large-scale pretraining with specialized training techniques to enable real-time execution and high-performance manipulation tasks.
vision-language-action cross-embodiment robot trajectories pre-trained VLM catastrophic forgetting asynchronous execution inference latency real-time rollouts bimanual manipulation simulation benchmarks real-robot tasks