Towards Synergistic, Generalized, and Efficient Dual-System for Robotic
  Manipulation
The increasing demand for versatile robotic systems to operate in diverse and
dynamic environments has emphasized the importance of a generalist policy,
which leverages a large cross-embodiment data corpus to facilitate broad
adaptability and high-level reasoning. However, the generalist would struggle
with inefficient inference and cost-expensive training. The specialist policy,
instead, is curated for specific domain data and excels at task-level precision
with efficiency. Yet, it lacks the generalization capacity for a wide range of
applications. Inspired by these observations, we introduce RoboDual, a
synergistic dual-system that supplements the merits of both generalist and
specialist policy. A diffusion transformer-based specialist is devised for
multi-step action rollouts, exquisitely conditioned on the high-level task
understanding and discretized action output of a vision-language-action (VLA)
based generalist. Compared to OpenVLA, RoboDual achieves 26.7% improvement in
real-world setting and 12% gain on CALVIN by introducing a specialist policy
with merely 20M trainable parameters. It maintains strong performance with 5%
of demonstration data only, and enables a 3.8 times higher control frequency in
real-world deployment. Code would be made publicly available. Our project page
is hosted at: https://opendrivelab.com/RoboDual/
670f53a8d75f11435280d907
Qingwen Bu
claimed_verified
2024-10-16T09:43:11.706Z
64ac1f169dcc5787461468a4
/avatars/c031a75989147009b7850df4eddfcb27.svg
Qingwen Bu
qwbu
user
qwbu
670f53a8d75f11435280d908
Hongyang Li
670f53a8d75f11435280d909
Li Chen
670f53a8d75f11435280d90a
Jisong Cai
670f53a8d75f11435280d90b
Jia Zeng
claimed_verified
2026-01-09T15:48:02.960Z
685d08b9fc7a0ff2f338dbd0
https://cdn-avatars.huggingface.co/v1/production/uploads/685d08b9fc7a0ff2f338dbd0/bW3G1ZM-tAXIIbCbUdwC9.png
Jia Zeng
Jia-Zeng
user
Jia-Zeng
670f53a8d75f11435280d90c
Heming Cui
670f53a8d75f11435280d90d
Maoqing Yao
670f53a8d75f11435280d90e
Yu Qiao
Towards Synergistic, Generalized, and Efficient Dual-System for Robotic
  Manipulation
The increasing demand for versatile robotic systems to operate in diverse and
dynamic environments has emphasized the importance of a generalist policy,
which leverages a large cross-embodiment data corpus to facilitate broad
adaptability and high-level reasoning. However, the generalist would struggle
with inefficient inference and cost-expensive training. The specialist policy,
instead, is curated for specific domain data and excels at task-level precision
with efficiency. Yet, it lacks the generalization capacity for a wide range of
applications. Inspired by these observations, we introduce RoboDual, a
synergistic dual-system that supplements the merits of both generalist and
specialist policy. A diffusion transformer-based specialist is devised for
multi-step action rollouts, exquisitely conditioned on the high-level task
understanding and discretized action output of a vision-language-action (VLA)
based generalist. Compared to OpenVLA, RoboDual achieves 26.7% improvement in
real-world setting and 12% gain on CALVIN by introducing a specialist policy
with merely 20M trainable parameters. It maintains strong performance with 5%
of demonstration data only, and enables a 3.8 times higher control frequency in
real-world deployment. Code would be made publicly available. Our project page
is hosted at: https://opendrivelab.com/RoboDual/
RoboDual integrates a diffusion transformer specialist with a vision-language-action generalist to enhance robotic adaptability and precision while maintaining efficiency.
diffusion transformer multi-step action rollouts vision-language-action high-level task understanding discretized action output OpenVLA CALVIN real-world setting control frequency