Unified Diffusion VLA: Vision-Language-Action Model via Joint Discrete
  Denoising Diffusion Process
Vision-language-action (VLA) models aim to understand natural language
instructions and visual observations and to execute corresponding actions as an
embodied agent. Recent work integrates future images into the
understanding-acting loop, yielding unified VLAs that jointly understand,
generate, and act -- reading text and images and producing future images and
actions. However, these models either rely on external experts for modality
unification or treat image generation and action prediction as separate
processes, limiting the benefits of direct synergy between these tasks. Our
core philosophy is to optimize generation and action jointly through a
synchronous denoising process, where the iterative refinement enables actions
to evolve from initialization, under constant and sufficient visual guidance.
We ground this philosophy in our proposed Unified Diffusion VLA and Joint
Discrete Denoising Diffusion Process (JD3P), which is a joint diffusion process
that integrates multiple modalities into a single denoising trajectory to serve
as the key mechanism enabling understanding, generation, and acting to be
intrinsically synergistic. Our model and theory are built on a unified
tokenized space of all modalities and a hybrid attention mechanism. We further
propose a two-stage training pipeline and several inference-time techniques
that optimize performance and efficiency. Our approach achieves
state-of-the-art performance on benchmarks such as CALVIN, LIBERO, and
SimplerEnv with 4times faster inference than autoregressive methods, and we
demonstrate its effectiveness through in-depth analysis and real-world
evaluations. Our project page is available at
https://irpn-eai.github.io/UD-VLA.github.io/.
69096c89812eca10f9cc619c
Jiayi Chen
claimed_verified
2025-11-04T20:26:42.597Z
661e3bc34273a439e171d806
https://cdn-avatars.huggingface.co/v1/production/uploads/661e3bc34273a439e171d806/-OFdIK39yQD702NFsdALE.jpeg
jiayi chen
chenpyyy
user
chenpyyy
69096c89812eca10f9cc619d
Wenxuan Song
69096c89812eca10f9cc619e
Pengxiang Ding
69096c89812eca10f9cc619f
Ziyang Zhou
69096c89812eca10f9cc61a0
Han Zhao
69096c89812eca10f9cc61a1
Feilong Tang
69096c89812eca10f9cc61a2
Donglin Wang
69096c89812eca10f9cc61a3
Haoang Li
Unified Diffusion VLA: Vision-Language-Action Model via Joint Discrete
  Denoising Diffusion Process
Vision-language-action (VLA) models aim to understand natural language
instructions and visual observations and to execute corresponding actions as an
embodied agent. Recent work integrates future images into the
understanding-acting loop, yielding unified VLAs that jointly understand,
generate, and act -- reading text and images and producing future images and
actions. However, these models either rely on external experts for modality
unification or treat image generation and action prediction as separate
processes, limiting the benefits of direct synergy between these tasks. Our
core philosophy is to optimize generation and action jointly through a
synchronous denoising process, where the iterative refinement enables actions
to evolve from initialization, under constant and sufficient visual guidance.
We ground this philosophy in our proposed Unified Diffusion VLA and Joint
Discrete Denoising Diffusion Process (JD3P), which is a joint diffusion process
that integrates multiple modalities into a single denoising trajectory to serve
as the key mechanism enabling understanding, generation, and acting to be
intrinsically synergistic. Our model and theory are built on a unified
tokenized space of all modalities and a hybrid attention mechanism. We further
propose a two-stage training pipeline and several inference-time techniques
that optimize performance and efficiency. Our approach achieves
state-of-the-art performance on benchmarks such as CALVIN, LIBERO, and
SimplerEnv with 4times faster inference than autoregressive methods, and we
demonstrate its effectiveness through in-depth analysis and real-world
evaluations. Our project page is available at
https://irpn-eai.github.io/UD-VLA.github.io/.
A Unified Diffusion VLA and Joint Discrete Denoising Diffusion Process (JD3P) model integrates multiple modalities through a synchronous denoising process, achieving state-of-the-art performance in vision-language-action tasks with faster inference.
Unified Diffusion VLA Joint Discrete Denoising Diffusion Process JD3P synchronous denoising process unified tokenized space hybrid attention mechanism two-stage training pipeline inference-time techniques CALVIN LIBERO SimplerEnv