Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action
  Model
Recently, augmenting Vision-Language-Action models (VLAs) with world modeling
has shown promise in improving robotic policy learning. However, it remains
challenging to jointly predict next-state observations and action sequences
because of the inherent difference between the two modalities. To address this,
we propose DUal-STream diffusion (DUST), a world-model augmented VLA framework
that handles the modality conflict and enhances the performance of VLAs across
diverse tasks. Specifically, we propose a multimodal diffusion transformer
architecture that explicitly maintains separate modality streams while still
enabling cross-modal knowledge sharing. In addition, we introduce independent
noise perturbations for each modality and a decoupled flow-matching loss. This
design enables the model to learn the joint distribution in a bidirectional
manner while avoiding the need for a unified latent space. Based on the
decoupling of modalities during training, we also introduce a joint sampling
method that supports test-time scaling, where action and vision tokens evolve
asynchronously at different rates. Through experiments on simulated benchmarks
such as RoboCasa and GR-1, DUST achieves up to 6% gains over baseline methods,
while our test-time scaling approach provides an additional 2-5% boost. On
real-world tasks with the Franka Research 3, DUST improves success rates by
13%, confirming its effectiveness beyond simulation. Furthermore, pre-training
on action-free videos from BridgeV2 yields significant transfer gains on
RoboCasa, underscoring DUST's potential for large-scale VLA pretraining.
690815d6812eca10f9cc5e0b
John Won
claimed_verified
2025-11-03T20:52:34.710Z
660a44cc8b022f13fd706d7a
/avatars/b01d9aed87b077e4c650944b4180cb12.svg
John Won
periphanes
user
periphanes
690815d6812eca10f9cc5e0c
Kyungmin Lee
690815d6812eca10f9cc5e0d
Huiwon Jang
690815d6812eca10f9cc5e0e
Dongyoung Kim
690815d6812eca10f9cc5e0f
Jinwoo Shin
Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action
  Model
Recently, augmenting Vision-Language-Action models (VLAs) with world modeling
has shown promise in improving robotic policy learning. However, it remains
challenging to jointly predict next-state observations and action sequences
because of the inherent difference between the two modalities. To address this,
we propose DUal-STream diffusion (DUST), a world-model augmented VLA framework
that handles the modality conflict and enhances the performance of VLAs across
diverse tasks. Specifically, we propose a multimodal diffusion transformer
architecture that explicitly maintains separate modality streams while still
enabling cross-modal knowledge sharing. In addition, we introduce independent
noise perturbations for each modality and a decoupled flow-matching loss. This
design enables the model to learn the joint distribution in a bidirectional
manner while avoiding the need for a unified latent space. Based on the
decoupling of modalities during training, we also introduce a joint sampling
method that supports test-time scaling, where action and vision tokens evolve
asynchronously at different rates. Through experiments on simulated benchmarks
such as RoboCasa and GR-1, DUST achieves up to 6% gains over baseline methods,
while our test-time scaling approach provides an additional 2-5% boost. On
real-world tasks with the Franka Research 3, DUST improves success rates by
13%, confirming its effectiveness beyond simulation. Furthermore, pre-training
on action-free videos from BridgeV2 yields significant transfer gains on
RoboCasa, underscoring DUST's potential for large-scale VLA pretraining.
A multimodal diffusion transformer framework, DUal-STream diffusion (DUST), enhances Vision-Language-Action models by maintaining separate modality streams and enabling cross-modal knowledge sharing, achieving improved performance in both simulated and real-world robotic tasks.
multimodal diffusion transformer modality streams cross-modal knowledge sharing independent noise perturbations decoupled flow-matching loss bidirectional learning joint sampling method test-time scaling action tokens vision tokens RoboCasa GR-1 Franka Research 3 BridgeV2 pre-training transfer gains