TwinBrainVLA: Unleashing the Potential of Generalist VLMs for Embodied Tasks via Asymmetric Mixture-of-Transformers
Standard Vision-Language-Action (VLA) models typically fine-tune a monolithic Vision-Language Model (VLM) backbone explicitly for robotic control. However, this approach creates a critical tension between maintaining high-level general semantic understanding and learning low-level, fine-grained sensorimotor skills, often leading to "catastrophic forgetting" of the model's open-world capabilities. To resolve this conflict, we introduce TwinBrainVLA, a novel architecture that coordinates a generalist VLM retaining universal semantic understanding and a specialist VLM dedicated to embodied proprioception for joint robotic control. TwinBrainVLA synergizes a frozen "Left Brain", which retains robust general visual reasoning, with a trainable "Right Brain", specialized for embodied perception, via a novel Asymmetric Mixture-of-Transformers (AsyMoT) mechanism. This design allows the Right Brain to dynamically query semantic knowledge from the frozen Left Brain and fuse it with proprioceptive states, providing rich conditioning for a Flow-Matching Action Expert to generate precise continuous controls. Extensive experiments on SimplerEnv and RoboCasa benchmarks demonstrate that TwinBrainVLA achieves superior manipulation performance compared to state-of-the-art baselines while explicitly preserving the comprehensive visual understanding capabilities of the pre-trained VLM, offering a promising direction for building general-purpose robots that simultaneously achieve high-level semantic understanding and low-level physical dexterity.
69706e6ea8be625b19c2afac
Bin Yu
claimed_verified
2026-01-21T09:19:14.460Z
63d3b5f1640bb0f77173baea
https://cdn-avatars.huggingface.co/v1/production/uploads/1674819020331-noauth.jpeg
yubin
VLyb
user
VLyb
69706e6ea8be625b19c2afad
Shijie Lian
claimed_verified
2026-01-22T08:47:15.246Z
65ec01fd770aa0e25d9374dc
https://cdn-avatars.huggingface.co/v1/production/uploads/65ec01fd770aa0e25d9374dc/yvLWwBEdAdHb-8EdUHg3n.jpeg
Shijie Lian
LiamLian0727
user
LiamLian0727
69706e6ea8be625b19c2afae
Xiaopeng Lin
69706e6ea8be625b19c2afaf
Yuliang Wei
69706e6ea8be625b19c2afb0
Zhaolong Shen
69706e6ea8be625b19c2afb1
Changti Wu
claimed_verified
2026-02-13T09:37:54.638Z
67b55e66d454cc4d10d21cfd
/avatars/3b18014fa7e603a5940175896f89372a.svg
Changti Wu
MaplesWCT
user
MaplesWCT
69706e6ea8be625b19c2afb2
Yuzhuo Miao
69706e6ea8be625b19c2afb3
Xinming Wang
69706e6ea8be625b19c2afb4
Bailing Wang
69706e6ea8be625b19c2afb5
Cong Huang
69706e6ea8be625b19c2afb6
Kai Chen
TwinBrainVLA: Unleashing the Potential of Generalist VLMs for Embodied Tasks via Asymmetric Mixture-of-Transformers
Standard Vision-Language-Action (VLA) models typically fine-tune a monolithic Vision-Language Model (VLM) backbone explicitly for robotic control. However, this approach creates a critical tension between maintaining high-level general semantic understanding and learning low-level, fine-grained sensorimotor skills, often leading to "catastrophic forgetting" of the model's open-world capabilities. To resolve this conflict, we introduce TwinBrainVLA, a novel architecture that coordinates a generalist VLM retaining universal semantic understanding and a specialist VLM dedicated to embodied proprioception for joint robotic control. TwinBrainVLA synergizes a frozen "Left Brain", which retains robust general visual reasoning, with a trainable "Right Brain", specialized for embodied perception, via a novel Asymmetric Mixture-of-Transformers (AsyMoT) mechanism. This design allows the Right Brain to dynamically query semantic knowledge from the frozen Left Brain and fuse it with proprioceptive states, providing rich conditioning for a Flow-Matching Action Expert to generate precise continuous controls. Extensive experiments on SimplerEnv and RoboCasa benchmarks demonstrate that TwinBrainVLA achieves superior manipulation performance compared to state-of-the-art baselines while explicitly preserving the comprehensive visual understanding capabilities of the pre-trained VLM, offering a promising direction for building general-purpose robots that simultaneously achieve high-level semantic understanding and low-level physical dexterity.
TwinBrainVLA addresses the tension between semantic understanding and motor skills in robot control by coordinating a generalist vision-language model with a specialist model through an asymmetric mixture-of-transformers mechanism.
Vision-Language-Action models Vision-Language Models robotic control catastrophic forgetting frozen Left Brain trainable Right Brain Asymmetric Mixture-of-Transformers Flow-Matching Action Expert embodied perception proprioception