SOP: A Scalable Online Post-Training System for Vision-Language-Action Models
Vision-language-action (VLA) models achieve strong generalization through large-scale pre-training, but real-world deployment requires expert-level task proficiency in addition to broad generality. Existing post-training approaches for VLA models are typically offline, single-robot, or task-specific, limiting effective on-policy adaptation and scalable learning from real-world interaction. We introduce a Scalable Online Post-training (SOP) system that enables online, distributed, multi-task post-training of generalist VLA models directly in the physical world. SOP tightly couples execution and learning through a closed-loop architecture in which a fleet of robots continuously streams on-policy experience and human intervention signals to a centralized cloud learner, and asynchronously receives updated policies. This design supports prompt on-policy correction, scales experience collection through parallel deployment, and preserves generality during adaptation. SOP is agnostic to the choice of post-training algorithm; we instantiate it with both interactive imitation learning (HG-DAgger) and reinforcement learning (RECAP). Across a range of real-world manipulation tasks including cloth folding, box assembly, and grocery restocking, we show that SOP substantially improves the performance of large pretrained VLA models while maintaining a single shared policy across tasks. Effective post-training can be achieved within hours of real-world interaction, and performance scales near-linearly with the number of robots in the fleet. These results suggest that tightly coupling online learning with fleet-scale deployment is instrumental to enabling efficient, reliable, and scalable post-training of generalist robot policies in the physical world.
695dc422c03d6d81e4399e60
Mingjie Pan
admin_assigned
2026-01-07T13:27:27.004Z
655ecd7c56e5ceaf05344b24
/avatars/faad88525197bb6c63be0068f19de418.svg
MingjieP
pmj110119
user
pmj110119
695dc422c03d6d81e4399e61
Siyuan Feng
claimed_verified
2026-01-07T09:26:03.276Z
620326e962b2b0e46e79971b
https://cdn-avatars.huggingface.co/v1/production/uploads/620326e962b2b0e46e79971b/1FVPRpsWng5q3An4qbuYQ.jpeg
Siyuan Feng
Eralien
user
Eralien
695dc422c03d6d81e4399e62
Qinglin Zhang
695dc422c03d6d81e4399e63
Xinchen Li
claimed_verified
2026-01-09T08:36:38.651Z
695dfcfdec19ab86cd2f926e
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/iTTmqxt1UhWRdXbP97uML.png
Xinchen Li
Felixinc
user
Felixinc
695dc422c03d6d81e4399e64
Jianheng Song
admin_assigned
2026-01-07T13:28:32.580Z
64993b27f8069251837b81ed
/avatars/9aa3956dc527dccfdc6b6014dedf761f.svg
Song Jianheng
JJH1998
user
JJH1998
695dc422c03d6d81e4399e65
Chendi Qu
695dc422c03d6d81e4399e66
Yi Wang
695dc422c03d6d81e4399e67
Chuankang Li
695dc422c03d6d81e4399e68
Ziyu Xiong
admin_assigned
2026-01-07T13:28:14.043Z
64d107229617774ce41b9467
/avatars/10d4dff031f2d8dd7047d7ea5ec0d4c1.svg
Ziyu Xiong
JennyZiyu
user
JennyZiyu
695dc422c03d6d81e4399e69
Zhi Chen
695dc422c03d6d81e4399e6a
Yi Liu
695dc422c03d6d81e4399e6b
Jianlan Luo
claimed_verified
2026-01-07T13:13:52.912Z
64f8cb8ed04a890f5380d9a4
/avatars/d6fdfdbb0c10141aa3b4c832d928121b.svg
Jianlan Luo
jianlanluo
user
jianlanluo
SOP: A Scalable Online Post-Training System for Vision-Language-Action Models
Vision-language-action (VLA) models achieve strong generalization through large-scale pre-training, but real-world deployment requires expert-level task proficiency in addition to broad generality. Existing post-training approaches for VLA models are typically offline, single-robot, or task-specific, limiting effective on-policy adaptation and scalable learning from real-world interaction. We introduce a Scalable Online Post-training (SOP) system that enables online, distributed, multi-task post-training of generalist VLA models directly in the physical world. SOP tightly couples execution and learning through a closed-loop architecture in which a fleet of robots continuously streams on-policy experience and human intervention signals to a centralized cloud learner, and asynchronously receives updated policies. This design supports prompt on-policy correction, scales experience collection through parallel deployment, and preserves generality during adaptation. SOP is agnostic to the choice of post-training algorithm; we instantiate it with both interactive imitation learning (HG-DAgger) and reinforcement learning (RECAP). Across a range of real-world manipulation tasks including cloth folding, box assembly, and grocery restocking, we show that SOP substantially improves the performance of large pretrained VLA models while maintaining a single shared policy across tasks. Effective post-training can be achieved within hours of real-world interaction, and performance scales near-linearly with the number of robots in the fleet. These results suggest that tightly coupling online learning with fleet-scale deployment is instrumental to enabling efficient, reliable, and scalable post-training of generalist robot policies in the physical world.
A scalable online post-training system enables real-world robot policy adaptation through distributed, multi-task learning that maintains generality while improving task proficiency.
Vision-language-action models post-training online learning distributed learning multi-task learning closed-loop architecture on-policy experience interactive imitation learning reinforcement learning fleet-scale deployment