RLinf-USER: A Unified and Extensible System for Real-World Online Policy Learning in Embodied AI
Online policy learning directly in the physical world is a promising yet challenging direction for embodied intelligence. Unlike simulation, real-world systems cannot be arbitrarily accelerated, cheaply reset, or massively replicated, which makes scalable data collection, heterogeneous deployment, and long-horizon effective training difficult. These challenges suggest that real-world policy learning is not only an algorithmic issue but fundamentally a systems problem. We present USER, a Unified and extensible SystEm for Real-world online policy learning. USER treats physical robots as first-class hardware resources alongside GPUs through a unified hardware abstraction layer, enabling automatic discovery, management, and scheduling of heterogeneous robots. To address cloud-edge communication, USER introduces an adaptive communication plane with tunneling-based networking, distributed data channels for traffic localization, and streaming-multiprocessor-aware weight synchronization to regulate GPU-side overhead. On top of this infrastructure, USER organizes learning as a fully asynchronous framework with a persistent, cache-aware buffer, enabling efficient long-horizon experiments with robust crash recovery and reuse of historical data. In addition, USER provides extensible abstractions for rewards, algorithms, and policies, supporting online imitation or reinforcement learning of CNN/MLP, generative policies, and large vision-language-action (VLA) models within a unified pipeline. Results in both simulation and the real world show that USER enables multi-robot coordination, heterogeneous manipulators, edge-cloud collaboration with large models, and long-running asynchronous training, offering a unified and extensible systems foundation for real-world online policy learning.
698ab6bc1b2dc6b37d61b010
Hongzhi Zang
698ab6bc1b2dc6b37d61b011
Shu'ang Yu
claimed_verified
2026-02-11T11:19:08.606Z
683fe1d7f28c4d5a999d36f1
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/c6RPM9spqBI1FBRU6xQVG.png
Shu'ang Yu
ysa2001
user
ysa2001
698ab6bc1b2dc6b37d61b012
Hao Lin
698ab6bc1b2dc6b37d61b013
Tianxing Zhou
698ab6bc1b2dc6b37d61b014
Zefang Huang
698ab6bc1b2dc6b37d61b015
Zhen Guo
698ab6bc1b2dc6b37d61b016
Xin Xu
698ab6bc1b2dc6b37d61b017
Jiakai Zhou
698ab6bc1b2dc6b37d61b018
Yuze Sheng
698ab6bc1b2dc6b37d61b019
Shizhe Zhang
698ab6bc1b2dc6b37d61b01a
Feng Gao
698ab6bc1b2dc6b37d61b01b
Wenhao Tang
698ab6bc1b2dc6b37d61b01c
Yufeng Yue
698ab6bc1b2dc6b37d61b01d
Quanlu Zhang
698ab6bc1b2dc6b37d61b01e
Xinlei Chen
698ab6bc1b2dc6b37d61b01f
Chao Yu
698ab6bc1b2dc6b37d61b020
Yu Wang
RLinf-USER: A Unified and Extensible System for Real-World Online Policy Learning in Embodied AI
Online policy learning directly in the physical world is a promising yet challenging direction for embodied intelligence. Unlike simulation, real-world systems cannot be arbitrarily accelerated, cheaply reset, or massively replicated, which makes scalable data collection, heterogeneous deployment, and long-horizon effective training difficult. These challenges suggest that real-world policy learning is not only an algorithmic issue but fundamentally a systems problem. We present USER, a Unified and extensible SystEm for Real-world online policy learning. USER treats physical robots as first-class hardware resources alongside GPUs through a unified hardware abstraction layer, enabling automatic discovery, management, and scheduling of heterogeneous robots. To address cloud-edge communication, USER introduces an adaptive communication plane with tunneling-based networking, distributed data channels for traffic localization, and streaming-multiprocessor-aware weight synchronization to regulate GPU-side overhead. On top of this infrastructure, USER organizes learning as a fully asynchronous framework with a persistent, cache-aware buffer, enabling efficient long-horizon experiments with robust crash recovery and reuse of historical data. In addition, USER provides extensible abstractions for rewards, algorithms, and policies, supporting online imitation or reinforcement learning of CNN/MLP, generative policies, and large vision-language-action (VLA) models within a unified pipeline. Results in both simulation and the real world show that USER enables multi-robot coordination, heterogeneous manipulators, edge-cloud collaboration with large models, and long-running asynchronous training, offering a unified and extensible systems foundation for real-world online policy learning.
USER is a unified systems framework that enables scalable, asynchronous online policy learning in physical robots by treating them as first-class hardware resources and supporting diverse learning paradigms including VLA models.
online policy learning embodied intelligence real-world systems heterogeneous robots hardware abstraction layer adaptive communication plane tunneling-based networking distributed data channels streaming-multiprocessor-aware weight synchronization asynchronous framework cache-aware buffer crash recovery reinforcement learning imitation learning vision-language-action models multi-robot coordination edge-cloud collaboration