RISE: Self-Improving Robot Policy with Compositional World Model
Despite the sustained scaling on model capacity and data acquisition, Vision-Language-Action (VLA) models remain brittle in contact-rich and dynamic manipulation tasks, where minor execution deviations can compound into failures. While reinforcement learning (RL) offers a principled path to robustness, on-policy RL in the physical world is constrained by safety risk, hardware cost, and environment reset. To bridge this gap, we present RISE, a scalable framework of robotic reinforcement learning via imagination. At its core is a Compositional World Model that (i) predicts multi-view future via a controllable dynamics model, and (ii) evaluates imagined outcomes with a progress value model, producing informative advantages for the policy improvement. Such compositional design allows state and value to be tailored by best-suited yet distinct architectures and objectives. These components are integrated into a closed-loop self-improving pipeline that continuously generates imaginary rollouts, estimates advantages, and updates the policy in imaginary space without costly physical interaction. Across three challenging real-world tasks, RISE yields significant improvement over prior art, with more than +35% absolute performance increase in dynamic brick sorting, +45% for backpack packing, and +35% for box closing, respectively.
698ee780cace060ff123af3e
Jiazhi Yang
698ee780cace060ff123af3f
Kunyang Lin
698ee780cace060ff123af40
Jinwei Li
698ee780cace060ff123af41
Wencong Zhang
698ee780cace060ff123af42
Tianwei Lin
698ee780cace060ff123af43
Longyan Wu
698ee780cace060ff123af44
Zhizhong Su
698ee780cace060ff123af45
Hao Zhao
698ee780cace060ff123af46
Ya-Qin Zhang
698ee780cace060ff123af47
Li Chen
claimed_verified
2026-02-17T15:51:17.546Z
63a57339fdda34b86663d481
/avatars/fba5f98b617c7dd128777dcda425754a.svg
Li Chen
ilnehc
user
ilnehc
698ee780cace060ff123af48
Ping Luo
698ee780cace060ff123af49
Xiangyu Yue
698ee780cace060ff123af4a
Hongyang Li
RISE: Self-Improving Robot Policy with Compositional World Model
Despite the sustained scaling on model capacity and data acquisition, Vision-Language-Action (VLA) models remain brittle in contact-rich and dynamic manipulation tasks, where minor execution deviations can compound into failures. While reinforcement learning (RL) offers a principled path to robustness, on-policy RL in the physical world is constrained by safety risk, hardware cost, and environment reset. To bridge this gap, we present RISE, a scalable framework of robotic reinforcement learning via imagination. At its core is a Compositional World Model that (i) predicts multi-view future via a controllable dynamics model, and (ii) evaluates imagined outcomes with a progress value model, producing informative advantages for the policy improvement. Such compositional design allows state and value to be tailored by best-suited yet distinct architectures and objectives. These components are integrated into a closed-loop self-improving pipeline that continuously generates imaginary rollouts, estimates advantages, and updates the policy in imaginary space without costly physical interaction. Across three challenging real-world tasks, RISE yields significant improvement over prior art, with more than +35% absolute performance increase in dynamic brick sorting, +45% for backpack packing, and +35% for box closing, respectively.
RISE is a robotic reinforcement learning framework that uses a compositional world model to predict multi-view futures and evaluate imagined outcomes, enabling policy improvement through virtual interactions rather than physical trials.
Vision-Language-Action models reinforcement learning on-policy RL Compositional World Model controllable dynamics model progress value model imaginary rollouts policy improvement self-improving pipeline