ViVa: A Video-Generative Value Model for Robot Reinforcement Learning
Vision-language-action (VLA) models have advanced robot manipulation through large-scale pretraining, but real-world deployment remains challenging due to partial observability and delayed feedback. Reinforcement learning addresses this via value functions, which assess task progress and guide policy improvement. However, existing value models built on vision-language models (VLMs) struggle to capture temporal dynamics, undermining reliable value estimation in long-horizon tasks. In this paper, we propose ViVa, a video-generative value model that repurposes a pretrained video generator for value estimation. Taking the current observation and robot proprioception as input, ViVa jointly predicts future proprioception and a scalar value for the current state. By leveraging the spatiotemporal priors of a pretrained video generator, our approach grounds value estimation in anticipated embodiment dynamics, moving beyond static snapshots to intrinsically couple value with foresight. Integrated into RECAP, ViVa delivers substantial improvements on real-world box assembly. Qualitative analysis across all three tasks confirms that ViVa produces more reliable value signals, accurately reflecting task progress. By leveraging spatiotemporal priors from video corpora, ViVa also generalizes to novel objects, highlighting the promise of video-generative models for value estimation.
69d8915f79548974da946693
Jindi Lv
69d8915f79548974da946694
Hao Li
69d8915f79548974da946695
Jie Li
69d8915f79548974da946696
Yifei Nie
69d8915f79548974da946697
Fankun Kong
69d8915f79548974da946698
Yang Wang
69d8915f79548974da946699
Xiaofeng Wang
claimed_verified
2026-04-10T07:42:45.667Z
6426616ea5ec4a5cbc535634
https://cdn-avatars.huggingface.co/v1/production/uploads/6426616ea5ec4a5cbc535634/5IfSFYd9QOxz8K9QmBCst.png
JeffWang
Jeff-Wang
user
Jeff-Wang
69d8915f79548974da94669a
Zheng Zhu
69d8915f79548974da94669b
Chaojun Ni
69d8915f79548974da94669c
Qiuping Deng
69d8915f79548974da94669d
Hengtao Li
69d8915f79548974da94669e
Jiancheng Lv
69d8915f79548974da94669f
Guan Huang
ViVa: A Video-Generative Value Model for Robot Reinforcement Learning
Vision-language-action (VLA) models have advanced robot manipulation through large-scale pretraining, but real-world deployment remains challenging due to partial observability and delayed feedback. Reinforcement learning addresses this via value functions, which assess task progress and guide policy improvement. However, existing value models built on vision-language models (VLMs) struggle to capture temporal dynamics, undermining reliable value estimation in long-horizon tasks. In this paper, we propose ViVa, a video-generative value model that repurposes a pretrained video generator for value estimation. Taking the current observation and robot proprioception as input, ViVa jointly predicts future proprioception and a scalar value for the current state. By leveraging the spatiotemporal priors of a pretrained video generator, our approach grounds value estimation in anticipated embodiment dynamics, moving beyond static snapshots to intrinsically couple value with foresight. Integrated into RECAP, ViVa delivers substantial improvements on real-world box assembly. Qualitative analysis across all three tasks confirms that ViVa produces more reliable value signals, accurately reflecting task progress. By leveraging spatiotemporal priors from video corpora, ViVa also generalizes to novel objects, highlighting the promise of video-generative models for value estimation.
ViVa, a video-generative value model, improves robot manipulation by leveraging pretrained video generators to estimate values based on anticipated embodiment dynamics rather than static observations.
Vision-language-action models reinforcement learning value functions vision-language models video-generative models proprioception spatiotemporal priors RECAP embodiment dynamics