PyVision-RL: Forging Open Agentic Vision Models via RL
Reinforcement learning for agentic multimodal models often suffers from interaction collapse, where models learn to reduce tool usage and multi-turn reasoning, limiting the benefits of agentic behavior. We introduce PyVision-RL, a reinforcement learning framework for open-weight multimodal models that stabilizes training and sustains interaction. Our approach combines an oversampling-filtering-ranking rollout strategy with an accumulative tool reward to prevent collapse and encourage multi-turn tool use. Using a unified training pipeline, we develop PyVision-Image and PyVision-Video for image and video understanding. For video reasoning, PyVision-Video employs on-demand context construction, selectively sampling task-relevant frames during reasoning to significantly reduce visual token usage. Experiments show strong performance and improved efficiency, demonstrating that sustained interaction and on-demand visual processing are critical for scalable multimodal agents.
699e674ddfbcf0b800aecae9
Shitian Zhao
claimed_verified
2026-02-25T17:29:58.018Z
62c66504031996c36c86976a
https://cdn-avatars.huggingface.co/v1/production/uploads/62c66504031996c36c86976a/wIq0YJhkWnEhlzsh-TGYO.png
steve z
stzhao
user
stzhao
699e674ddfbcf0b800aecaea
Shaoheng Lin
699e674ddfbcf0b800aecaeb
Ming Li
699e674ddfbcf0b800aecaec
Haoquan Zhang
claimed_verified
2026-02-25T17:29:51.660Z
67ff7f687351095d4b606b84
https://cdn-avatars.huggingface.co/v1/production/uploads/67ff7f687351095d4b606b84/KhNPmbBC3zghuP5h1MK-c.png
Haoquan Zhang
haoquan03
user
haoquan03
699e674ddfbcf0b800aecaed
Wenshuo Peng
699e674ddfbcf0b800aecaee
Kaipeng Zhang
claimed_verified
2026-03-25T10:51:55.340Z
65f1713552c38a91e0a445e8
/avatars/47ab3ada51c9b9976ac1cd0c4301c373.svg
kaipeng
kpzhang996
user
kpzhang996
699e674ddfbcf0b800aecaef
Chen Wei
PyVision-RL: Forging Open Agentic Vision Models via RL
Reinforcement learning for agentic multimodal models often suffers from interaction collapse, where models learn to reduce tool usage and multi-turn reasoning, limiting the benefits of agentic behavior. We introduce PyVision-RL, a reinforcement learning framework for open-weight multimodal models that stabilizes training and sustains interaction. Our approach combines an oversampling-filtering-ranking rollout strategy with an accumulative tool reward to prevent collapse and encourage multi-turn tool use. Using a unified training pipeline, we develop PyVision-Image and PyVision-Video for image and video understanding. For video reasoning, PyVision-Video employs on-demand context construction, selectively sampling task-relevant frames during reasoning to significantly reduce visual token usage. Experiments show strong performance and improved efficiency, demonstrating that sustained interaction and on-demand visual processing are critical for scalable multimodal agents.
PyVision-RL framework addresses interaction collapse in multimodal models through enhanced reinforcement learning techniques and efficient video processing strategies.
reinforcement learning agentic multimodal models interaction collapse oversampling-filtering-ranking accumulative tool reward unified training pipeline on-demand context construction task-relevant frames visual token usage