When Does RL Help Medical VLMs? Disentangling Vision, SFT, and RL Gains
Reinforcement learning (RL) is increasingly used to post-train medical Vision-Language Models (VLMs), yet it remains unclear whether RL improves medical visual reasoning or mainly sharpens behaviors already induced by supervised fine-tuning (SFT). We present a controlled study that disentangles these effects along three axes: vision, SFT, and RL. Using MedMNIST as a multi-modality testbed, we probe visual perception by benchmarking VLM vision towers against vision-only baselines, quantify reasoning support and sampling efficiency via Accuracy@1 versus Pass@K, and evaluate when RL closes the support gap and how gains transfer across modalities. We find that RL is most effective when the model already has non-trivial support (high Pass@K): it primarily sharpens the output distribution, improving Acc@1 and sampling efficiency, while SFT expands support and makes RL effective. Based on these findings, we propose a boundary-aware recipe and instantiate it by RL post-training an OctoMed-initialized model on a small, balanced subset of PMC multiple-choice VQA, achieving strong average performance across six medical VQA benchmarks.
69a65974663a122a7ce66703
Ahmadreza Jeddi
claimed_verified
2026-03-03T10:38:59.156Z
65e8df170cda621164769f6f
/avatars/b5d7c7c49d1a777a1bd41d06d299868a.svg
Armen Jeddi
armenjeddi
user
armenjeddi
69a65974663a122a7ce66704
Kimia Shaban
claimed_verified
2026-03-03T10:38:56.898Z
6631b79e351231c428035e78
https://cdn-avatars.huggingface.co/v1/production/uploads/6631b79e351231c428035e78/-QtkAybw7keXqkXivEvXU.jpeg
Kimia Shaban
kimiashaban
user
kimiashaban
69a65974663a122a7ce66705
Negin Baghbanzadeh
69a65974663a122a7ce66706
Natasha Sharan
69a65974663a122a7ce66707
Abhishek Moturu
69a65974663a122a7ce66708
Elham Dolatabadi
69a65974663a122a7ce66709
Babak Taati
When Does RL Help Medical VLMs? Disentangling Vision, SFT, and RL Gains
Reinforcement learning (RL) is increasingly used to post-train medical Vision-Language Models (VLMs), yet it remains unclear whether RL improves medical visual reasoning or mainly sharpens behaviors already induced by supervised fine-tuning (SFT). We present a controlled study that disentangles these effects along three axes: vision, SFT, and RL. Using MedMNIST as a multi-modality testbed, we probe visual perception by benchmarking VLM vision towers against vision-only baselines, quantify reasoning support and sampling efficiency via Accuracy@1 versus Pass@K, and evaluate when RL closes the support gap and how gains transfer across modalities. We find that RL is most effective when the model already has non-trivial support (high Pass@K): it primarily sharpens the output distribution, improving Acc@1 and sampling efficiency, while SFT expands support and makes RL effective. Based on these findings, we propose a boundary-aware recipe and instantiate it by RL post-training an OctoMed-initialized model on a small, balanced subset of PMC multiple-choice VQA, achieving strong average performance across six medical VQA benchmarks.
Reinforcement learning enhances medical vision-language model performance primarily by sharpening output distributions when models already have sufficient reasoning support, with supervised fine-tuning expanding initial capability and enabling effective reinforcement learning.
Vision-Language Models reinforcement learning supervised fine-tuning visual reasoning MedMNIST accuracy@1 pass@k octomedi multiple-choice VQA