MedVLSynther: Synthesizing High-Quality Visual Question Answering from
  Medical Documents with Generator-Verifier LMMs
Large Multimodal Models (LMMs) are increasingly capable of answering medical
questions that require joint reasoning over images and text, yet training
general medical VQA systems is impeded by the lack of large, openly usable,
high-quality corpora. We present MedVLSynther, a rubric-guided
generator-verifier framework that synthesizes high-quality multiple-choice VQA
items directly from open biomedical literature by conditioning on figures,
captions, and in-text references. The generator produces self-contained stems
and parallel, mutually exclusive options under a machine-checkable JSON schema;
a multi-stage verifier enforces essential gates (self-containment, single
correct answer, clinical validity, image-text consistency), awards fine-grained
positive points, and penalizes common failure modes before acceptance. Applying
this pipeline to PubMed Central yields MedSynVQA: 13,087 audited questions over
14,803 images spanning 13 imaging modalities and 28 anatomical regions.
Training open-weight LMMs with reinforcement learning using verifiable rewards
improves accuracy across six medical VQA benchmarks, achieving averages of
55.85 (3B) and 58.15 (7B), with up to 77.57 on VQA-RAD and 67.76 on PathVQA,
outperforming strong medical LMMs. A Ablations verify that both generation and
verification are necessary and that more verified data consistently helps, and
a targeted contamination analysis detects no leakage from evaluation suites. By
operating entirely on open literature and open-weight models, MedVLSynther
offers an auditable, reproducible, and privacy-preserving path to scalable
medical VQA training data.
6904a82d36711a7e91ed4d2f
Xiaoke Huang
claimed_verified
2025-11-24T07:59:34.051Z
63318b2349a9563915469f3b
https://cdn-avatars.huggingface.co/v1/production/uploads/63318b2349a9563915469f3b/zlbeB2997i8YkoyOTb9FL.jpeg
Xiaoke Huang
xk-huang
user
xk-huang
6904a82d36711a7e91ed4d30
Ningsen Wang
6904a82d36711a7e91ed4d31
Hui Liu
6904a82d36711a7e91ed4d32
Xianfeng Tang
6904a82d36711a7e91ed4d33
Yuyin Zhou
MedVLSynther: Synthesizing High-Quality Visual Question Answering from
  Medical Documents with Generator-Verifier LMMs
Large Multimodal Models (LMMs) are increasingly capable of answering medical
questions that require joint reasoning over images and text, yet training
general medical VQA systems is impeded by the lack of large, openly usable,
high-quality corpora. We present MedVLSynther, a rubric-guided
generator-verifier framework that synthesizes high-quality multiple-choice VQA
items directly from open biomedical literature by conditioning on figures,
captions, and in-text references. The generator produces self-contained stems
and parallel, mutually exclusive options under a machine-checkable JSON schema;
a multi-stage verifier enforces essential gates (self-containment, single
correct answer, clinical validity, image-text consistency), awards fine-grained
positive points, and penalizes common failure modes before acceptance. Applying
this pipeline to PubMed Central yields MedSynVQA: 13,087 audited questions over
14,803 images spanning 13 imaging modalities and 28 anatomical regions.
Training open-weight LMMs with reinforcement learning using verifiable rewards
improves accuracy across six medical VQA benchmarks, achieving averages of
55.85 (3B) and 58.15 (7B), with up to 77.57 on VQA-RAD and 67.76 on PathVQA,
outperforming strong medical LMMs. A Ablations verify that both generation and
verification are necessary and that more verified data consistently helps, and
a targeted contamination analysis detects no leakage from evaluation suites. By
operating entirely on open literature and open-weight models, MedVLSynther
offers an auditable, reproducible, and privacy-preserving path to scalable
medical VQA training data.
MedVLSynther synthesizes high-quality medical VQA data from open literature, improving LMM performance across benchmarks through a generator-verifier framework.
multimodal models VQA MedVLSynther generator-verifier framework biomedical literature JSON schema reinforcement learning MedSynVQA VQA-RAD PathVQA privacy-preserving