Beyond Multiple Choice: Verifiable OpenQA for Robust Vision-Language RFT
Multiple-choice question answering (MCQA) has been a popular format for evaluating and reinforcement fine-tuning (RFT) of modern multimodal language models. Its constrained output format allows for simplified, deterministic automatic verification. However, we find that the options may leak exploitable signals, which makes the accuracy metrics unreliable for indicating real capabilities and encourages explicit or implicit answer guessing behaviors during RFT. We propose ReVeL (Rewrite and Verify by LLM), a framework that rewrites multiple-choice questions into open-form questions while keeping answers verifiable whenever possible. The framework categorizes questions according to different answer types, apply different rewriting and verification schemes, respectively. When applied for RFT, we converted 20k MCQA examples and use GRPO to finetune Qwen2.5-VL models. Models trained on ReVeL-OpenQA match MCQA accuracy on multiple-choice benchmarks and improve OpenQA accuracy by about six percentage points, indicating better data efficiency and more robust reward signals than MCQA-based training. When used for evaluation, ReVeL also reveals up to 20 percentage points of score inflation in MCQA benchmarks (relative to OpenQA), improves judging accuracy, and reduces both cost and latency. We will release code and data publicly.
692526e316eb3a9f1310398a
Yesheng Liu
claimed_verified
2025-11-25T09:05:35.186Z
657db8c97cf2a47c8cd790f3
/avatars/35d5dfae09b79f37d8c4d08ebc0bcd10.svg
yes liu
zhizhou57
user
zhizhou57
692526e316eb3a9f1310398b
Hao Li
692526e316eb3a9f1310398c
Haiyu Xu
claimed_verified
2025-11-25T09:05:37.241Z
68b97ed277d4f089dfded859
/avatars/ff20a41afea3bd5fd310c3e050862843.svg
Xu Haiyu
lindaxu525
user
lindaxu525
692526e316eb3a9f1310398d
Baoqi Pei
692526e316eb3a9f1310398e
Jiahao Wang
692526e316eb3a9f1310398f
Mingxuan Zhao
692526e316eb3a9f13103990
Jingshu Zheng
692526e316eb3a9f13103991
Zheqi He
692526e316eb3a9f13103992
JG Yao
692526e316eb3a9f13103993
Bowen Qin
692526e316eb3a9f13103994
Xi Yang
692526e316eb3a9f13103995
Jiajun Zhang
Beyond Multiple Choice: Verifiable OpenQA for Robust Vision-Language RFT
Multiple-choice question answering (MCQA) has been a popular format for evaluating and reinforcement fine-tuning (RFT) of modern multimodal language models. Its constrained output format allows for simplified, deterministic automatic verification. However, we find that the options may leak exploitable signals, which makes the accuracy metrics unreliable for indicating real capabilities and encourages explicit or implicit answer guessing behaviors during RFT. We propose ReVeL (Rewrite and Verify by LLM), a framework that rewrites multiple-choice questions into open-form questions while keeping answers verifiable whenever possible. The framework categorizes questions according to different answer types, apply different rewriting and verification schemes, respectively. When applied for RFT, we converted 20k MCQA examples and use GRPO to finetune Qwen2.5-VL models. Models trained on ReVeL-OpenQA match MCQA accuracy on multiple-choice benchmarks and improve OpenQA accuracy by about six percentage points, indicating better data efficiency and more robust reward signals than MCQA-based training. When used for evaluation, ReVeL also reveals up to 20 percentage points of score inflation in MCQA benchmarks (relative to OpenQA), improves judging accuracy, and reduces both cost and latency. We will release code and data publicly.
ReVeL, a framework that converts multiple-choice questions to open-form questions, improves data efficiency and robustness in fine-tuning multimodal language models and reveals score inflation in MCQA benchmarks.
reinforcement fine-tuning multimodal language models multiple-choice question answering ReVeL automatic verification open-form questions answer verification GRPO Qwen2.5-VL OpenQA score inflation judging accuracy