S-Chain: Structured Visual Chain-of-Thought For Medicine
Faithful reasoning in medical vision-language models (VLMs) requires not only
accurate predictions but also transparent alignment between textual rationales
and visual evidence. While Chain-of-Thought (CoT) prompting has shown promise
in medical visual question answering (VQA), no large-scale expert-level dataset
has captured stepwise reasoning with precise visual grounding. We introduce
S-Chain, the first large-scale dataset of 12,000 expert-annotated medical
images with bounding boxes and structured visual CoT (SV-CoT), explicitly
linking visual regions to reasoning steps. The dataset further supports 16
languages, totaling over 700k VQA pairs for broad multilingual applicability.
Using S-Chain, we benchmark state-of-the-art medical VLMs (ExGra-Med,
LLaVA-Med) and general-purpose VLMs (Qwen2.5-VL, InternVL2.5), showing that
SV-CoT supervision significantly improves interpretability, grounding fidelity,
and robustness. Beyond benchmarking, we study its synergy with
retrieval-augmented generation, revealing how domain knowledge and visual
grounding interact during autoregressive reasoning. Finally, we propose a new
mechanism that strengthens the alignment between visual evidence and reasoning,
improving both reliability and efficiency. S-Chain establishes a new benchmark
for grounded medical reasoning and paves the way toward more trustworthy and
explainable medical VLMs.
6901537e646208eac0d1f175
Khai Le-Duc
claimed_verified
2025-10-30T14:42:51.361Z
653c43909d0209606188bf6f
https://cdn-avatars.huggingface.co/v1/production/uploads/653c43909d0209606188bf6f/bnl8dDLLs4Tk9ODqhMrDL.jpeg
Khai Le-Duc
leduckhai
user
leduckhai
6901537e646208eac0d1f176
Duy M. H. Nguyen
6901537e646208eac0d1f177
Phuong T. H. Trinh
6901537e646208eac0d1f178
Tien-Phat Nguyen
6901537e646208eac0d1f179
Nghiem T. Diep
6901537e646208eac0d1f17a
An Ngo
6901537e646208eac0d1f17b
Tung Vu
6901537e646208eac0d1f17c
Trinh Vuong
6901537e646208eac0d1f17d
Anh-Tien Nguyen
6901537e646208eac0d1f17e
Mau Nguyen
6901537e646208eac0d1f17f
Van Trung Hoang
6901537e646208eac0d1f180
Khai-Nguyen Nguyen
6901537e646208eac0d1f181
Hy Nguyen
6901537e646208eac0d1f182
Chris Ngo
6901537e646208eac0d1f183
Anji Liu
6901537e646208eac0d1f184
Nhat Ho
6901537e646208eac0d1f185
Anne-Christin Hauschild
6901537e646208eac0d1f186
Khanh Xuan Nguyen
6901537e646208eac0d1f187
Thanh Nguyen-Tang
6901537e646208eac0d1f188
Pengtao Xie
6901537e646208eac0d1f189
Daniel Sonntag
6901537e646208eac0d1f18a
James Zou
6901537e646208eac0d1f18b
Mathias Niepert
6901537e646208eac0d1f18c
Anh Totti Nguyen
S-Chain: Structured Visual Chain-of-Thought For Medicine
Faithful reasoning in medical vision-language models (VLMs) requires not only
accurate predictions but also transparent alignment between textual rationales
and visual evidence. While Chain-of-Thought (CoT) prompting has shown promise
in medical visual question answering (VQA), no large-scale expert-level dataset
has captured stepwise reasoning with precise visual grounding. We introduce
S-Chain, the first large-scale dataset of 12,000 expert-annotated medical
images with bounding boxes and structured visual CoT (SV-CoT), explicitly
linking visual regions to reasoning steps. The dataset further supports 16
languages, totaling over 700k VQA pairs for broad multilingual applicability.
Using S-Chain, we benchmark state-of-the-art medical VLMs (ExGra-Med,
LLaVA-Med) and general-purpose VLMs (Qwen2.5-VL, InternVL2.5), showing that
SV-CoT supervision significantly improves interpretability, grounding fidelity,
and robustness. Beyond benchmarking, we study its synergy with
retrieval-augmented generation, revealing how domain knowledge and visual
grounding interact during autoregressive reasoning. Finally, we propose a new
mechanism that strengthens the alignment between visual evidence and reasoning,
improving both reliability and efficiency. S-Chain establishes a new benchmark
for grounded medical reasoning and paves the way toward more trustworthy and
explainable medical VLMs.
S-Chain, a large-scale multilingual dataset with structured visual chain-of-thought annotations, enhances the interpretability, grounding, and robustness of medical vision-language models.
Chain-of-Thought CoT prompting medical visual question answering VQA bounding boxes structured visual CoT SV-CoT ExGra-Med LLaVA-Med Qwen2.5-VL InternVL2.5 retrieval-augmented generation autoregressive reasoning grounded medical reasoning trustworthy explainable medical VLMs