Boosting Medical Visual Understanding From Multi-Granular Language Learning
Recent advances in image-text pretraining have significantly enhanced visual understanding by aligning visual and textual representations. Contrastive Language-Image Pretraining (CLIP) has played a pivotal role in multimodal learning. However, its focus on single-label, single-granularity alignment limits its effectiveness in complex domains such as medical imaging, where images often correspond to multiple high-level labels (e.g., disease categories) across different annotation granularities (e.g., diagnostic description, clinical explanation). To address this, we propose Multi-Granular Language Learning (MGLL), a contrastive learning framework designed to improve both multi-label and cross-granularity alignment. MGLL leverages structured multi-label supervision, integrates textual descriptions across granularities, and introduces soft-label supervision with point-wise constraints to enhance alignment. MGLL employs smooth Kullback-Leibler (KL) divergence to ensure cross-granularity consistency while maintaining computational efficiency as a plug-and-play module for vision-language models. Pretrained on our constructed large-scale multi-granular datasets and evaluated across multiple datasets, MGLL outperforms other state-of-the-art methods in downstream tasks. The code is available at https://github.com/HUANGLIZI/MGLL{https://github.com/HUANGLIZI/MGLL}.
692023d0cce0eb9b6387b069
Zihan Li
claimed_verified
2025-11-21T08:59:16.511Z
64176d65b03817ada642b6e2
/avatars/27fe82eaf9db2894c0bf86acb59543fe.svg
Zihan Li
zl111
user
zl111
692023d0cce0eb9b6387b06a
Yiqing Wang
claimed_verified
2026-01-27T09:44:24.438Z
68261812ca862732ad98ea45
/avatars/4b5701b9968b1d39f3c39765181646c3.svg
Yiqing Wang
yqwang01
user
yqwang01
692023d0cce0eb9b6387b06b
Sina Farsiu
692023d0cce0eb9b6387b06c
Paul Kinahan
Boosting Medical Visual Understanding From Multi-Granular Language Learning
Recent advances in image-text pretraining have significantly enhanced visual understanding by aligning visual and textual representations. Contrastive Language-Image Pretraining (CLIP) has played a pivotal role in multimodal learning. However, its focus on single-label, single-granularity alignment limits its effectiveness in complex domains such as medical imaging, where images often correspond to multiple high-level labels (e.g., disease categories) across different annotation granularities (e.g., diagnostic description, clinical explanation). To address this, we propose Multi-Granular Language Learning (MGLL), a contrastive learning framework designed to improve both multi-label and cross-granularity alignment. MGLL leverages structured multi-label supervision, integrates textual descriptions across granularities, and introduces soft-label supervision with point-wise constraints to enhance alignment. MGLL employs smooth Kullback-Leibler (KL) divergence to ensure cross-granularity consistency while maintaining computational efficiency as a plug-and-play module for vision-language models. Pretrained on our constructed large-scale multi-granular datasets and evaluated across multiple datasets, MGLL outperforms other state-of-the-art methods in downstream tasks. The code is available at https://github.com/HUANGLIZI/MGLL{https://github.com/HUANGLIZI/MGLL}.
Multi-Granular Language Learning (MGLL) enhances visual understanding by improving multi-label and cross-granularity alignment in image-text pretraining, outperforming existing methods in complex domains like medical imaging.
Contrastive Language-Image Pretraining (CLIP) multi-granular alignment structured multi-label supervision soft-label supervision smooth Kullback-Leibler (KL) divergence vision-language models