Uncertainty-Aware Vision-Language Segmentation for Medical Imaging
We introduce a novel uncertainty-aware multimodal segmentation framework that leverages both radiological images and associated clinical text for precise medical diagnosis. We propose a Modality Decoding Attention Block (MoDAB) with a lightweight State Space Mixer (SSMix) to enable efficient cross-modal fusion and long-range dependency modelling. To guide learning under ambiguity, we propose the Spectral-Entropic Uncertainty (SEU) Loss, which jointly captures spatial overlap, spectral consistency, and predictive uncertainty in a unified objective. In complex clinical circumstances with poor image quality, this formulation improves model reliability. Extensive experiments on various publicly available medical datasets, QATA-COVID19, MosMed++, and Kvasir-SEG, demonstrate that our method achieves superior segmentation performance while being significantly more computationally efficient than existing State-of-the-Art (SoTA) approaches. Our results highlight the importance of incorporating uncertainty modelling and structured modality alignment in vision-language medical segmentation tasks. Code: https://github.com/arya-domain/UA-VLS
6997c5907a658569d5a10142
Aryan Das
claimed_verified
2026-02-25T17:32:15.867Z
64cd33bb7c24890fb477f341
/avatars/60b1acd773acef5a51a314b49867f5ca.svg
Aryan Das
aryadomain
user
aryadomain
6997c5907a658569d5a10143
Tanishq Rachamalla
claimed_verified
2026-02-20T08:36:27.151Z
663f3b7be527874400e6fa25
/avatars/e0f6ac7d9a27473c0cfa3817b56c2eda.svg
Tanishq Rachamalla
Tanishq165
user
Tanishq165
6997c5907a658569d5a10144
Koushik Biswas
6997c5907a658569d5a10145
Swalpa Kumar Roy
6997c5907a658569d5a10146
Vinay Kumar Verma
Uncertainty-Aware Vision-Language Segmentation for Medical Imaging
We introduce a novel uncertainty-aware multimodal segmentation framework that leverages both radiological images and associated clinical text for precise medical diagnosis. We propose a Modality Decoding Attention Block (MoDAB) with a lightweight State Space Mixer (SSMix) to enable efficient cross-modal fusion and long-range dependency modelling. To guide learning under ambiguity, we propose the Spectral-Entropic Uncertainty (SEU) Loss, which jointly captures spatial overlap, spectral consistency, and predictive uncertainty in a unified objective. In complex clinical circumstances with poor image quality, this formulation improves model reliability. Extensive experiments on various publicly available medical datasets, QATA-COVID19, MosMed++, and Kvasir-SEG, demonstrate that our method achieves superior segmentation performance while being significantly more computationally efficient than existing State-of-the-Art (SoTA) approaches. Our results highlight the importance of incorporating uncertainty modelling and structured modality alignment in vision-language medical segmentation tasks. Code: https://github.com/arya-domain/UA-VLS
A novel uncertainty-aware multimodal segmentation framework uses radiological images and clinical text with Modality Decoding Attention Blocks and Spectral-Entropic Uncertainty Loss for improved medical diagnosis accuracy.
Modality Decoding Attention Block State Space Mixer cross-modal fusion long-range dependency modelling Spectral-Entropic Uncertainty Loss vision-language medical segmentation uncertainty modelling multimodal segmentation