Medical SAM3: A Foundation Model for Universal Prompt-Driven Medical Image Segmentation
Promptable segmentation foundation models such as SAM3 have demonstrated strong generalization capabilities through interactive and concept-based prompting. However, their direct applicability to medical image segmentation remains limited by severe domain shifts, the absence of privileged spatial prompts, and the need to reason over complex anatomical and volumetric structures. Here we present Medical SAM3, a foundation model for universal prompt-driven medical image segmentation, obtained by fully fine-tuning SAM3 on large-scale, heterogeneous 2D and 3D medical imaging datasets with paired segmentation masks and text prompts. Through a systematic analysis of vanilla SAM3, we observe that its performance degrades substantially on medical data, with its apparent competitiveness largely relying on strong geometric priors such as ground-truth-derived bounding boxes. These findings motivate full model adaptation beyond prompt engineering alone. By fine-tuning SAM3's model parameters on 33 datasets spanning 10 medical imaging modalities, Medical SAM3 acquires robust domain-specific representations while preserving prompt-driven flexibility. Extensive experiments across organs, imaging modalities, and dimensionalities demonstrate consistent and significant performance gains, particularly in challenging scenarios characterized by semantic ambiguity, complex morphology, and long-range 3D context. Our results establish Medical SAM3 as a universal, text-guided segmentation foundation model for medical imaging and highlight the importance of holistic model adaptation for achieving robust prompt-driven segmentation under severe domain shift. Code and model will be made available at https://github.com/AIM-Research-Lab/Medical-SAM3.
696ee0455750539a88acc784
Chongcong Jiang
claimed_verified
2026-01-20T09:39:15.230Z
668ca2bad10c3be5d3514449
https://cdn-avatars.huggingface.co/v1/production/uploads/668ca2bad10c3be5d3514449/LqgRQzwqI3tg0M4G6vKec.jpeg
ChongCongJiang
ChongCong
user
ChongCong
696ee0455750539a88acc785
Tianxingjian Ding
696ee0455750539a88acc786
Chuhan Song
claimed_verified
2026-01-21T09:23:57.907Z
67e5fcd2221779e971ea31a7
/avatars/cf2448aed24ec40cb4851d5eec519772.svg
song
chuhanaaaaa
user
chuhanaaaaa
696ee0455750539a88acc787
Jiachen Tu
696ee0455750539a88acc788
Ziyang Yan
696ee0455750539a88acc789
Yihua Shao
696ee0455750539a88acc78a
Zhenyi Wang
696ee0455750539a88acc78b
Yuzhang Shang
696ee0455750539a88acc78c
Tianyu Han
696ee0455750539a88acc78d
Yu Tian
claimed_verified
2026-01-21T09:23:55.797Z
66da0a14e79d9dea0f692e1f
/avatars/ee189aa30ae566a50b0346a3a709eb6f.svg
Yu Tian
tianyu0207
user
tianyu0207
Medical SAM3: A Foundation Model for Universal Prompt-Driven Medical Image Segmentation
Promptable segmentation foundation models such as SAM3 have demonstrated strong generalization capabilities through interactive and concept-based prompting. However, their direct applicability to medical image segmentation remains limited by severe domain shifts, the absence of privileged spatial prompts, and the need to reason over complex anatomical and volumetric structures. Here we present Medical SAM3, a foundation model for universal prompt-driven medical image segmentation, obtained by fully fine-tuning SAM3 on large-scale, heterogeneous 2D and 3D medical imaging datasets with paired segmentation masks and text prompts. Through a systematic analysis of vanilla SAM3, we observe that its performance degrades substantially on medical data, with its apparent competitiveness largely relying on strong geometric priors such as ground-truth-derived bounding boxes. These findings motivate full model adaptation beyond prompt engineering alone. By fine-tuning SAM3's model parameters on 33 datasets spanning 10 medical imaging modalities, Medical SAM3 acquires robust domain-specific representations while preserving prompt-driven flexibility. Extensive experiments across organs, imaging modalities, and dimensionalities demonstrate consistent and significant performance gains, particularly in challenging scenarios characterized by semantic ambiguity, complex morphology, and long-range 3D context. Our results establish Medical SAM3 as a universal, text-guided segmentation foundation model for medical imaging and highlight the importance of holistic model adaptation for achieving robust prompt-driven segmentation under severe domain shift. Code and model will be made available at https://github.com/AIM-Research-Lab/Medical-SAM3.
Medical SAM3 adapts the SAM3 foundation model through comprehensive fine-tuning on diverse medical imaging datasets to achieve robust prompt-driven segmentation across various modalities and anatomical structures.
foundation model prompt-driven segmentation medical image segmentation SAM3 fine-tuning medical imaging modalities anatomical structures domain shift text prompts segmentation masks geometric priors universal segmentation