MedSAM3: Delving into Segment Anything with Medical Concepts
Medical image segmentation is fundamental for biomedical discovery. Existing methods lack generalizability and demand extensive, time-consuming manual annotation for new clinical application. Here, we propose MedSAM-3, a text promptable medical segmentation model for medical image and video segmentation. By fine-tuning the Segment Anything Model (SAM) 3 architecture on medical images paired with semantic conceptual labels, our MedSAM-3 enables medical Promptable Concept Segmentation (PCS), allowing precise targeting of anatomical structures via open-vocabulary text descriptions rather than solely geometric prompts. We further introduce the MedSAM-3 Agent, a framework that integrates Multimodal Large Language Models (MLLMs) to perform complex reasoning and iterative refinement in an agent-in-the-loop workflow. Comprehensive experiments across diverse medical imaging modalities, including X-ray, MRI, Ultrasound, CT, and video, demonstrate that our approach significantly outperforms existing specialist and foundation models. We will release our code and model at https://github.com/Joey-S-Liu/MedSAM3.
69267465243b2216fb75ca9d
Anglin Liu
claimed_verified
2025-11-26T09:27:33.112Z
66214b651fc3a06144ef8f4b
/avatars/dab669ad56b67805d55fef8c4fcf1326.svg
Anglin Liu
lal-Joey
user
lal-Joey
69267465243b2216fb75ca9e
Rundong Xue
claimed_verified
2026-02-04T12:35:09.604Z
68161968cf94c50b9e6fe265
/avatars/9c17dad2a2b522acabb22bd862049599.svg
Rundong Xue
Rendon2002
user
Rendon2002
69267465243b2216fb75ca9f
Xu R. Cao
claimed_verified
2025-11-26T09:27:30.698Z
5fc9f05d52770aca770bd3d9
https://cdn-avatars.huggingface.co/v1/production/uploads/5fc9f05d52770aca770bd3d9/axiKCXPYttXEWC-RqCSfh.jpeg
Xu Cao
IrohXu
user
IrohXu
69267465243b2216fb75caa0
Yifan Shen
claimed_verified
2025-11-27T09:59:55.425Z
65e387095132c2edd193ae49
/avatars/39278e5b026bcbdde88c560fc54018c5.svg
Yifan Shen
SivanSX
user
SivanSX
69267465243b2216fb75caa1
Yi Lu
69267465243b2216fb75caa2
Xiang Li
69267465243b2216fb75caa3
Qianqian Chen
69267465243b2216fb75caa4
Jintai Chen
MedSAM3: Delving into Segment Anything with Medical Concepts
Medical image segmentation is fundamental for biomedical discovery. Existing methods lack generalizability and demand extensive, time-consuming manual annotation for new clinical application. Here, we propose MedSAM-3, a text promptable medical segmentation model for medical image and video segmentation. By fine-tuning the Segment Anything Model (SAM) 3 architecture on medical images paired with semantic conceptual labels, our MedSAM-3 enables medical Promptable Concept Segmentation (PCS), allowing precise targeting of anatomical structures via open-vocabulary text descriptions rather than solely geometric prompts. We further introduce the MedSAM-3 Agent, a framework that integrates Multimodal Large Language Models (MLLMs) to perform complex reasoning and iterative refinement in an agent-in-the-loop workflow. Comprehensive experiments across diverse medical imaging modalities, including X-ray, MRI, Ultrasound, CT, and video, demonstrate that our approach significantly outperforms existing specialist and foundation models. We will release our code and model at https://github.com/Joey-S-Liu/MedSAM3.
MedSAM-3, a text-promptable medical segmentation model fine-tuned on SAM 3 architecture, achieves superior performance across various medical imaging modalities using semantic conceptual labels and multimodal large language models.
Segment Anything Model (SAM) text promptable medical segmentation Promptable Concept Segmentation (PCS) Multimodal Large Language Models (MLLMs) X-ray MRI Ultrasound CT video