TAT: Task-Adaptive Transformer for All-in-One Medical Image Restoration
Medical image restoration (MedIR) aims to recover high-quality medical images from their low-quality counterparts. Recent advancements in MedIR have focused on All-in-One models capable of simultaneously addressing multiple different MedIR tasks. However, due to significant differences in both modality and degradation types, using a shared model for these diverse tasks requires careful consideration of two critical inter-task relationships: task interference, which occurs when conflicting gradient update directions arise across tasks on the same parameter, and task imbalance, which refers to uneven optimization caused by varying learning difficulties inherent to each task. To address these challenges, we propose a task-adaptive Transformer (TAT), a novel framework that dynamically adapts to different tasks through two key innovations. First, a task-adaptive weight generation strategy is introduced to mitigate task interference by generating task-specific weight parameters for each task, thereby eliminating potential gradient conflicts on shared weight parameters. Second, a task-adaptive loss balancing strategy is introduced to dynamically adjust loss weights based on task-specific learning difficulties, preventing task domination or undertraining. Extensive experiments demonstrate that our proposed TAT achieves state-of-the-art performance in three MedIR tasks--PET synthesis, CT denoising, and MRI super-resolution--both in task-specific and All-in-One settings. Code is available at https://github.com/Yaziwel/TAT.
694244985d5b2dc10527492c
Zhiwen Yang
admin_assigned
2025-12-17T10:13:01.563Z
638a2bdb34cf0480e9abd6a4
/avatars/70e84c5c188688db8b29455061a410a1.svg
Zhiwen Yang
upyzwup
user
upyzwup
694244985d5b2dc10527492d
Jiaju Zhang
admin_assigned
2025-12-17T10:13:07.908Z
63faca946b75d93aa1363c00
https://cdn-avatars.huggingface.co/v1/production/uploads/63faca946b75d93aa1363c00/UNFznCNTXnO4e-GnkImgr.jpeg
Jiaju Zhang
Jiaju
user
Jiaju
694244985d5b2dc10527492e
Yang Yi
694244985d5b2dc10527492f
Jian Liang
694244985d5b2dc105274930
Bingzheng Wei
694244985d5b2dc105274931
Yan Xu
TAT: Task-Adaptive Transformer for All-in-One Medical Image Restoration
Medical image restoration (MedIR) aims to recover high-quality medical images from their low-quality counterparts. Recent advancements in MedIR have focused on All-in-One models capable of simultaneously addressing multiple different MedIR tasks. However, due to significant differences in both modality and degradation types, using a shared model for these diverse tasks requires careful consideration of two critical inter-task relationships: task interference, which occurs when conflicting gradient update directions arise across tasks on the same parameter, and task imbalance, which refers to uneven optimization caused by varying learning difficulties inherent to each task. To address these challenges, we propose a task-adaptive Transformer (TAT), a novel framework that dynamically adapts to different tasks through two key innovations. First, a task-adaptive weight generation strategy is introduced to mitigate task interference by generating task-specific weight parameters for each task, thereby eliminating potential gradient conflicts on shared weight parameters. Second, a task-adaptive loss balancing strategy is introduced to dynamically adjust loss weights based on task-specific learning difficulties, preventing task domination or undertraining. Extensive experiments demonstrate that our proposed TAT achieves state-of-the-art performance in three MedIR tasks--PET synthesis, CT denoising, and MRI super-resolution--both in task-specific and All-in-One settings. Code is available at https://github.com/Yaziwel/TAT.
A task-adaptive Transformer framework addresses interference and imbalance challenges in multi-task medical image restoration through adaptive weight generation and loss balancing strategies.
task-adaptive Transformer task interference task imbalance gradient update directions task-specific weight parameters loss balancing medical image restoration PET synthesis CT denoising MRI super-resolution