Generalizable Knowledge Distillation from Vision Foundation Models for Semantic Segmentation
Knowledge distillation (KD) has been widely applied in semantic segmentation to compress large models, but conventional approaches primarily preserve in-domain accuracy while neglecting out-of-domain generalization, which is essential under distribution shifts. This limitation becomes more severe with the emergence of vision foundation models (VFMs): although VFMs exhibit strong robustness on unseen data, distilling them with conventional KD often compromises this ability. We propose Generalizable Knowledge Distillation (GKD), a multi-stage framework that explicitly enhances generalization. GKD decouples representation learning from task learning. In the first stage, the student acquires domain-agnostic representations through selective feature distillation, and in the second stage, these representations are frozen for task adaptation, thereby mitigating overfitting to visible domains. To further support transfer, we introduce a query-based soft distillation mechanism, where student features act as queries to teacher representations to selectively retrieve transferable spatial knowledge from VFMs. Extensive experiments on five domain generalization benchmarks demonstrate that GKD consistently outperforms existing KD methods, achieving average gains of +1.9% in foundation-to-foundation (F2F) and +10.6% in foundation-to-local (F2L) distillation. The code will be available at https://github.com/Younger-hua/GKD.
69a831718b5b27c7ea9fa460
Chonghua Lv
admin_assigned
2026-03-09T15:40:43.889Z
6448cef781c086e0a4202ba2
/avatars/a857873bd16d6097df4829c0c015079e.svg
Chonghua Lv
yongers
user
yongers
69a831718b5b27c7ea9fa461
Dong Zhao
69a831718b5b27c7ea9fa462
Shuang Wang
69a831718b5b27c7ea9fa463
Dou Quan
69a831718b5b27c7ea9fa464
Ning Huyan
69a831718b5b27c7ea9fa465
Nicu Sebe
69a831718b5b27c7ea9fa466
Zhun Zhong
Generalizable Knowledge Distillation from Vision Foundation Models for Semantic Segmentation
Knowledge distillation (KD) has been widely applied in semantic segmentation to compress large models, but conventional approaches primarily preserve in-domain accuracy while neglecting out-of-domain generalization, which is essential under distribution shifts. This limitation becomes more severe with the emergence of vision foundation models (VFMs): although VFMs exhibit strong robustness on unseen data, distilling them with conventional KD often compromises this ability. We propose Generalizable Knowledge Distillation (GKD), a multi-stage framework that explicitly enhances generalization. GKD decouples representation learning from task learning. In the first stage, the student acquires domain-agnostic representations through selective feature distillation, and in the second stage, these representations are frozen for task adaptation, thereby mitigating overfitting to visible domains. To further support transfer, we introduce a query-based soft distillation mechanism, where student features act as queries to teacher representations to selectively retrieve transferable spatial knowledge from VFMs. Extensive experiments on five domain generalization benchmarks demonstrate that GKD consistently outperforms existing KD methods, achieving average gains of +1.9% in foundation-to-foundation (F2F) and +10.6% in foundation-to-local (F2L) distillation. The code will be available at https://github.com/Younger-hua/GKD.
Generalizable Knowledge Distillation (GKD) enhances out-of-domain generalization in semantic segmentation by decoupling representation learning from task learning and using query-based soft distillation to transfer knowledge from vision foundation models.
knowledge distillation semantic segmentation vision foundation models domain generalization representation learning task learning selective feature distillation query-based soft distillation transferable spatial knowledge