KD-OCT: Efficient Knowledge Distillation for Clinical-Grade Retinal OCT Classification
Age-related macular degeneration (AMD) and choroidal neovascularization (CNV)-related conditions are leading causes of vision loss worldwide, with optical coherence tomography (OCT) serving as a cornerstone for early detection and management. However, deploying state-of-the-art deep learning models like ConvNeXtV2-Large in clinical settings is hindered by their computational demands. Therefore, it is desirable to develop efficient models that maintain high diagnostic performance while enabling real-time deployment. In this study, a novel knowledge distillation framework, termed KD-OCT, is proposed to compress a high-performance ConvNeXtV2-Large teacher model, enhanced with advanced augmentations, stochastic weight averaging, and focal loss, into a lightweight EfficientNet-B2 student for classifying normal, drusen, and CNV cases. KD-OCT employs real-time distillation with a combined loss balancing soft teacher knowledge transfer and hard ground-truth supervision. The effectiveness of the proposed method is evaluated on the Noor Eye Hospital (NEH) dataset using patient-level cross-validation. Experimental results demonstrate that KD-OCT outperforms comparable multi-scale or feature-fusion OCT classifiers in efficiency- accuracy balance, achieving near-teacher performance with substantial reductions in model size and inference time. Despite the compression, the student model exceeds most existing frameworks, facilitating edge deployment for AMD screening. Code is available at https://github.com/erfan-nourbakhsh/KD- OCT.
693faa8ff516c6932468128a
Erfan Nourbakhsh
claimed_verified
2025-12-16T09:48:48.896Z
6824d49eae455f19806a14bf
https://cdn-avatars.huggingface.co/v1/production/uploads/6824d49eae455f19806a14bf/fg09QTfAcLk3O3_sG6Ak3.png
Erfan Nourbakhsh
Erfan-Nourbakhsh
user
Erfan-Nourbakhsh
693faa8ff516c6932468128b
Nasrin Sanjari
693faa8ff516c6932468128c
Ali Nourbakhsh
KD-OCT: Efficient Knowledge Distillation for Clinical-Grade Retinal OCT Classification
Age-related macular degeneration (AMD) and choroidal neovascularization (CNV)-related conditions are leading causes of vision loss worldwide, with optical coherence tomography (OCT) serving as a cornerstone for early detection and management. However, deploying state-of-the-art deep learning models like ConvNeXtV2-Large in clinical settings is hindered by their computational demands. Therefore, it is desirable to develop efficient models that maintain high diagnostic performance while enabling real-time deployment. In this study, a novel knowledge distillation framework, termed KD-OCT, is proposed to compress a high-performance ConvNeXtV2-Large teacher model, enhanced with advanced augmentations, stochastic weight averaging, and focal loss, into a lightweight EfficientNet-B2 student for classifying normal, drusen, and CNV cases. KD-OCT employs real-time distillation with a combined loss balancing soft teacher knowledge transfer and hard ground-truth supervision. The effectiveness of the proposed method is evaluated on the Noor Eye Hospital (NEH) dataset using patient-level cross-validation. Experimental results demonstrate that KD-OCT outperforms comparable multi-scale or feature-fusion OCT classifiers in efficiency- accuracy balance, achieving near-teacher performance with substantial reductions in model size and inference time. Despite the compression, the student model exceeds most existing frameworks, facilitating edge deployment for AMD screening. Code is available at https://github.com/erfan-nourbakhsh/KD- OCT.
A knowledge distillation framework compresses a large ConvNeXtV2-Large model into a lightweight EfficientNet-B2 for efficient AMD and CNV classification while maintaining high diagnostic performance.
ConvNeXtV2-Large EfficientNet-B2 knowledge distillation optical coherence tomography patient-level cross-validation stochastic weight averaging focal loss real-time distillation soft teacher knowledge transfer hard ground-truth supervision