What Drives AI Diagnosis in Diabetic Retinopathy? Insights from Heatmap Analysis.

Introduction:
Artificial intelligence (AI) is increasingly used in healthcare to support disease detection and expand clinical access. AI systems screening for features of for diabetic retinopathy (DR) (microaneurysms, haemorrhages, exudates, venous beading, intraretinal microvascular abnormalities and neovascularisation) are now implemented in Australia and internationally. While diagnostic performance is well documented, retinal regions and clinical features that most influence AI predictions remain unclear.
Aims:
To identify retinal locations and DR features that most influence outputs in DR detection algorithms.
Methods:
Twenty seven AI models were developed to detect referable DR from macular centred photographs. Models were trained on 35,158 images spanning a range of severities. Each model analysed 757 annotated images, generating GradCAM heatmaps to identify influential regions. Heatmaps were aggregated to identify common patterns. Locations of specific features were compared with heatmaps using Sørensen-Dice coefficients to assess feature influence.
Results:
Across models, mean sensitivity was 65.7%, specificity 89.2% and F1-score 0.77. Heatmap patterns varied: 11 models (41%) predominantly highlighted the central and/or temporal macula, 10 (37%) highlighted broad retinal regions, three (11%) emphasised photo boundaries, and three (11%) showed no interpretable pattern. Heatmaps correlated most strongly with exudates (0.39), with weaker correlation for venous beading (0.11) and neovascularisation at the disc (0.11).
Conclusions:
AI models differ regions detected as abnormal and appear more influenced by exudates than venous beading and neovascularisation. Paradoxically, exudates do not influence human grading whilst venous beading and neovascularisation are highly predictive of blindness.
Impact:
Annotations of retinal locations and features that most influence AI categorisations are vital for clinical contextualisation of AI outputs and improving clinician acceptance.