Comparative Effectiveness of AI-Assisted vs Manual Annotations of Geographic Atrophy Using OPTIMEyes

Document Type

Conference Proceeding

Publication Date

6-2025

Publication Title

Investigative Ophthalmology and Visual Science

Abstract

Purpose : Manual annotation of geographic atrophy (GA) secondary to age-related macular degeneration is time consuming and subjective. This study evaluates the OptimEYES platform, incorporating AI assistance for the annotation of GA in fundus autofluorescence images. We compare manual and AI assisted segmentation on the same dataset to assess annotation efficiency and quality.

Methods : We used a retrospective GA dataset collected from the Sue Anschutz Rodgers Eye Center, comprised of 110 images, which were manually annotated by a retina specialist. The dataset was divided into 100 images for training and 10 images for testing a MedSAM segmentation model. The annotators consisted of residents, fellows, and medical students from the University of Colorado Anschutz and were assigned 10 training image subsets with no overlap and the test set. The first two rounds of annotation on the test set were conducted using manual annotation and compared with the AI –assisted annotations using the trained MedSAM model. Efficiency, or time taken per annotation, and quality (DICE) metrics were used to assess performance.

Results : The distribution of time differences between manual and ai-assisted annotations (Figure 1a) shows that it takes on average 93 seconds more to annotate images manually than with ai-assistance. The two manual annotation sessions (Figure 1b) had a small time difference (Mean: -2.02s, Median: 61.50s). Results for manual versus untrained ROIs (not shown) were similar though things took longer with untrained ROIs (Mean: -454.96s, Median: -0.50s). Regarding quality, between ai-assisted and manual annotations, there was minimal change in DICE (Mean: 0.01, Median: 0.01s) (Figure 2a). Ai-assistance helped more difficult cases produce better quality results, as shown in Figure 2b, where higher DICE gains are obtained for images whose manual segmentation was incorrect (i.e., lower manual DICE score).

Conclusions : This study showed the promise of OptimEYES in enhancing efficiency while maintaining high annotation accuracy for GA segmentation from fundus autofluorescence.

Volume

66

Issue

8

First Page

600

Comments

Association for Research in Vision and Ophthalmology ARVO Annual Meeting, May 4-8, 2025, Salt Lake City, UT

Last Page

600

Link to Full Text

Share

COinS