Publicly available retinal image data and the use of competitions to standardize algorithm performance comparison

Abstract

In recent years a number of retinal image databases have been made publicly available. A popular image database in the retinal image analysis field is the DRIVE retinal vasculature database. In general this type of image database contains a reference standard, in case of the DRIVE database these are segmentations of the vasculature, usually produced by experts. The public availability of this type of medical data stimulates research and innovation within the field by enabling direct comparison between different algorithms. Using this data the evaluation itself is still performed by the individual researchers and this can lead to the use of different evaluation measures and implementation differences. To remove these sources of variation, the organization of competitions for segmentation and detection in which the participants send their results to the competition organizers who perform the evaluation has been proposed. This guarantees a standardized evaluation process. In this work we will describe several different publicly available databases of retinal data and their specifications. The importance of standardized evaluation will be discussed using an example from the DRIVE database. Special attention will be paid to competitive evaluation of algorithms and the first results obtained by the ROC: microaneurysm detection competition. © 2009 Springer-Verlag.