Strengths and weaknesses of three software programs for the comparison of systems based on ROC curves

Abstract

The Receiver Operating Characteristic (ROC) analysis is a technique that is applied in medical diagnostic testing, especially for discriminating between two health status of a patient: normal (negative) and abnormal (positive). Its ability to compare the performance of different diagnostic systems based on an empirical estimation of the area under the ROC curve (AUC) has made this technique very attractive. Thus, it is important to select an appropriate software program to carry out this comparison. However, this selection has been a difficult task, considering the operational features available in each program. In this work, we aimed to demonstrate how three of the software programs available on the market allow comparing different systems based on AUC indicator, and which tests they use. The features, functionality and performance of the three software programs were evaluated, as well as advantages and disadvantages associated with their use. For illustrative purposes, we used one dataset of the Clinical Risk Index for Babies (CRIB) from Neonatal Intensive Care Units (NICUs) in Portugal.