Replicability, Repeatability, and Long-term Reproducibility of Cerebellar Morphometry

Abstract

To identify robust and reproducible methods of cerebellar morphometry that can be used in future large-scale structural MRI studies, we investigated the replicability, repeatability, and long-term reproducibility of three fully automated software tools: FreeSurfer, CEREbellum Segmentation (CERES), and automatic cerebellum anatomical parcellation using U-Net with locally constrained optimization (ACAPULCO). Replicability was defined as computational replicability, determined by comparing two analyses of the same high-resolution MRI data set performed with identical analysis software and computer hardware. Repeatability was determined by comparing the analyses of two MRI scans of the same participant taken during two independent MRI sessions on the same day for the Kirby-21 study. Long-term reproducibility was assessed by analyzing two MRI scans of the same participant in the longitudinal OASIS-2 study. We determined percent difference, the image intraclass correlation coefficient, the coefficient of variation, and the intraclass correlation coefficient between two analyses. Our results show that CERES and ACAPULCO use stochastic algorithms that result in surprisingly high differences between identical analyses for ACAPULCO and small differences for CERES. Changes between two consecutive scans from the Kirby-21 study were less than ± 5% in most cases for FreeSurfer and CERES (i.e., demonstrating high repeatability). As expected, long-term reproducibility was lower than repeatability for all software tools. In summary, CERES is an accurate, as demonstrated before, and reproducible tool for fully automated segmentation and parcellation of the cerebellum. We conclude with recommendations for the assessment of replicability, repeatability, and long-term reproducibility in future studies on cerebellar structure.