Interobserver reliability in the classification of pairs of records formed by probabilistic linkage of SISMAMA databases

Abstract

Introduction: The study assessed interobserver reliability in the classification of record pairs formed during probabilistic linkage of health-related databases, a key step in the methodology validation to be used in a larger on-going study on inequalities in the access to breast and cervical cancer control activities in Brazil (DAAC-SIS). Methodology: The RecLink software was used to link two databases of the Breast Cancer Control Information System (SISMAMA) in the state of Minas Gerais, Brazil: A reference database, which included 301 screening mammograms with probable benign diagnosis (BI-RADS 3 category) recorded in October 2010, and a database comprising 158,517 mammograms registered in 2011. Subsequently, the 215 pairs of records that were not assigned the maximum RecLink score were independently classified as being true or false by ten independent evaluators from four participating centers. Results: The Kappa coefficient ranged from 0.87 to 1.00. Six evaluators were in perfect agreement with one or more evaluators from the other centers. The global Kappa was 0.96 (95% confidence interval — 95%CI 0.94 – 0.99). Discussion: Assessment of interobserver reliability is key to ensuring the quality of the record linkage, and it should be routine practice in studies of this nature. The disclosure of such results contributes to transparency in the conduct of such studies and in the reporting of their findings. Conclusion: Interobserver reliability in this study was excellent, indicating satisfactory team consistency in the classification of record pairs.