Validation and update of a multivariable prediction model for the identification and management of patients at risk for hepatocellular carcinoma

Abstract

Background: A hepatocellular carcinoma (HCC) prediction model (ASAP), including age, sex, and the biomarkers alpha-fetoprotein and prothrombin induced by vitamin K absence-II, showed potential clinical value in the early detection of HCC. We validated and updated the model in a real-world cohort and promoted its transferability to daily clinical practice. Methods: This retrospective cohort analysis included 1012 of the 2479 eligible patients aged 35 years or older undergoing surveillance for HCC. The data were extracted from the electronic medical records. Biomarker values within the test-to-diagnosis interval were used to validate the ASAP model. Due to its unsatisfactory calibration, three logistic regression models were constructed to recalibrate and update the model. Their discrimination, calibration, and clinical utility were compared. The performance statistics of the final updated model at several risk thresholds are presented. The outcomes of 855 non-HCC patients were further assessed during a median of 10.2 months of follow-up. Statistical analyses were performed using packages in R software. Results: The ASAP model had superior discriminative performance in the validation cohort [C-statistic = 0.982, (95% confidence interval 0.972–0.992)] but significantly overestimated the risk of HCC (intercept − 3.243 and slope 1.192 in the calibration plot), reducing its clinical usefulness. Recalibration-in-the-large, which exhibited performance comparable to that of the refitted model revision, led to the retention of the excellent discrimination and substantial improvements in the calibration and clinical utility, achieving a sensitivity of 100% at the median prediction probability of the absence of HCC (1.3%). The probability threshold of 1.3% and the incidence of HCC in the cohort (15.5%) were used to stratify the patients into low-, medium-, and high-risk groups. The cumulative HCC incidences in the non-HCC patients significantly differed among the risk groups (log-rank test, p-value < 0.001). The 3-month, 6-month and 18-month cumulative incidences in the low-risk group were 0.6%, 0.9% and 0.9%, respectively. Conclusions: The ASAP model is an accurate tool for HCC risk estimation that requires recalibration before use in a new region because calibration varies with clinical environments. Additionally, rational risk stratification and risk-based management decision-making, e.g., 3-month follow-up recommendations for targeted individuals, helped improve HCC surveillance, which warrants assessment in larger cohorts.