Machine Learning to Predict Remission Between 6 and 24 Months in Rheumatoid Arthritis: Insights From JAK, an International Registry Collaboration

Abstract

Objective: To develop, externally validate, and simplify a machine learning model to predict remission between 6 and 24 months in patients with rheumatoid arthritis (RA) initiating tumor necrosis factor inhibitors, JAK inhibitors, interleukin-6 inhibitors, abatacept, or rituximab using data from 11 international registries in the JAK-pot collaboration. Methods: We analyzed 21,675 treatment courses for model training, 5,418 courses for internal validation, and 1,807 courses from Switzerland for external validation. Remission was defined as a Clinical Disease Activity Index score ≤2.8 within 6 to 24 months after treatment initiation or switching. An XGBoost model was trained on 63 baseline demographic and clinical variables. Model performance was evaluated using the area under the curve (AUC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Predictor contributions were quantified using Shapley Additive Explanations values. A simplified model using the top 10 predictors and a logistic regression benchmark were also evaluated. Results: In external validation, the full model achieved an AUC of 0.797 (95% confidence interval 0.774–0.821), sensitivity of 0.804, specificity of 0.650, PPV of 0.454, and NPV of 0.902. The simplified model performed comparably (AUC 0.802). Logistic regression using the same 10 predictors achieved an AUC of 0.809. Key predictors included patient global assessment, 28-tender joint count, previous biologic/targeted synthetic disease-modifying antirheumatic drug exposure, and Health Assessment Questionnaire Disability Index score. Conclusion: Using routinely collected baseline data across 11 registries, prediction of remission showed limited discrimination and was best suited to ruling out remission. Performance was similar for a 10-variable model and logistic regression, indicating limited incremental value of model complexity without richer predictors. (Figure presented.).