Sequential Multi-task Learning for Histopathology-Based Prediction of Genetic Mutations with Extremely Imbalanced Labels

Abstract

H &E images can be utilized to predict genetic mutations as biomarkers to potentially substitute many molecular biomarker assays in order to aid patients. Having a single model built by conducting prediction tasks simultaneously can save computation resources and provide a more generalizable model for future usage. A basic technique for generating such a comprehensive and efficient model is to employ a multi-task learning approach. However, overfitting the model to the trivial answers can occur in training for multiple tasks with extremely imbalanced class labels where resampling and rebalancing for all minor classes simultaneously are prohibited. Herein we propose a sequential multi-task learning approach to train a single model capable of predicting multiple genetic mutations while avoiding overfitting to trivial answers for imbalanced classes. We compared our strategy to the baseline multi-task training, as well as two more advanced approaches: (1) using weighted loss and (2) using self-supervised pre-training. We also used a trimming method to deal with noisy labels. To assess our methods, we trained models to predict 10 genetic mutations on the H &E images of the TCGA-LUAD dataset. AUROC and F1 score are reported, while we demonstrate that F1 score may be a more suitable metric for multi-task learning with imbalanced labels. It is shown that our proposed trimming strategy combined with sequential learning could improve the predictions on all of the mutations compared with other multi-task learning approaches. Also, we investigated the application of continual learning.