End-to-end learning to predict survival in patients with gastric cancer using convolutional neural networks

Abstract

Background: While established deep learning approaches for histopathology usually consist of a two-step process, a cell or region segmentation and subsequent feature calculation, end-to-end learning has been used to predict patient survival directly from digital tissue sections. We aimed to apply a deep learning approach in a series of gastric cancer (GC) tissue microarrays (TMAs) in order to identify regions in the tissue related to a high-risk of poor survival, and subsequently stratify patients into two risk groups. Methods: Image patches (size 80mm) were extracted from 469 TMA cores constructed from 248 GC resection specimens which were scanned after immunohistochemistry for CD8 and KI67. For each stain, a survival convolutional neural network (CNN) was trained to maximize a log partial likelihood derived from the Cox proportional hazards model [Mobadersany, PNAS, 2018] and to predict patch-based risks for cancer-specific death in a 10-fold pre-validation procedure, creating risk heatmaps for each core. Aggregation from patch to patient level was done by averaging the risks from all patches of each patient. Results: We generated risk heatmaps comprising on median 1300 image patches per patient for the CD8 and KI67 stained tissue sections. Stratifying patients into low- and high-risk groups by taking the cohort median as threshold led to a significant log-rank test p-value (<0.01). Regarding the Lauren classification, the diffuse type was associated with higher risks than the intestinal type (T-test p-value < 0.015). Visual assessment of the risk heatmaps revealed an association of low-risk regions in CD8-stained sections with clusters of CD8(+) cells and presence of CD8(+) cells in stroma, whereas tumor epithelium and stroma regions with a low density of CD8(+) cells are associated with higher risks. Conclusions: We applied survival CNNs to IHC stained gastric cancer tissue samples to directly associate image regions with cancer-specific death risks. This information may be used to deepen our knowledge on how tissue morphology relates to survival risk, and to stratify patients into high and low risk groups. Our results will be extended to other biomarkers and will be validated using data from another clinical site.