SAINTENS: Self-Attention and Intersample Attention Transformer for Digital Biomarker Development Using Tabular Healthcare Real World Data

Abstract

Background: Deep learning currently struggles with tabular data, but it can benefit from multimodal learning. SAINT is a deep learning model for tabular data on which we base our presented developments. Objectives: In this study, we introduce SAINTENS as a new deep learning method, specifically for the in healthcare predominant tabular real world data. Methods: For this purpose, we compare SAINTENS with SAINT and the State of the Art Machine Learning methods for tabular data. We use tabular data from geriatrics to predict four different targets (dysphagia, pressure ulcers, decompensated heart failure and delirium). We determine the relevant feature sets and train the models on these sets. Results: Both SAINTENS and SAINT models are at least on the same performance level as the current State of the Art (Gradient Boosting Decision Trees). Conclusion: In combination with multimodal learning SAINTENS and SAINT may be used on real world data comprising tabular, text and image data, for discovery and development of new digital biomarkers.