A review of multimodal medical data fusion techniques for personalized medicine

Abstract

With the advent of Industry 4.0, the field of medicine has generated a large amount of heterogeneous multimodal data, including structured data (such as electronic medical records, and laboratory results), unstructured data (such as medical images), and semi-structured data (such as Electronic Health Records (EHR) logs). These data types have their characteristics and advantages, but due to their heterogeneity, it is very difficult to directly fuse the original data. Different modalities of data often require specific preprocessing and feature extraction methods to provide high-quality data representations suitable for artificial intelligence models. Multimodal data fusion is the key to solving this problem. Integrating complementary information from different modalities overcomes the limitations of single-modality methods and provides more accurate and comprehensive support for disease diagnosis, personalized treatment, and clinical decision-making. This review systematically investigates current multimodal medical data fusion technologies, with a particular focus on deep learning-based data representation and fusion methods. This paper firstly briefly introduces commonly used methods for representing healthcare data, secondly summarizes existing data preprocessing techniques and explores classical deep learning fusion techniques for personalized healthcare at three different levels of fusion. Finally, this article explores the challenges and future development directions brought by artificial intelligence technology in this field and elucidates its profound impact on multimodal medical data fusion.