Batch and Streaming Data Ingestion towards Creating Holistic Health Records

Abstract

The healthcare sector has been moving toward Electronic Health Record (EHR) systems that produce enormous amounts of healthcare data due to the increased emphasis on getting the appropriate information to the right person, wherever they are, at any time. This highlights the need for a holistic approach to ingest, exploit, and manage these huge amounts of data for achieving better health management and promotion in general. This manuscript proposes such an approach, providing a mechanism allowing all health ecosystem entities to obtain actionable knowledge from heterogeneous data in a multimodal way. The mechanism includes diverse techniques for automatically ingesting healthcare-related information from heterogeneous sources that produce batch/streaming data, managing, fusing, and aggregating this data into new data structures (i.e., Holistic Health Records (HHRs)). The latter enable the aggregation of data coming from different sources, such as Internet of Medical Things (IoMT) devices, online/offline platforms, while to effectively construct the HHRs, the mechanism develops various data management techniques covering the overall data path, from data acquisition and cleaning to data integration, modelling, and interpretation. The mechanism has been evaluated upon different healthcare scenarios, ranging from hospital-retrieved data to patient platforms, combined with data obtained from IoMT devices, having produced useful insights towards its successful and wide adaptation in this domain. In order to implement a paradigm shift from heterogeneous and independent data sources, limited data exploitation, and health records, the mechanism has combined multidisciplinary technologies toward HHRs.