HBase, MapReduce, and integrated data visualization for processing clinical signal data

Abstract

Processing high-density clinical signal data (data from biomedical sensors deployed in the clinical environment) is resource intensive and time consuming. We propose a novel approach to storing and processing clinical signal data based on the Apache HBase distributed column-store and the MapReduce programming paradigm with an integrated webbased data visualization layer. An integrated solution negates the need to marshal data into and out of the storage system while also easily parallelizing the computation, a problem that is becoming more and more important due to increasing numbers of sensors and resulting data. We estimate upwards of 50TB of clinical signal data for a 200-bed medical center within the next 5 years. Consequently, efficient processing of clinical signal data is a vital step towards multivariate analysis of the signal data in order to develop better ways of describing a patient's clinical status. Copyright © 2011, Association for the Advancement of Artificial Intelligence. All rights reserved.