Opening the black box of electronic health: Collecting, analyzing, and interpreting log data

Abstract

In electronic health (eHealth) research, limited insight has been obtained on process outcomes or how the use of technology has contributed to the users' ability to have a healthier life, improved well-being, or activate new attitudes in their daily tasks. As a result, eHealth is often perceived as a black box. To open this black box of eHealth, methodologies must extend beyond the classic effect evaluations. The analyses of log data (anonymous records of real-time actions performed by each user) can provide continuous and objective insights into the actual usage of the technology. However, the possibilities of log data in eHealth research have not been exploited to their fullest extent. The aim of this paper is to describe how log data can be used to improve the evaluation and understand the use of eHealth technology with a broader approach than only descriptive statistics. This paper serves as a starting point for using log data analysis in eHealth research. Here, we describe what log data is and provide an overview of research questions to evaluate the system, the context, the users of a technology, as well as the underpinning theoretical constructs. We also explain the requirements for log data, the starting points for the data preparation, and methods for data collection. Finally, we describe methods for data analysis and draw a conclusion regarding the importance of the results for both scientific and practical applications. The analysis of log data can be of great value for opening the black box of eHealth. A deliberate log data analysis can give new insights into how the usage of the technology contributes to found effects and can thereby help to improve the persuasiveness and effectiveness of eHealth technology and the underpinning behavioral models.