An information theoretic approach for measuring data discovery and utilization during analytical and decision-making processes

Abstract

Across many commercial, government, and military environments, multi-level decision-making processes rely on complex sociotechnical systems. Human dynamics are a significant driver of the overall effectiveness of these processes, yet the characterization of both the intraand inter-individual performance contribution is limited by sparse, qualitative, and often subjective observations. Recent advances in quantitative human-machine instrumentation have made possible greater objective study of users interacting with data; however, performance metrics leveraging these measurements are often narrow and ad hoc. When assessing the analytical and decision-making performance of teams, it is critical to know the information that members have observed, synthesized, and acted upon, and ad hoc approaches can be insufficient. In this paper we present a novel assessment framework based on the principles of Shannon information theory. We detail how this framework can holistically characterize decision information flows and describe its application to assess teams’ abilities to effectively discover data during serious games.