Forecasters’ cognitive task analysis and mental workload analysis of issuing probabilistic hazard information (PHI) during facets PHI prototype experiment

Abstract

During spring 2016 the Probabilistic Hazard Information (PHI) prototype experiment was run in the National Oceanic and Atmospheric Administration (NOAA) Hazardous Weather Testbed (HWT) as part of the Forecasting a Continuum of Environmental Threats (FACETS) program. Nine National Weather Service forecasters were trained to use the web-based PHI prototype tool to produce dynamic PHI for severe weather threats. Archived and real-time weather scenarios were used to test this new paradigm of issuing probabilistic information, rather than deterministic information. The forecasters’ mental workload was evaluated after each scenario using the NASA-Task Load Index (TLX) questionnaire. This study summarizes the analysis results of mental workload experienced by forecasters while using the PHI prototype. Six subdimensions of mental workload: mental demand, physical demand, temporal demand, performance, effort, and frustration were analyzed to derive top contributing factors to workload. Average mental workload was 46.6 (out of 100, standard deviation: 19, range 70.8). Top contributing factors to workload included using automated guidance, PHI object quantity, multiple displays, and formulating probabilities in the new paradigm. Automated guidance provided support to forecasters in maintaining situational awareness and managing increased quantities of threats. The results of this study provided understanding of forecasters’ mental workload and task strategies and developed insights to improve usability of the PHI prototype tool.