On producing reliable and Affordable numerical weather forecasts on public cloud-computing infrastructure

Abstract

Cloud-computing resources are increasingly used in atmospheric research and real-time weather forecasting. The aim of this study is to explore new ways to reduce cloud-computing costs for real-time numerical weather prediction (NWP). One way is to compress output files to reduce data egress costs. File compression techniques can reduce data egress costs by over 50%. Data egress costs can be further minimized by postprocessing in the cloud and then exporting the smaller resulting files while discarding the bulk of the raw NWP output. Another way to reduce costs is to use preemptible resources, which are virtual machines (VMs) on the Google Cloud Platform (GCP) that clients can use at an 80% discount (compared to nonpreemptible VMs), but which can be turned off by the GCP without warning. By leveraging the restart functionality in the Weather Research and Forecasting (WRF) Model, preemptible resources can be used to save 60%-70% in weather simulation costs without compromising output reliability. The potential cost savings are demonstrated in forecasts over the Canadian Arctic and in a case study of NWP runs for the West African monsoon (WAM) of 2017. The choice in model physics, VM specification, and use of the aforementioned cost-saving measures enable simulation costs to be low enough such that the cloud can be a viable platform for running short-range ensemble forecasts when compared to the cost of purchasing new computer hardware.