Real-time plotting and evaluation of the data quality control from the CSIR-NGRI magnetic observatories

Abstract

Earth's magnetic field, a dynamic shield influenced by internal and external forces, holds critical insights into space weather forecasting and the planet's core dynamics. The Choutuppal (CPL) and Hyderabad (HYB) magnetic observatories in India are pioneering this field by delivering high-resolution geomagnetic data to INTERMAGNET with unprecedented speed and precision. Utilizing a novel, low-cost protocol, CPL transmits 1 s resolution data and HYB provides 1 min data, both achieving a latency of less than 300 s, making CPL one of the first Indian observatories to send 1 s real-time data to GIN (Geomagnetic Information Node). This rapid data transmission enhances global collaboration in space weather prediction, safeguarding critical infrastructure like satellites and power grids from solar storms. To further elevate data utility, we developed Python-based software for real-time visualization and quality control at both observatories. This tool generates plots, performs initial quality checks, and computes first differences at 1 s and 1 min intervals, with a latency under 300 s. By enabling daily evaluation of data quality, the software facilitates the identification of anomalies and noise, supporting the preparation of quasi-definitive data essential for geomagnetic research. Our Python server and web applications are designed with the future in mind, integrating artificial intelligence (AI) and machine learning (ML) capabilities. These advancements at CPL and HYB are set to transform the processing, forecasting, and visualization of geomagnetic data. By improving both the accuracy and accessibility of these data, we aim to revolutionize geomagnetic research, making it more precise, accessible, and actionable.