Lossless compression of ultraspectral sounder data

Abstract

In the era of contemporary and future ultraspectral sounders such as Atmospheric Infrared Sounder (AIRS) [7], Cross-track Infrared Sounder (CrIS) [9], Infrared Atmospheric Sounding Interferometer (lASI) [40], Geosynchronous Imaging Fourier Transform Spectrometer (GIFTS) [52], and Hyperspectral Environmental Suite (HES) [26], better inference of atmospheric, cloud and surface parameters is feasible. An ultraspectral sounder generates an unprecedented amount of three-dimensional (3D) data, consisting of two spatial and one spectral dimension. For example, the HES is the next-generation NOAA/NESDIS Geostationary Operational Environmental Satellite (GOES) sounder, slated for launch in the 2013 timeframe. It would be either a Michelson interferometer or a grating spectrometer, with high spectral resolution (over one thousand infrared channels with spectral widths on the order of 0.5 wavenumber), high temporal resolution (better than 1 hour), high spatial resolution (less than 10km) and hemispheric coverage. Given the large volume of 3D data that will be generated by an ultraspectral sounder each day, the use of robust data compression techniques will be beneficial to data transfer and archive. There exist differences between ultraspectral sounder data and hyperspectral imager data in terms of application areas and subsequent user constraints on the data compression. The hyperspectral imager data (e.g. the well-known AVIRIS data [6, 43]) has hundreds of bands in the visible or near-infrared regions with major application categories of anomaly detection, target recognition and background characterization [50]. Lossy compression is usually acceptable for imager data as long as the tolerance limits in application-specific metrics are met [46]. These metrics include those that signify scientific loss for end users [42, 45], content-independent metrics [51], and even visual comparisons [18]. On the other hand, the ultraspectral sounder data has over a thousand channels in the infrared region with the main purpose of retrieving atmospheric temperature, moisture and trace gases profiles, surface temperature and emissivity, and cloud and aerosol optical properties for better weather and climate prediction. The physical retrieval of these geophysical parameters from the sounder data via the inverse solution of the radiative transfer equation is a mathematically illposed problem [25], which is sensitive to the data quality. Therefore there is a need for lossless or near-lossless compression of ultraspectral sounder data to avoid potential retrieval degradation of geophysical parameters due to lossy compression. Here, near-lossless compression implies that the error spectrum between the reconstructed data set and original data set is significantly less than the sensor noise spectrum. This chapter explores lossless compression of ultraspectral sounder data. These investigations are divided into transform-based, prediction-based, and clustering-based methods. The ultraspectral sounder data features strong correlations in disjoint spectral regions affected by the same type of absorbing gases at various altitudes. To take advantage of this fact, a biasadjusted reordering (BAR) data preprocessing scheme [29] is devised that is applicable to any 2D compression method. We begin with a description of the ultraspectral sounder data used.