Exploiting A Synthesis-based Representation Learning Pre-Training Task To Repair Hyperspectral Satellite Image Cubes

Kurzfassung

Challenge

Data acquired by remote sensing platforms might be affected by different types of pixel defects, due to aging, degradation of electronic components, mechanical vibrations and data transmission failures. These can produce from missing to low quality data i.e. low- and high-gain linearity effects, non-uniformity effects (photo-response non-uniformity [PRNU], dark-signal non-uniformity [DSNU]) as well as low and high radiance values outside of the allowed dynamic range. Taking a look at the processing of acquired radiance measurements of spaceborne sensors, transformer models exhibit state-of-the-art results regarding a multitude of applications. A key factor on the performance of this kind of deep learning architectures is how they extract the meaningful features of the input data in terms of representations generated by an internal encoder. Therefore, healthy data and well fit internal representations of it, are essential for advanced transformer-based downstream tasks, i.e. classification and surface reflectance retrieval, that generalize well, independent of one particular sensors configuration. This contribution presents a combined approach to tackle both challenges, in which a hyperspectral sequence transformer encoder is trained to learn well generalizing representations on synthetic data while being able to reconstruct real-world hyper- and multi-spectral satellite measurements from DESIS, EnMAP and Sentinel-2 respectively.

Methodology

A representation pre-training task is presented that aims to solve an un-supervised reconstruction problem by fitting a customized transformer architecture and setup, utilizing spectral positional encoding, learnable-query spectrum representation reprojection, input- independent, shallow decoding and advanced cross-reconstruction loss that enable the network to focus on the representation learning while being able to reconstruct input spectra of a very high spectral resolution and therefore single-sample sequence lengths. The learning of well generalizing encoder representations is evaluated by applying different corruptions to full and partial, unseen synthetic input spectra as well as real-world acquisitions from aforementioned mission and of varying sensor configuration, that were not used during training. Study area and data The underlying training data is an extensive dataset of radiative transfer calculations for a wide variety of different atmospheric states, which were chosen using advanced sampling techniques i.e. Halton samples to account for the high dimensionality of the feature space as well as and linear combinations of the USGS Spectral Library Version 7, which were performed by the 6SV radiative transfer code. Evaluation satellite data used in the study was acquired in the course of the DLR DESIS and EnMAP hyperspectral missions as well as the multispectral ESA Sentinel-2 mission over several regions in Europe exhibiting different types of surface reflectance and observation conditions.

Discussion and Conclusions

It is shown that the presented approach successfully trains a light-weight and extensible transformer encoder that is able to generate fixed-size mutual internal representations independent from the spectral configuration of the sensor the input spectra are based on. An outlook is given on how those can be introduced to complex downstream tasks like atmospheric correction of top-of-atmosphere radiance measurements from different satellites. Further, it is discussed how the shallow decoders capability to repair corrupted hyperspectral satellite image cubes can be exploited to provide constant and continuous input data to processing chains of operational DLR satellite missions.