Spectral Super-Resolution for Greenhouse Gas Detection

Abstract

Hyperspectral imaging offers significant potential for greenhouse gas (GHG) monitoring due to its ability to capture detailed spectral information across broad wavelength ranges. This coverage includes key wavelength regions of the absorption spectra to detect gases such as carbon dioxide (CO2), methane (CH4), and nitrogen dioxide (NO2). Today, the spatial and temporal coverage of hyperspectral missions such as EnMAP, PRISMA, and EMIT is limited. In contrast, multispectral remote sensing imagery generated by Sentinel-2 is available with global coverage on a weekly basis, but lacks the spectral granularity and specific coverage in the critical absorption regions of these gases. Our research overcomes these limitations by reconstructing hyperspectral data from multispectral inputs. We developed and pre-trained a spectral transformer model to capture spectral dependencies and subsequently fine-tuned it on multispectral data for spectral reconstruction. The model is based on a self-supervised masked autoencoder and pre-trained with the majority of spectral channels randomly masked. To achieve this, each channel is individually tokenized, and a wavelength-based positional embedding is added. The pre-training task is to reconstruct the masked channels from the unmasked input, simulating the downstream application of spectral super-resolution. During pre-training, we either apply spatial-spectral attention between all tokens or spectral attention only. The later apporach reduces the computational cost, which for self-attention scales quatratic with the token count. We used a large-scale hyperspectral dataset of over 538k EnMAP samples for pre-training and constructed a dataset of 11k aligned Sentinel-2 and EnMAP images for the fine-tuning. During the fine-tuning, the model performs spectral super-resolution and predicts hyperspectral data from the Sentinel-2 input. The study evaluates the utility of the reconstructed hyperspectral data in two GHG downstream tasks, comparing its performance against original hyperspectral and multispectral data. For these tasks, we aligned the EnMAP-Sentinel-2 dataset with CH4 and NO2 mesurements from Sentinel-5P. The results indicate that the reconstructed data improves GHG prediction accuracy compared to models based on only multispectral data, suggesting its potential to bridge the trade-off between spectral resolution and spatio-temporal coverage. This approach offers insights into leveraging the complementary strengths of hyperspectral and multispectral imaging systems for enhanced atmospheric monitoring.