Compressed learning based onboard semantic compression for remote sensing platforms

Kurzfassung

Earth observation (EO) plays a crucial role in creating and sustaining a resilient and prosperous society that has far reaching consequences for all life and the planet itself. Satellites are mostly used for EO and the data archive for the Sentinel missions alone exceed 77.52 PB [1] and is increasing. Bottleneck for such high throughput acquisition is the downlink bandwidth. Data-centric solutions to image compression is required to address this deluge. Though compute on remote platforms have increased significantly with the use of commercial-off-the-shelf (COTS) components, however, minimizing resource utilization is still a highly important design criterion. In this work, instead of focusing on the more common encoder-decoder architectures, we study semantic compression through a compressed learning framework [2] that utilizes only fast and sparse matrix-vector multiplication to encode the data. We consider camera noise and a communication channel as the sources of distortion. The complete semantic communication pipeline then consists of a learnt low-complexity compression matrix that acts on the noisy camera output to generate onboard a vector of observations that is downlinked through a communication channel, processed through a partially-unrolled network and then fed to a deep learning model performing the necessary downstream tasks; image classification and semantic segmentation are studied. Distortions are compensated by unrolling few layers of NA-ALISTA [3] with a wavelet sparsity prior. Decoding is thus a plug-n-play approach designed according to the camera/environment information and downstream task. The GLODISMO [4] framework is used for end-to-end training of the pipeline. The deep learning model for the downstream task is jointly fine-tuned with the compression matrix through the loss function. We show that addition of a recovery loss along with the task dependent losses improves the downstream performance in noisy settings. We compare our work with other encoder-decoder based approaches for semantic compression through evaluation on remote sensing datasets. References: [1] https://dashboard.dataspace.copernicus.eu/#/service-insight [2] E. Zisselman, A. Adler and M. Elad, “Compressed learning for image classification: a deep neural network approach,” in Processing, Analyzing and Learning of Images, Shapes, and Forms: Part 1, Elsevier, 2018. [3] F. Behrens, J. Sauder and P. Jung, “Neurally augmented ALISTA”, International conference on learning representations (ICLR), 2021. [4] J. Sauder, M. Genzel and P. Jung, "Gradient-Based Learning of Discrete Structured Measurement Operators for Signal Recovery," in IEEE Journal on Selected Areas in Information Theory, vol. 3, no. 3, pp. 481-492, Sept. 2022.