Exploring and processing large data sets in earth observation on HPC-systems with Heat

Kurzfassung

Handling and analyzing massive data sets efficiently is particularly important in the field of earth observation. Nevertheless, this can be challenging, especially for researchers and developers without a background in high-performance computing (HPC). The Python library {\itshape Heat} aims at supporting such researchers and developers by providing general-purpose, memory-distributed and hardware-accelerated array manipulation, data analytics, and machine learning algorithms in Python, targeting the usage by non-experts in HPC. This paper show-cases how Heat can help to facilitate exploration and processing of large amounts of data in earth observation on HPC-systems. This is done on behalf of two ongoing applications of Heat in the context of anomaly detection in remote sensing data of German coastal regions.