DLR-Logo -> http://www.dlr.de
DLR Portal Home | Imprint | Privacy Policy | Contact | Deutsch
Fontsize: [-] Text [+]

Improving Data Locality in Distributed Processing of Multi-Channel Remote Sensing Data with Potentially Large Stencils

Posovszky, Philipp (2020) Improving Data Locality in Distributed Processing of Multi-Channel Remote Sensing Data with Potentially Large Stencils. Master's, Ludwig-Maximilians-Universität München.

[img] PDF


Distributing a multi-channel remote sensing data processing with potentially large stencils is a difficult challenge. The goal of this master thesis was to evaluate and investigate the performance impacts of such a processing on a distributed system and if it is possible to improve the total execution time by exploiting data locality or memory alignments. The thesis also gives a brief overview of the actual state of the art in remote sensing distributed data processing and points out why distributed computing will become more important for it in the future. For the experimental part of this thesis an application to process huge arrays on a distributed system was implemented with DASH, a C++ Template Library for Distributed Data Structures with Support for Hierarchical Locality for High Performance Computing and Data-Driven Science. On the basis of the first results an optimization model was developed which has the goal to reduce network traffic while initializing a distributed data structure and executing computations on it with potentially large stencils. Furthermore, a software to estimate the memory layouts with the least network communication cost for a given multi-channel remote sensing data processing workflow was implemented. The results of this optimization were executed and evaluated afterwards. The results show that it is possible to improve the initialization speed of a large image by considering the brick locality by 25%. The optimization model also generate valid decisions for the initialization of the PGAS memory layouts. However, for a real implementation the optimization model has to be modified to reflect implementation-dependent sources of overhead. This thesis presented some approaches towards solving challenges of the distributed computing world that can be used for real-world remote sensing imaging applications and contributed towards solving the challenges of the modern Big Data world for future scientific data exploitation.

Item URL in elib:https://elib.dlr.de/134649/
Document Type:Thesis (Master's)
Title:Improving Data Locality in Distributed Processing of Multi-Channel Remote Sensing Data with Potentially Large Stencils
AuthorsInstitution or Email of AuthorsAuthor's ORCID iDORCID Put Code
Posovszky, PhilippUNSPECIFIEDhttps://orcid.org/0000-0003-0656-3691UNSPECIFIED
Date:February 2020
Refereed publication:Yes
Open Access:Yes
Gold Open Access:No
In ISI Web of Science:No
Number of Pages:95
Keywords:Data Distribution; HPC; HPDA; SAR; Big Data;
Institution:Ludwig-Maximilians-Universität München
HGF - Research field:Aeronautics, Space and Transport
HGF - Program:Space
HGF - Program Themes:Earth Observation
DLR - Research area:Raumfahrt
DLR - Program:R EO - Earth Observation
DLR - Research theme (Project):R - Aircraft SAR
Location: Oberpfaffenhofen
Institutes and Institutions:Microwaves and Radar Institute > Spaceborne SAR Systems
Deposited By: Posovszky, Philipp
Deposited On:15 Apr 2020 07:56
Last Modified:15 Apr 2020 07:56

Repository Staff Only: item control page

Help & Contact
electronic library is running on EPrints 3.3.12
Website and database design: Copyright © German Aerospace Center (DLR). All rights reserved.