Spatial datafusion with Apache Airflow

Abstract

In this workshop we want to show the possibilities to use Apache Airflow as a tool to create modular, reusable pipeline for complex spatial data fusion. Apache Airflow is an open source workflow management platform which is designed to execute tasks in a predefined matter and ensures that it order as well as timing is corrects. This is done by creating a directed acyclic graph of all operations and thereby creating a pipeline. The additional benefit is that Apache Airflow uses different plugins and can use other tools and languages such as PostGIS operations or Python code. In this workshop we present an overview of the capabilities and introduce it more in-depth with code and data examples from the AIFER research project - combining earth observation, social media data as well as additional data sources. The participants can then modify this pipeline to gain experience with the tool as well as ask questions. The target audience are more technically inclined members of the GI community with some basic experience with Python as well as databases.