BigDataCube: Making Big Data a Commodity

Abstract

The Big DataCube project aims at advancing the innovative datacube paradigm - i.e ., analysis - ready spatio - temporal raster data - from the level of a scientific prototype to precommercial Earth Observation (EO) services. To this end, the European Datacube Engine (in database lingo: ‖Array Database System ‖), rasdaman, will be installed on the public German Copernicus hub, CODE - DE , as well as in a comme rc ial c loud environment to exe mp larily offer analyt ics services and to federate both, thereby demonstrating an integrated public/private service . Started in January 2018 with a runtime of 18 months, Big DataCube will co mple ment the batch - orien ted Hadoop service already available on CODE - DE with rasdaman thereby offering important additional functionality, in particular a paradig m of -any query, any t ime, on any size‖, strictly based on open geo standards and federated with other data centers. On this platform novel, specialized services can be established by third parties in a fast, fle xible, and scalable manner. To this end, several features crucial for operational services need to be tested and/or imple mented, such as securing access (in parti cular in a distributed processing context), tuning to the specific cloud configuration of CODE - DE, and further items to be determined in the initial requirements analysis phase. The result will be the prototype of a federation of rasdaman installations on CODE-DE, cloudeo, as we ll as further (external) data centers; further, best practices on the use of array databases in operational environments will be established. This will pave the way for individual value - adding services by third parties.