Evaluation of Provenance Serialisations for Astronomical Provenance

Abstract

Provenance data from astronomical pipelines are instrumental in establishing trust and reproducibility in the data processing. In addition, astronomers may also query their provenance to answer questions routed in areas such as anomaly detection, recommendation, and prediction. In order to determine how astronomers should store and query their provenance data, this paper reports on a comparison between the triple store Apache Jena Fuseki and the graph database system Neo4j for this purpose. The metrics measured for comparison were the accuracy and timing of the queries as well as the data upload times. The provenance data used was simulated data that was derived from the provenance describing two astronomical pipelines. It was found that both implementations are competent for this purpose, however Fuseki was more efficient at storing and uploading the data as well as querying over small datasets (<2.5MB). However, Neo4j was more efficient for queries across larger provenance datasets.