Traceability and Reproducibility of Big Data Analytics Workflows using Provenance

Abstract

The provenance of data provides detailed information about the origin of that data. That includes information about ownership and both actions and modifications performed on the data. With provenance information, data will be traceable and reproducible. In data science, results that are not reproducible by peer scientists are valueless and of no significance. In engineering, users can be more confident in the quality of products that ware developed based on simulations and data analytics workflows. To specify and store provenance information, W3C has standardized the provenance model PROV. Using PROV and associated implementations, users can record provenance of data analytics processes. The provenance information are directed acyclic graphs that can be analyzed to get insight into the data analytics processes. The talk describes the architecture of provenance management and how to apply provenance recording and provenance analytics to data science and big data analytics workflows.