Toward a DataOps Framework for Enhancing Data Quality in Data Science

Abstract

Data Science (DS) leverages Data Analytics to help organizations extract value from large datasets and enhance performance. A significant focus in DS is on data collection, cleaning, and transformation to create high-quality datasets for analysis. However, traditional manual data preparation methods are often inefficient and error-prone, particularly in Big Data environments. DataOps seeks to automate data lifecycle stages by integrating DevOps practices, enhancing the quality and reliability of data pipelines. This paper proposes a framework for implementing DataOps, demonstrated through a case study on urban mobility analytics.