Building a Reproducible Machine Learning Pipeline
This addresses reproducibility issues for machine learning practitioners in industry and academia, though it is incremental as it builds on existing concepts of modular pipelines.
The paper tackles the problem of model reproducibility in machine learning by developing a framework with four components (data, feature, scoring, and evaluation layers) to enable exact replication and reuse of transformations, resulting in dramatically increased speed for offline and online experimentation.
Reproducibility of modeling is a problem that exists for any machine learning practitioner, whether in industry or academia. The consequences of an irreproducible model can include significant financial costs, lost time, and even loss of personal reputation (if results prove unable to be replicated). This paper will first discuss the problems we have encountered while building a variety of machine learning models, and subsequently describe the framework we built to tackle the problem of model reproducibility. The framework is comprised of four main components (data, feature, scoring, and evaluation layers), which are themselves comprised of well defined transformations. This enables us to not only exactly replicate a model, but also to reuse the transformations across different models. As a result, the platform has dramatically increased the speed of both offline and online experimentation while also ensuring model reproducibility.