Assessing the Quality of Computational Notebooks for a Frictionless Transition from Exploration to Production
This addresses the gap between data scientists and software engineers in AI projects, but it is incremental as it builds on existing software engineering solutions.
The paper tackles the challenge of transitioning from exploratory data science to production-ready AI systems by studying best practices for collaboration with computational notebooks and proposing proof-of-concept tools to ensure guideline compliance.
The massive trend of integrating data-driven AI capabilities into traditional software systems is rising new intriguing challenges. One of such challenges is achieving a smooth transition from the explorative phase of Machine Learning projects - in which data scientists build prototypical models in the lab - to their production phase - in which software engineers translate prototypes into production-ready AI components. To narrow down the gap between these two phases, tools and practices adopted by data scientists might be improved by incorporating consolidated software engineering solutions. In particular, computational notebooks have a prominent role in determining the quality of data science prototypes. In my research project, I address this challenge by studying the best practices for collaboration with computational notebooks and proposing proof-of-concept tools to foster guidelines compliance.