Building Reproducible ML Models

This talk draws from software engineering principles to introduce an end-to-end data science workflow that makes it easy to reproduce given results. In a 2016 study conducted by Nature [1], it was found that more than 70% of researchers surveyed were unable to reproduce research published by others, and more than 50% were unable to reproduce their own results. In general, the lack of reproducibility in research holds back other groups from building on existing results, holding back overall progress. Even in industry, it is often extremely hard for data scientists to share analysis methods, and to build on the work done by others in the company. In general, there is a belief that reproducibility is achieved by sharing raw code and data, and while this is a step in the right direction, there are often confounding environmental variables, especially in more complex models and analyses, that still make the results inconsistent. In this talk, I will provide a workflow rooted in Git and Jupyter that makes it easy to share research results, both in academia as well as within a company. I will also share some general principles and tips derived from software engineering that will allow anyone to tailor their work techniques to make their analysis more consistent and sharable. [1] https://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970