Managing Randomness to Enable Reproducible Machine Learning

Abstract

The National Information Standards Organization defines scientific reproducibility as "obtaining consistent results using the same input data, computational steps, methods, and code, and conditions of analysis'' [12] reproducibility. Reproducibility in machine learning (ML) refers to the ability to regenerate an ML model precisely guaranteeing identical accuracy and transparency. While a model may offer reproducible inference, reproducing the model itself is frequently problematic at best due to the presence of pseudo-random numbers as part of the model generation. One way to ensure that models are trustworthy is by managing the random numbers produced during model training. This paper establishes examples of the impact of randomness in model generation and offers a preliminary investigation into how random number generation can be controlled to make ML models more reproducible.