Understanding Implementation Challenges in Machine Learning Documentation

Abstract

The lack of transparency in machine learning (ML) systems makes it difficult to identify sources of potential risks and harms. In recent years, various organizations have proposed standardized frameworks and processes for documentation for ML systems. However, it remains unclear how practitioners should implement and operationalize ML documentation in their workflows. We conducted semi-structured interviews with 24 practitioners in various organizational contexts to identify key implementation challenges and strategies for alleviating these challenges. Our findings indicated that addressing the why, how, and what of documentation is critical for implementing robust documentation practices.