Validating Goal-Oriented Hypotheses of Business Problems Using Machine Learning: An Exploratory Study of Customer Churn

Abstract

Organizations are investing in Big Data and Machine Learning (ML) projects, but most of these projects are predicted to fail. A study shows that one of the biggest obstacles is the lack of understanding of how to use data analytics to improve business value. This paper presents Metis, a method for ensuring that business goals and the corresponding business problems are explicitly traceable to ML projects and where potential (i.e., hypothesized) complex problems can be properly validated before investing in costly solutions. Using this method, business goals are captured to provide context for hypothesizing business problems, which can be further refined into more detailed problems to identify features of data that are suitable for ML. A Supervised ML algorithm is then used to generate a prediction model that captures the underlying patterns and insights about the business problems in the data. An ML Explainability model is used to extract from the prediction model the individual features and their degree of contribution to each problem. The extracted weighted data feature are then fed back to the goal-oriented problem model to validate the most important business problems. Our experiment results show that Metis can detect the most influential problem when it was not apparent through data analysis. Metis is illustrated using a real-world customer churn (customer attrition) problem for a bank and a publicly available customer churn dataset.