Beyond the hype: Why do data-driven projects fail?

Abstract

Despite substantial investments, data science has failed to deliver significant business value in many companies. So far, the reasons for this problem have not been explored systematically. This study tries to find possible explanations for this shortcoming and analyses the specific challenges in data-driven projects. To identify the reasons that make data-driven projects fall short of expectations, multiple rounds of qualitative semi-structured interviews with domain experts with different roles in data-driven projects were carried out. This was followed by a questionnaire surveying 112 experts with experience in data projects from eleven industries. Our results show that the main reasons for failure in data-driven projects are (1) the lack of understanding of the business context and user needs, (2) low data quality, and (3) data access problems. It is interesting, that 54% of respondents see a conceptual gap between business strategies and the implementation of analytics solutions. Based on our results, we give recommendations for how to overcome this conceptual distance and carrying out data-driven projects more successfully in the future.