Designing a data mining process for the financial services domain

Abstract

The implementation of data mining projects in complex organisations requires well-defined processes. Standard data mining processes, such as CRISP-DM, have gained broad adoption over the past two decades. However, numerous studies demonstrated that organisations often do not apply CRISP-DM and related processes as-is, but rather adapt them to address industry-specific requirements. Accordingly, a number of sector-specific adaptations of standard data mining processes have been proposed. So far, however, no such adaptation has been suggested for the financial services sector. This paper addresses the gap by designing and evaluating a Financial Industry Process for Data Mining (FIN-DM). FIN-DM adapts and extends CRISP-DM to address regulatory compliance, governance, and risk management requirements inherent in the financial sector, and to embed quality assurance as an integral part of the data mining project life-cycle. The framework has been iteratively designed and validated with data mining and IT experts in a financial services organisation.