Practices and barriers for big data projects

Abstract

The adoption of big data analytics is increasing in every major industry, demanding investments in new projects, technologies, architectures, and processes to allow the integration of big data platforms with legacy systems; however, many organizations have failed to incorporate it effectively into their decision-making processes and project benefits have not been adequately captured. This study aims to further investigate how a big data analytics project can be implemented in insurance companies. A case study was conducted on one of the largest insurance companies in Brazil with interviews and document analysis. The study identified five main practices that were adopted to successfully implement a big data analytics project (implement automatic autoscaling alerts, use specialized big data tools, integrate the platform with legacy systems, comply with privacy legislation, and ensure the documentation of technical architecture using business process modeling), as well as four barriers that prevent its proper adoption (complexity of access to multicloud data sources, high processing requirements of unstructured data analysis, failure to attend to business necessities at the right time, and project delays brought by bureaucratic interdepartmental processes); some of these have not previously been identified. Finally, an action plan to address these issues is presented.