Benefits of Applying Big-Data Tools for Log-Centralisation in SMEs

Abstract

The benefits of big-data have been proven to ensure more control over the data, adding improvements in security and complex query capabilities across many datasets. However, a problem faced by many companies, especially by small and medium-sized companies (SMEs), is to define when it is necessary to apply big-data tools. Log management becomes a relevant challenge when the volume starts to grow. This paper aims to define the benefits of applying big-data tools to dealing with log-management. In addition, it provides implementation of log-centralisation based on a cluster made up of commodity nodes for medium-volume data environments using big-data technologies. The proposed system is tested on a real study case, in particular on a medium-sized telecommunication company. The results show that the implemented system brings efficiency in storing and analysing medium-volume datasets. Furthermore, the proposed solution scales the performance based on the data size and number of nodes, providing improvements in data security, data analysis and data storage.