An interactive web-based toolset for knowledge discovery from short text log data

Abstract

Many companies maintain human-written logs to capture data on events such as workplace incidents and equipment failures. However, the sheer volume and unstructured nature of this data prevent it from being utilised for knowledge acquisition. Our web-based prototype software system provides a cohesive computational methodology for analysing and visualising log data that requires minimal human involvement. It features an interface to support customisable, modularised log data processing and knowledge discovery. This enables owners of event-based datasets containing short textual descriptions, such as occupational health & safety officers and machine operators, to identify latent knowledge not previously acquirable without significant time and effort. The software system comprises five distinct stages, corresponding to standard data mining milestones: exploratory analysis, data warehousing, association rule mining, entity clustering, and predictive analysis. To the best of our knowledge, it is the first dedicated system to computationally analyse short text log data and provides a powerful interface that visualises the analytical results and supports human interaction.