Generating competitive intelligence digests with a LDA-based method: A case of BT intellact

Abstract

Internet has transformed the ways that organizations gather, produce and transmit competitive intelligence (CI), especially in the age of big data. This paper introduces a competitive intelligence digest generation method based on LDA topic modelling and representative text extraction. With the incorporated metric of perplexity, the proposed method is capable of automatic grouping of the texts and generating CI digests in an appropriate number of topics. Moreover, the method is applied to the context of BT Plc in the form of a case study, demonstrating its effectiveness in practical use.