DTR: A Novel Topic Generate Algorithm Based on Dbscan and TextRank

Abstract

With the advent of the era of big data, information overload has become a universal problem. How to extract valuable information from the complicated data, and let users quickly understand the main content of these data has become the focus of research. Current topic generation is a way to solve this problem. Some unsupervised topic models generate results for the probability distribution of topics and words, which are less explanatory to the topic, unable to generate subject matter that can be read smoothly. Supervised topic classification methods rely too much on tagged training data and are not universally applicable. This paper proposes a novel topic generation algorithm DTR, which is based on Dbscan and TextRank. By combining text clustering with abstract algorithm, it can automatically generate refined and understandable document topics. At the same time, the similarity analysis method is used to quickly process the document, which narrows the data range of the generated topic and improves the accuracy of the generated topic. The effectiveness of this algorithm is verified by experiments based on a large amount of real data.