Detecting temporal patterns of importance indices about technical phrases

Abstract

In text mining, importance indices of terms such as simple frequency, document frequency including the terms, and tf-idf of the terms, play a key role for finding valuable patterns in documents. As for the documents, they are often published daily, monthly, annually, and irregularly for each purpose. Although the purposes of each set of documents are not changed, roles of terms and the relationship among them in the documents change temporally. In order to detect such temporal changes, we decomposed the process into three sub-processes: automatic term extraction, importance index calculation, and temporal trend detection. On the basis of the consideration, we propose a method for detecting temporal trends of technical terms based on importance indices and clustering methods. By focusing on technical phrases, we carried out an experimentation to detect emergent and subsiding trends in a set of research document. The result shows that our method determined the temporal trends of technical phrases related to finding of patterns for innovations of research topics. © 2009 Springer Berlin Heidelberg.