Automatic keyphrase extraction with a refined candidate set

Abstract

In this paper, we develop and evaluate an automatic keyphrase extraction technique for scientific documents. A new candidate phrase generation method is proposed based on the core word expansion algorithm, which can reduce the size of candidate set by about 75% without increasing the computational complexity. Then in the step of feature calculation, when a phrase and its sub-phrases coexist as candidates, an inverse document frequency related feature is introduced for selecting the proper granularity. Experimental results show the efficiency and effectiveness of the refined candidate set and demonstrate that the overall performance of our system compares favorably with other known keyphrase extraction systems. © 2009 IEEE.