PAKE: a supervised approach for Persian automatic keyword extraction using statistical features

Abstract

Keywords are a collection of important words in a document that are the core topic of the discussion. This paper proposes a hybrid method for automatically extracting keywords from Persian documents and web pages. In the proposed method, firstly, based on linguistic knowledge, processing was performed at word and letter levels to optimize of the analysis. Then a new statistical features set is defined and extracted at the word level. At the final stage, keywords are determined using the SVM algorithm. Also, in this paper, due to the lack of a corpus for evaluating the methods of automatic extraction of Persian keywords, a large-scale corpus has been developed and introduced. The achieved F-measure for keywords and non-keywords are 99.89% and 99.99% respectively.