Named Entity Recognition in Marathi Language

Abstract

Named Entity Recognition is the methodology of distinguishing named entities which are present in documents. NER mostly used and applied for information extraction. In this paper, a system comprising two approaches for NER has been proposed for Marathi. Random Marathi text documents are given as an input to the system. These documents are from news domain which consist text related to history, sports, entertainment etc. Input text is processed to create an annotated NER tagged data set by linguistic experts. All the input documents will be trained first and then tested in HMM based approach. In the training phase of HMM the annotated text is tokenized and all the parameters for HMM like transition, emission, etc. are computed. In the testing phase correct NER tag is predicted using Viterbi algorithm. In the Rule Based approach first the input is tokenized and POS tagged. By using the NER tagset tokens in the sentences are assigned correct NER tags. The words which are left to be tagged are then tagged by using handcrafted rules. Finally the system output is NER tag sentences.