Adaptive learning for lemmatization in morphology analysis

Abstract

Morphological analysis is used to study the internal structure words by reducing the number of vocabularies used while retaining the semantic meaning of the knowledge in NLP system. Most of the existing algorithms are focusing on stemmatization instead of lemmatization process. Even with technology advancement, yet none of the available lemmatization algorithms able to produce 100% accurate result. The base words produced by the current algorithm might be unusable as it alters the overall meaning it tried to represent, which will directly affect the outcome of NLP systems. This paper proposed a new method to handle lemmatization process during the morphological analysis. The method consists three layers of lemmatization process, which incorporatethe used of Stanford parser API, WordNet database and adaptive learning technique. The lemmatized words yields from the proposed method are more accurate, thus it will improve the semantic knowledge represented and stored in the knowledge base.