Analyzing the stemming paradigm

Abstract

This paper discusses affix removal and statistical based Stemming algorithms in detail with stemmer-generated output from some Standard English text and dictionary. Comparative empirical studies of all these stemmers are also discussed here with respect to number of stem token generation from single root morphed word variants and computation time. First part of the paper deals with introductory discussion of stemming and lemmatization. Second part of the paper focuses on algorithms of affix and statistical based stemmers with their empirical output. Last part describes the steps of the comparative tool for the same. Finally conclusion section wraps up whole discussion about stemming. This paper can assist researchers working in the field of text mining.