On morphological relatedness

Abstract

In this paper, we discuss the results of a new unsupervised and computationally lightweight scoring of how two words are morphologically related to each other. This measure is meant to be an alternative to stemming, radicals (root) extraction, and morphological analysis in a wide range of applications; especially information extraction related ones. Compared to light stemming, which seems to be the most convenient approach for systems with efficiency concerns, our measure does not neglect unconditionally a prefix or a suffix as the light stemming does. Instead, our measure takes into account all letters of the word but with different weights. This prevents the missing of a significant letter. Compared to heavy stemming, morphological analysis, or radicals extraction, which rely on dictionaries and compatibility databases, our measure does not rely on any language-specific morphology knowledge. This makes our approach unsupervised and theoretically language independent and computationally much lighter. Our tests targeted Arabic: a Semitic language recognized to have a complex morphology due to its highly inflectional lexicon. Copyright © 2012 Cambridge University Press.