Evaluation and NLP

Abstract

F-measure is an indicator used since 25 years to evaluate classification algorithms in textmining, from precision and recall. For classification and information retrieval, some ones prefer to use the break even point. Nevertheless, these measures have some inconvenient: they use a binary logic and don't allow applying a user (judge) assessment. This paper proposes a new approach of evaluation. First, we distinguish classification and categorization from a semantic point of view. Then, we introduce a new measure: the K-measure, which is an overall of F-measure and break even point, and allows applying user requirements. Finally, we propose a methodology for evaluation. © Springer-Verlag Berlin Heidelberg 2005.