Assessing classification accuracy in the revision stage of a CBR spam filtering system

Abstract

In this paper we introduce a quality metric for characterizing the solutions generated by a successful CBR spam filtering system called SPAMHUNTING. The proposal is denoted as relevant information amount rate and it is based on combining estimations about relevance and amount of information recovered during the retrieve stage of a CBR system. The results obtained from experimentation show how this measure can successfully be used as a suitable complement for the classifications computed by our SPAMHUNTING system. In order to evaluate the performance of the quality estimation index, we have designed a formal benchmark procedure that can be used to evaluate any accuracy metric. Finally, following the designed test procedure, we show the behaviour of the proposed measure using two well-known publicly available corpus. © Springer-Verlag Berlin Heidelberg 2007.