An improved text categorization methodology based on second and third order probabilistic feature extraction and neural network classifiers

Abstract

In this paper we deal with the main aspect of the problem of extracting meaning from documents, namely, with the problem of text categorization, outlining a novel and systematic approach to its solution. We present a text categorization system for non-domain specific full-text documents. The main contribution of this paper lies on the feature extraction methodology which, first, involves word semantic categories and not raw words as other rival approaches. As a consequence of coping with the problem of dimensionality reduction, the proposed approach introduces a novel second and third order feature extraction approach for text categorization by considering word semantic categories cooccurrence analysis. The suggested methodology compares favorably to widely accepted, raw word frequency based techniques in a collection of documents concerning the Dewey Decimal Classification (DDC) system. In these comparisons different Multilayer Perceptrons (MLP) algorithms, the LVQ and the conventional k-NN technique are involved. © Springer-Verlag Berlin Heidelberg 2006.