A*-Reduct: A Heuristic Rough Set Based Feature Selection Algorithm and Its Application to Text Summarization

Abstract

With the emergence Big data scenario selection of features has assumed a very important role in today’s data processing world. Handling a large volume of data, irrespective of the underlying domain (E.g. spam mails, text data, fraud transactions, speaker identification), and the content (e.g. gene data, text data, image data, audio-video data) is always computationally challenging for both time and cost. A very important step towards efficient handling of the data comes by reducing the volume of the data through selection of appropriate features, and thereby reducing its dimension. Moreover, inferring from large dimensional data often proves to be difficult in terms of time vs. accuracy; since with the increase in dimensionality the amount of noise content also increases. As a consequence, the need for an efficient feature selection scheme is more pronounced these days in order to eliminate the redundant features (i.e. that do not play any important role in data classification); and then to select most important part of data that is useful for classification and/or further processing. The basic assumption here is that the entire data knowledge is represented by the original feature set. The selected subset of features should be such that its classification ability is of high accuracy, i.e. the classificatory property of the reduced data set should be (almost) same as the original set of features. In this paper we present an Artificial Intelligence technique for feature selection that use two popular techniques namely (i) AI based Heuristic Search and (ii) Rough Sets. Further, we have applied the technique on the task of supervised text summarization which involves high dimensional representation. The results obtained are motivating.