A feature selection algorithm for anomaly detection in grid environment using k-fold cross validation technique

Abstract

An Intrusion Detection System (IDS) seeks to identify unauthorized access to computer systems’ resources and data. The spreading of a data set size, in number of records as well as of attributes, as trigger the development of a number of big data platforms as well as parallel data analysis algorithms. This paper proposed a state-of-the-art technique to reduce the number of input features in dataset by using the Sequential Forward Selection (SFS) with k-Fold Cross Validation Model. Before reaching the feature reduction stage, the pre-processing analysis for detecting unusual observations that do not seem to belong to the pattern of variability produced by the other observations. The pre-processing analysis consists of outlier’s detection and Transformation. Outliers are best detected visually whenever this is possible. This paper explains the steps for detecting outliers’ data and describes the transformation method that transforms them to normality. The transformation obtained by maximizing Lamda functions usually improves the approximation to normality.