Outliers detection in regressions by nonparametric parzen kernel estimation

Abstract

A certain observation which is unusual or different from all other ones is called the outlier or anomaly. Appropriate evaluation of data is a crucial problem in modelling of the real objects or phenomena. Actually investigated problems often are based on data mass-produced by computer systems, without careful inspection or screening. The great amount of generated and processed information (e.g. so-called Big-Data) cause that possible outliers often go unnoticed and the result is that they can be masked. However, in regression, this situation can be more complicated. The identification and evaluation of the extremely atypical measurements in observations, for instance in some areas of medicine, geology, particularly in seismology (earthquakes), is precisely the outliers that are the subjects of interest. In this paper, a nonparametric procedure based on Parzen kernel for estimation of unknown function is applied. Evaluation of which measurements in input data-set could be recognized as outliers and possibly should be removed has been performed using the Cook’s Distance formula. Anomaly detection is still an important problem to be researched within diverse areas and application domains.