Density ratio estimation: A new versatile tool for machine learning

Abstract

A new general framework of statistical data processing based on the ratio of probability densities has been proposed recently and gathers a great deal of attention in the machine learning and data mining communities [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17]. This density ratio framework includes various statistical data processing tasks such as non-stationarity adaptation [18,1,2,4,13], outlier detection [19,20,21,6], and conditional density estimation [22,23,24,15]. Furthermore, mutual information-which plays a central role in information theory [25]-can also be estimated via density ratio estimation. Since mutual information is a measure of statistical independence between random variables [26,27,28], density ratio estimation can be used also for variable selection [29,7,11], dimensionality reduction [30,16], and independent component analysis [31,12]. © 2009 Springer-Verlag Berlin Heidelberg.