Improving Maximum Entropy Model by GIS

Abstract

Maximum Entropy Model (MEM) [1] [4] estimates probability distribution functions, by which current state of knowledge is described in the context of prior data. Here we examine Generalized Iterative Scaling (GIS) [1] algorithm to determine optimum feature weights with feature selection during learning. Maximum Entropy principle [1] provides us with all the characteristics of the data given in advance and we could expect robust distribution against outlier. However it takes much time until convergence because the computation depends heavily on the number of classes. We introduce a novel approach random sampling of Monte Carlo method into GIS for improved computation.