A Bayesian network based approach for data classification using structural learning

Abstract

This paper describes the theory and implementation of Bayesian networks in the context of data classification. Bayesian networks provide a very general and yet effective graphical language for factoring joint probability distributions which in turn make them very popular for classification. Finding the optimal structure of Bayesian networks from data has been shown to be NP-hard. In this paper score-based algorithms such as K2, Hill Climbing, Iterative Hill Climbing and simulated annealing have been developed to provide more efficient structure learning through more investigation on MDL, BIC and AIC scores borrowed from information theory. Our experimental results show that the BIC score is the best one though it is very time consuming. Bayesian naive classifier is the simplest Bayesian network with known structure for data classification. For the purpose of comparison, we considered several cases and applied general Bayesian networks along with this classifier to these cases. The simulation results approved that using structural learning in order to find Bayesian networks structure improves the classification accuracy. Indeed it was shown that the Iterative Hill Climbing is the most appropriate search algorithm and K2 is the simplest one with the least time complexity. © 2008 Springer-Verlag.