New measure of Boolean factor analysis quality

Abstract

Learning of objects from complex patterns is a long-term challenge in philosophy, neuroscience, machine learning, data mining, and in statistics. There are some approaches in literature trying to solve this difficult task consisting in discovering hidden structure of high-dimensional binary data and one of them is Boolean factor analysis. However there is no expert independent measure for evaluating this method in terms of the quality of solutions obtained, when analyzing unknown data. Here we propose information gain, model-based measure of the rate of success of individual methods. This measure presupposes that observed signals arise as Boolean superposition of base signals with noise. For the case whereby a method does not provide parameters necessary for information gain calculation we introduce the procedure for their estimation. Using an extended version of the "Bars Problem" generation of typical synthetics data for such a task, we show that our measure is sensitive to all types of data model parameters and attains its maximum, when best fit is achieved. © 2011 Springer-Verlag.