Comprehensive Potentiality Maximization to Improve and Interpret Multi-Layered Neural Networks

Abstract

The present paper aims to propose a new method to maximize the potentiality of components to deal with new inputs in multi-layered neural networks. This means that the implicit ability of components is supposed to be maximized, namely, comprehensive potentiality maximization. The paper in particular states that learning in neural networks should focus not on specific components for specific inputs but on all available components to cope with as many different inputs as possible. In terms of learning for specific inputs, the main objective of learning lies in learning as little as possible. However, for the first approximation, we deal here with two types of potentiality among many to be maximized, defined for connection weights and hidden layers, namely, inner and outer potentiality. Under the comprehensive potentiality maximization, the inner and outer potentiality should be as large as possible. When the potentiality cannot be easily increased, it is forced to be increased by making the strength of weights larger excessively and repeatedly. We applied the method to the bankruptcy data set. The results show that by forcing networks to have larger inner and outer potentiality, generalization performance was improved. In addition, by the comprehensive interpretation, taking into account as many representations as possible and treating them as equally as possible, we could extract new important inputs.