Broken symmetries in multilayered perceptrons

Abstract

The statistical mechanics of two-layered perceptrons with N input units, K hidden units, and a single output unit that makes a decision based on a majority rule (Committee Machine) are studied. Two architectures are considered. In the nonoverlapping case the hidden units do not share common inputs. In the fully connected case each hidden unit is connected to the entire input layer. In both cases the network realizes a random dichotomy of P inputs. The statistical properties of the space of solutions as a function of P is studied, using the replica method, and by numerical simulations, in the regime where NK. In the nonoverlapping architecture with continuously varying weights the capacity, defined as the maximal number of P per weight, (c) is calculated under a replica-symmetric (RS) ansatz. At large K, c diverges as K1/2 in contradiction with the rigorous upper bound, c