Capacity of the Upstart Algorithm

Abstract

The storage capacity of multilayer networks with overlapping receptive fields is investigated for a constIUctive algorithm within a one-step replica symmetry breaking (RSB) treatment. We find that the storage capacity increases logarithmically with the number of hidden units K without saturating the Mitchison-Durbin bound. The slope of the logarithmic increase decays exponentionally with the stability with which the patterns have been stored.