Designing a counter: Another case study of dynamics and activation landscapes in recurrent networks

Abstract

Inspired by the work of Elman and Wiles [15] and based on the notions developed within the theory of dynamical systems it is shown how a simple recurrent connectionist network with a single hidden unit implements a counter. This result is exemplified by showing that such a network can be used to recognize the context-free language anb n consisting of strings of some number n of a's followed by the same number n of b's for n250, whereby the maximum value for n is only restricted by the computing accuracy of the used hardware.