A characterization of simple recurrent neural networks with two hidden units as a language recognizer

Abstract

We give a necessary condition that a simple recurrent neural network with two sigmoidal hidden units to implement a recognizer of the formal language {a n b n | n > 0 } which is generated by a set of generating rules {S→aSb, S→ab } and show that by setting parameters so as to conform to the condition we get a recognizer of the language. The condition implies instability of learning process reported in previous studies. The condition also implies, contrary to its success in implementing the recognizer, difficulty of getting a recognizer of more complicated languages. © 2008 Springer-Verlag Berlin Heidelberg.