Learning context-free grammars with a simplicity bias

Abstract

We examine the role of simplicity in directing the induction of context-free grammars from sample sentences. We present a rational reconstruction of Wolff’s SNPR - the Grids system - which incorporates a bias toward grammars that minimize description length. The algorithm alternates between merging existing non terminal symbols and creating new symbols, using a beam search to move from complex to simpler grammars. Experiments suggest that this approach can induce accurate grammars and that it scales reasonably to more difficult domains.