Weighted parsing for grammar-based language models

Abstract

We develop a general framework for weighted parsing which is built on top of grammarbased language models and employs flexible weight algebras. It generalizes previous work in that area (semiring parsing, weighted deductive parsing) and also covers applications outside the classical scope of parsing, e.g., algebraic dynamic programming. We show an algorithm which terminates and is correct for a large class of weighted grammar-based language models.