Efficient parallel and incremental parsing of practical context-free languages

Abstract

We present a divide-and-conquer algorithm for parsing context-free languages efficiently. Our algorithm is an instance of Valiant's (1975; General context-free recognition in less than cubic time. J. Comput. Syst. Sci. 10(2), 308-314), who reduced the problem of parsing to matrix multiplications. We show that, while the conquer step of Valiant's is O(n 3), it improves to O(log2 n) under certain conditions satisfied by many useful inputs that occur in practice, and if one uses a sparse representation of matrices. The improvement happens because the multiplications involve an overwhelming majority of empty matrices. This result is relevant to modern computing: divide-and-conquer algorithms with a polylogarithmic conquer step can be parallelized relatively easily.