A cognitive approach to parsing with neural networks

Abstract

According to Cognitive Grammar (CG) theory, the overall structure of a natural language is motivated by a relatively small set of domain-independent cognitive abilities. In this paper, we draw insights from CG to propose an approach to natural language parsing with little syntactic annotation. A sentence functions as a cohesive whole because its parts are meaningfully linked. We propose that every part of a sentence can be analysed along three axes: composition, interaction and autonomy. When two expressions semantically correspond in all the three axes we call them cohesive. We present an algorithm that reads parts of sentences incrementally, recognises their construction schemas along the three axes, assembles any two component schemas into one composite schema if they are cohesive, parses a span of text as incrementally successive assembly of components into composites, retains multiple running parses within the span and chooses the best parse. The basic construction schema definitions and their patterns of assembly are implemented as dictionary-cum-rules because they are fewer in number, largely language-independent and can be extended to handle language-specific variations. A basic feedforward neural network component was trained to learn all valid patterns of assemblies possible in a span of text and to choose the best parse. A successful parse exhausts all the words in the sentence and ensures local cohesion and assembly at every stage of analysis. We present our approach, parser implementation and evaluation results in Welsh and English. By adding WordNet synsets we are able to show improvements in parser performance.