A probabilistic approach to grammatical analysis of written english by computer

Abstract

Work at the Unit for Computer Research on the Eaglish Language at the University of Lancaster has been directed towards producing a grammatically snnotated version of the Lancaster-Oslo/ Bergen (LOB) Corpus of written British English texts as the prelminary stage in developing computer programs and data files for providing a grammatical analysis of -nestricted English text. From 1981-83, a suite of PASCAL programs was devised to automatically produce a single level of grammatical description with one word tag representing the word class or part of speech of each word token in the corpus. Error analysis and subsequent modification to the system resulted in over 96 per cent of word tags being correctly assigned automatically. The remaining 3 to ~ per cent were corrected by human post-editors. brk is now in progress to devise a suite of programs to provide a constituent analysis of the sentences in the corpus. So far, sample sentences have been automatically assigned phrase and clause tags using a probabilistic system similar to word tagging. It is hoped that the entire corpus will eventually be parsed.