Natural language understanding by combining statistical methods and extended context-free grammars

Abstract

This paper introduces an novel framework for speech understanding using extended context-free grammars (ECFGs) by combining statistical methods and rule based knowledge. By only using 1st level labels a considerable lower expense of annotation effort can be achieved. In this paper we derive hierarchical non-deterministic automata from the ECFGs, which are transformed into transition networks (TNs) representing all kinds of labels. A sequence of recognized words is hierarchically decoded by using a Viterbi algorithm. In experiments the difference between a hand-labeled tree bank annotation and our approach is evaluated. The conducted experiments show the superiority of our proposed framework. Comparing to a hand-labeled baseline system ( ) we achieve 95,4 % acceptance rate for complete sentences and 97.8 % for words. This induces an accuray rate of 95.1 % and error rate of 4.9 %, respectively F1-measure 95.6 % in a corpus of 1 300 sentences. © 2008 Springer-Verlag Berlin Heidelberg.