Implementation of an algorithm to classify discourse segments from documents for knowledge acquisition

Abstract

The overall objective of this paper is to acquire diagnostic knowledge about aircraft assembly in an automated manner, in order to minimize issues from occurring in new, similar situations. This research uses documents, prepared by experts, as a source of knowledge. The first step of the process of knowledge acquisition is segmentation of relevant sections of documents. From many methods that currently exist for such segmentation and classification, one method, namely ‘discourse analysis’ is chosen for analyzing documents (with future knowledge considerations in mind). Using discourse analysis, entities from sentences are extracted to identify what is being discussed in a chunk of text. These entities are then compared to a domain knowledge base, such as an ontology, to see how (semantically) close the discussion is to the domain of interest. A method for such segmentation had been previously proposed, and is summarised here. This paper describes the efforts for partial implementation of this method. Computer-based tools are used for this implementation, such as Natural Language Toolkit, Boxer, and Ontologies. The Natural Language Toolkit is used for performing text processing, such as tokenization; Boxer is used for Discourse Analysis; Ontologies are used as a knowledge base for domain related terminologies. The method calculates a semantic score for each sentence against the terms taken from related domain ontologies. If the sentence has terms matching those in the ontology, that sentence is classified as being related to the domain of aircraft assembly. The implementation is then applied on test documents to evaluate its performance.