Evaluating the generation of domain ontologies in the knowledge puzzle project

Abstract

One of the goals of the Knowledge Puzzle Project is to automatically generate a domain ontology from plain text documents and use this ontology as the domain model in computer-based education. This paper describes the generation procedure followed by TEXCOMON, the Knowledge Puzzle Ontology Learning Tool, to extract concept maps from texts. It also explains how these concept maps are exported into a domain ontology. Data sources and techniques deployed by TEXCOMON for ontology learning from texts are briefly described herein. Then, the paper focuses on evaluating the generated domain ontology and advocates the use of a three-dimensional evaluation: structural, semantic, and comparative. Based on a set of metrics, structural evaluations consider ontologies as graphs. Semantic evaluations rely on human expert judgment, and finally, comparative evaluations are based on comparisons between the outputs of state-of-the-art tools and those of new tools such as TEXCOMON, using the very same set of documents in order to highlight the improvements of new techniques. Comparative evaluations performed in this study use the same corpus to contrast results from TEXCOMON with those of one of the most advanced tools for ontology generation from text. Results generated by such experiments show that TEXCOMON yields superior performance, especially regarding conceptual relation learning. © 2006 IEEE.