Selective text utilization and text traversal

Abstract

Many large collections of full-text documents are currently stored in machine-readable form and processed automatically in various ways. These collections may include different types of documents, such as messages, research articles, and books, and the subject matter may vary widely. To process such collections, robust text analysis methods must be used, capable of handling materials in arbitrary subject areas, and flexible access must be provided to texts and text excerpts of varying size. In this study, global text comparison methods are used to identify similarities between text elements, followed by local context-checking operations that resolve ambiguities and distinguish superficially similar texts from texts that actually cover identical topics. A linked text structure, known as a text relationship map, is then created that relates similar texts at various levels of detail. In particular, text links are available for full texts, as well as text sections, paragraphs, and sentence groups. The relationship graphs are usable as conceptualization tools to illustrate various text manipulation operations and may also serve as browsing maps in situations where searches or text traversal operations are conducted under user control. In this study, the relationship maps are used to identify important text passages, to traverse texts selectively both within particular documents and between documents, and to provide flexible text access to large text collections in response to various kinds of user needs. An automated 29-volume encyclopedia is used as an example to illustrate various possible text accessing and traversal operations. Implementation details are not included in this initial study. © 1995 Academic Press, Inc.