Unsupervised trained functional discourse parser for e-learning materials scaffolding

Abstract

The article describes the way of automatic segmentation of natural language text into fragments with different functional semantics. The proposed solution is based on the analysis of how the various parts of speech are distributed through the text. The amount and variety of nouns, verbs and adjectives is calculated for a set of sliding windows with the same length. The text is divided into fragments using clustering of windows set. We considered two clustering methods: ISODATA and a method based on the minimum spanning tree. The results of comparison of the methods with each other and with the manually text markup are shown.