Measurement method research of Chinese texts’ difficulty based on two-characters continuations

Abstract

Two-characters continuation, which is a string with two characters emerging in linear sequence, can break through the encapsulation and independence of long solidified language chunks (words and phrases). In this way, two-characters continuation can measure the information of not only static language units (words and phrases) but also their combination in the text. Therefore, two-characters continuation is used as a measurement unit for investigating Chinese text’s difficulty, to enhance the accuracy of measuring text’s difficulty. Three different measurement methods of text’s difficulty are proposed, which are respectively based on "continuation index of character", "new and stable two-characters continuation" and "emerging tendency of two-characters continuation". The results show that compared to other two methods, the measurement method of text’s difficulty based on new and stable two-characters continuations has better effectiveness, whose accuracies for measuring text’s difficulty with 6 levels, 3 levels and 2 levels difficulties can reach 36.4%, 64.6% and 79.6%, respectively. In addition, compared to Jiang and Wu’s research works, the above measurement method also shows a better effectiveness.