A random walk framework to compute textual semantic similarity: A unified model for three benchmark tasks

Abstract

A network of concepts is built from Wikipedia documents using a random walk approach to compute distances between documents. Three algorithms for distance computation are considered: hitting/commute time, personalized page rank, and truncated visiting probability. In parallel, four types of weighted links in the document network are considered: actual hyperlinks, lexical similarity, common category membership, and common template use. The resulting network is used to solve three benchmark semantic tasks - word similarity, paraphrase detection between sentences, and document similarity - by mapping pairs of data to the network, and then computing a distance between these representations. The model reaches stateof-the-art performance on each task, showing that the constructed network is a general, valuable resource for semantic similarity judgments. © 2010 IEEE.