DalGTM at SemEval-2016 task 1: Importance-aware compositional approach to short text similarity

Abstract

This paper describes our system submission to the SemEval 2016 English Semantic Textual Similarity (STS) shared task. The proposed system is based on the compositional text similarity model, which aggregates pairwise word similarities for computing the semantic similarity between texts. In addition, our system combines word importance and word similarity to build an importance-similarity matrix. Three different word similarity measures are used in our three submitted runs. The evaluation results show that taking into account context dependent word importance information improves performance. However, the performance of the system varies drastically between different evaluation subsets. The best of our submitted runs achieves rank 60th with weighted mean Pearson correlation to human judgements of 0.6892.