On evaluation and training-set construction for duplicate detection

Abstract

A variety of experimental methodologies have been used to evalu- ate the accuracy of duplicate-detection systems. We advocate pre- senting precision-recall curves as the most informative evaluation methodology. We also discuss a number of issues that arise when evaluating and assembling training data for adaptive systems that use machine learning to tune themselves to specific applications. We consider several different application scenarios and experimen- tally examine the effectiveness of alternative methods of collecting training data under each scenario. We propose two new approaches to collecting training data called static-active learning and weakly- labeled non-duplicates, and present experimental results on their effectiveness.