On the Robustness of Entropy-Based Similarity Measures in Evaluation of Subcategorization Acquisition Systems

Abstract

Some statistical learning systems are evaluated using measures of distributional similarity. To deal with the problem of zero events in the distributions under comparison, smoothing is frequently performed before similarity measures are applied. Smoothing alters the information in the original distribution, and may add noise to the results. Here, we investigate the sensi tivity of entropy-based similarity measures to noise from uninformative smoothing. Our experiments with two subcategorization acquisi tion systems show that similarity measures vary in their robustness. While some are led astray by noise from smoothing, others are more resilient.