Improved estimation of entropy for evaluation ofword sense induction

3Citations
Citations of this article
74Readers
Mendeley users who have this article in their library.

Abstract

Information-theoretic measures are among the most standard techniques for evaluation of clustering methods including word sense induction (WSI) systems. Such measures rely on sample-based estimates of the entropy. However, the standard maximum likelihood estimates of the entropy are heavily biased with the bias dependent on, among other things, the number of clusters and the sample size. This makes the measures unreliable and unfair when the number of clusters produced by different systems vary and the sample size is not exceedingly large. This corresponds exactly to the setting of WSI evaluation where a ground-truth cluster sense number arguably does not exist and the standard evaluation scenarios use a small number of instances of each word to compute the score.We describe more accurate entropy estimators and analyze their performance both in simulations and on evaluation of WSI systems.

Cite

CITATION STYLE

APA

Li, L., Titov, I., & Sporleder, C. (2014). Improved estimation of entropy for evaluation ofword sense induction. Computational Linguistics, 40(3), 671–685. https://doi.org/10.1162/COLI_a_00196

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free