Towards better evaluation for human language technology

Abstract

Both research and evaluation in human language technology have enjoyed a big surge for the last fifteen years. Performance has made major advances, partially due to the availability of resources and the interest in the many evaluation forums present today. But there is much more to do, both in terms of new areas of research and in improved evaluation for these areas. This paper addresses the current state-of-the-art in evaluation and then discusses some ideas for improving this evaluation. © 2008 Springer-Verlag Berlin Heidelberg.