Accuracy Rate in Live Subtitling: The NER Model

Abstract

For some time now, subtitling companies have been providing broadcasters with regular data on the accuracy of their live subtitles. In some cases this is actually a contractual obligation, as companies are required to obtain a given accuracy rate in their subtitles. However, given that accuracy calculations vary greatly between countries and even companies, the question arises of whether we are effectively comparing incomparable data. The aim of this paper is, first of all, to review different accuracy models used around the world. Then, using the model put forward in Romero-Fresco (2011) as a starting point, a new model is presented to assess the accuracy of live subtitles in different countries and in different languages by analyzing the extent to which errors affect the coherence of the subtitled text or modify its content. Real-life examples in English, Spanish, Italian and German obtained from a corpus of 35,000 words are included. The focus is placed on respoken subtitles, which are nowadays the most common type of live subtitles. However, the model is also applicable to automatic subtitles, since, given the rapid progress of speech recognition technology, they are likely to be introduced in the near future. The model is currently being used by regulators, broadcasters, companies and training institutions in Spain,