Deaf and Hard of Hearing Viewers’ Preference for Speaker Identifier Type in Live TV Programming

Abstract

When there are multiple people shown onscreen at one time, people who are Deaf and Hard of Hearing (DHH) viewing captions may find it challenging to determine who the current speaker is, especially when speakers interrupt each other abruptly or when there is a lot of turn-taking. Prior research has proposed several methods of indicating speakers, including in-text methods and methods in which the caption is dynamically located onscreen. However, prior work has not examined the effectiveness of various speaker-identifier methods for conveying who is speaking when the number of speakers on the screen increases. To determine which speaker-identifier methods are effective for DHH viewers, as the number of speakers on screen varies, we have conducted an empirical study with 31 DHH participants. We observed DHH viewers preference for speaker-identifier types, for videos that vary in the number of speakers shown onscreen. Determining the relationship between DHH viewers’ preference for speaker-identifier methods and the number of onscreen speakers can guide broadcasters to select appropriate speaker-identifier methods based on the number of speakers that appear on the screen.