Coupling the users: The benefits of paired user testing for iDTV

Abstract

Interactive digital television (iDTV) is a social medium and must therefore be tested in a context as close to real life as possible. This explains why we saw the potential and importance for the involvement of real-life couples in iDTV usability testing. In this article, an experiment that compares single user testing and coparticipation testing with couples for the evaluation of several Flemish iDTV applications is described. The study found that, first, there was less probing needed by the facilitator to think out loud in the think aloud/coparticipation method with couples than in the think aloud/single test user method. Second, couples did not encounter difficulties working together with the iDTV applications. Further, couples did not lose time by discussing irrelevant issues during the test session. A fourth finding is that couples detected more usability hits than single test users. The quality of comments, however, was the same in both conditions. Sixty perecent of the comments consisted of intrinsic suggestions and 40% of general problem detections. Another issue was raised through findings during the test. Couples in general were enthusiastic to participate, put in little effort on their part in the test session, and evaluated the test session as easy and fun to do. On the contrary, single test users in general were not sure whether they would like to participate again in future tests, declared that the test session demanded considerable effort, and evaluated the test session less positively.