Testing interaction with a mobile MT postediting app

Abstract

Kanjingo is a post-editing application for iOS devices developed at the ADAPT Centre (formerly CNGL) at Dublin City University (DCU). The first stage of user testing was conducted in 2014 (reported in O'Brien, Moorkens & Vreeke, 2014), and improvements were made based on the initial feedback. This abstract describes further exploratory testing based on the second iteration of the Kanjingo application. The new tests were designed with several aims: (1) testing Kanjingo for post-editing using the phone's keyboard (2) testing Kanjingo for post-editing with voice input; (3) testing Kanjingo for revision of post-edited texts; (4) testing Kanjingo general usability; and (5) testing Kanjingo interface design. This paper presents the results of the various tests, issues identified, and ideas for improvements. For example, the use of Kanjingo for post-editing with voice input, one of the most innovative forms of interaction with MT in the test, worked much better than participants expected, and this mode of input was preferred for translating from scratch when MT quality was very poor, whereas post-editing short words or phrases was found to be faster with the iPhone keyboard. In addition, we present some reflections on the strengths and weaknesses of the testing methods employed.