Investigating effective ECAs: An experiment on modality and initiative

Abstract

This paper investigates the effectiveness of conversational agent-based delivery of task strategy and operational help for an interactive search tool. The study tested three modalities of advice (text-only, text-and-audio, and text-audio-and-agent) in addition to a control group with no advice. User- and system- initiated advice modes were also compared. Subjects in the text-only group outperformed other modality groups in usability errors, search performance, advice uptake and in their positive comments in the debriefing interview and post-test questionnaire. User-initiated advice was preferred and was more effective. Users criticized speech advice for being too long and difficult to control. The results suggested that the computer as social actor paradigm might not be effective for advisory applications. © IFIP International Federation for Information Processing 2007.