Towards the development of the multilingual multimodal virtual agent

Abstract

The mobile virtual agent (assistant) is one of today’s most intriguing new technologies. Development of such an agent is multidisciplinary work, and natural language processing is an indispensable part of this work. Our goal is to develop a multilingual multimodal virtual agent. In this paper, we describe the first steps towards this goal – the design, development, and evaluation of the intelligent translation agent. The agent provides speech to speech translation of words, phrases, and sentences from English into Spanish, French, or Russian. The initial evaluation performed for natural language components, as well as for the agent in general, indicated that there is user interest and that such an application is useful.