Simulation Meets Hollywood

Abstract

(Introduction) For many researchers, software integration is often regarded as a kind of necessary evil – something that must be done to make sure that all the research components of a large system fit together and interoperate properly – but not as something that is likely to contribute new research insights or suggest new solutions. Our work on constructing virtual humans to interact with people in virtual environments has involved large-scale integration of a number of software technologies that support the simulation of human behaviors, ranging from speech recognition and dialogue management through task reasoning, gesture generation and emotion modeling. In addition, because we use virtual humans in training simulations, the characters behave in the context of a scenario, so another aspect of integration has been to bring together story content with virtual human behavior. In integrating these various components and content, we have been surprised to find that the conventional wisdom about integration does not hold: the integration process has raised new research issues and at the same time has suggested new approaches to long-standing issues. This paper describes how that has taken place and our discoveries. We begin with a brief description of the background behind our work in training and the approach we have taken to improving training. We then describe the technology components we have developed, the system architecture we use, and we conclude with the insights we have gained from the integration process.