Comput Math Organ Theory (2010) 16: 271–299
DOI 10.1007/s10588-010-9065-3
The synthetic teammate project
Jerry Ball · Christopher Myers · Andrea Heiberg ·
Nancy J. Cooke · Michael Matessa ·
Mary Freiman · Stuart Rodgers
Published online: 1 August 2010
© US Government 2010
Abstract The main objective of the Synthetic Teammate project is to develop lan-
guage and task enabled synthetic agents capable of being integrated into team train-
ing simulations. To achieve this goal, the agents must be able to closely match hu-
man behavior. The initial application for the synthetic teammate research is creation
of an agent able to perform the functions of a pilot for an Unmanned Aerial Vehi-
cle (UAV) simulation as part of a three-person team. The agent, or synthetic team-
J. Ball (

)
Air Force Research Laboratory, Mesa, AZ, USA
e-mail: Jerry.Ball@mesa.afmc.af.mil
C. Myers
Air Force Research Laboratory, Wright-Patterson Air Force Base, OH, USA
e-mail: Christopher.Myers2@wpafb.af.mil
A. Heiberg
General Motors, Mesa, AZ, USA
e-mail: ninaheib@cox.net
N.J. Cooke
Cognitive Engineering Research Institute, Mesa, AZ, USA
e-mail: Nancy.Cooke@asu.edu
M. Matessa
Alion, Boulder, CO, USA
e-mail: mmatessa@alionscience.com
M. Freiman
L3 Communications, Mesa, AZ, USA
e-mail: Mary.Freiman@mesa.afmc.af.mil
S. Rodgers
AGS TechNet, Dayton, OH, USA
e-mail: stu@agstechnet.com

272 J. Ball et al.
mate, is being developed in the ACT-R cognitive architecture. The major compo-
nents include: language comprehension and generation, dialog management, agent-
environment interaction, and situation assessment. Initial empirical results suggest
that the agent-environment interaction is a good approximation to human behavior
in the UAV environment, and we are planning further empirical tests of the synthetic
teammate operating with human teammates. This paper covers the project’s modeling
approach, challenges faced, progress made toward an integrated synthetic teammate,
and lessons learned during development.
Keywords Synthetic teammate · Language comprehension/generation · Dialog
management · Situation model · Agent-environment interaction
1 Project overview
Previous research has shown the benefits of developing synthetic agents to support
team training (Jones et al. 1999; Tambe et al. 1995; Zachary et al. 2001) and for evalu-
ation of computer interfaces (Byrne et al. 1994; Ritter et al. 2002). The main objective
of the Synthetic Teammate project is to develop synthetic agents capable of being in-
tegrated into team training simulations. To achieve this goal while maintaining train-
ing efficacy, the synthetic agents must be capable of closely matching human behav-
ior across several capacities, including situation assessment, task behavior, language
comprehension and generation, and dialog management. Matching human behavior
is a goal of computational cognitive architectures, which replicate human perceptual,
cognitive, and motor abilities. Models developed within a cognitive architecture are
empirically validated against human data. However, most models developed within
cognitive architectures model laboratory tasks which isolate specific cognitive and
perceptual phenomena. These models are typically small in scale and do not general-
ize to other tasks. Although this project takes advantage of previous work done with
the ACT-R cognitive architecture (Anderson 2007), it broadens that work consider-
ably, integrating multiple components into a large-scale model of a complex task. In
developing a large-scale model of multiple cognitive capacities, this research aligns
with research aimed at the development of Artificial General Intelligence. However,
most AI research adopts a black-box approach that focuses on modeling input-output
behavior and makes little or no commitment to the cognitive plausibility of internal
mechanisms, relying instead on use of high powered algorithmic mechanisms that
are typically not cognitively plausible. By way of contrast, our research is focused on
glass-box modeling of the internal mechanisms in a cognitively plausible manner in
support of developing a functional system (Ball 2006, 2008).
In attempting to build a large-scale model of a complex task, we are pushing the
field of computational cognitive modeling outside of its comfort zone. Complex in-
tegration issues which do not arise in small-scale models become major challenges.
Determining the appropriate level of cognitive fidelity for the different components
of the system hinges on the availability of time and resources, and interacts with the
overall goal of building an end-to-end system. Empirical validation becomes a real
challenge. The complexity of the model constrains the use of standard empirical val-
idation methodologies (Cassimatis et al. 2008). It is unclear to what extent empirical