Evaluating Training Effectiveness : Instructional Support Software from Squads to Schoolhouses

Abstract

As instructional simulations become more prolific, both official and unofficial instructors are expected to facilitate their use in training and education. For formally trained and billeted instructors, this entails the expected instructional duties as well as often-challenging technology administration. Informal instructors (such as small unit leaders) are expected to incorporate simulation into their units’ training activities, too; however, small unit leaders typically receive minimal guidance on how to effectively facilitate simulation-based training. Instead, when it comes to implementing simulations, many small unit leaders may “wing it,” rather than carefully planning, executing, and evaluating the training in accordance with instructional best practices. To address these concerns, the project team developed an Instructional Support System (ISS) that, in part, helps instructors (across all experience levels) design and deliver more effective and efficient simulation-based training. The ISS software integrates with the Deployable Virtual Training Environment (DVTE), a laptop-based simulation platform used throughout the Marine Corps. The ISS facilitates numerous instructional tasks, from trainee management to after-action review, in addition to providing resources that help users develop more systematic, instructionally sound curricula. A two-fold investigation was conducted to assess the operational support capabilities of the ISS and to determine whether the ISS’s lesson development support enabled small unit leaders to create more effective lesson plans. First, Marines (N = 57), trained in DVTE, were asked to perform two key tasks (i.e., launching a scenario and identifying a scenario for a given training objective) using either DVTE alone or DVTE and the ISS. We examined their efficiency and effectiveness to assess the systems’ operational utility. Second, Marine Sergeants (N=80) at the Enlisted Professional Military Education (E-PME) schoolhouse were asked to develop lessons on Call For Fire. Using control and experimental conditions, we compared the utility, effectiveness, and appeal of the ISS against DVTE alone (i.e., the status quo). In both cases, the participants using the ISS significantly outperformed those in the DVTE-only conditions. ABOUT