Using automatic failure detection for cognitive agents in eclipse (AAMAS 2016 DEMONSTRATION)

Abstract

In order to reduce debugging effort and enable automated failure detection, we proposed an automated testing framework for detecting failures in cognitive agent programs in previous work. This approach is based on a minimal set of temporal operators that enable the specification of test conditions with sufficient expressiveness for detecting all failures in an existing failure taxonomy. We also introduced an according approach for specifying test templates that supports a programmer in writing tests. In this demonstration paper, the automated test framework for the Goal agent programming language that has been created for the Eclipse platform is introduced, with a focus on its practical aspects, i.e., how to use it to detect failures in cognitive agents. As fault localization is an important follow-up to failure detection, the integration of the test framework in the existing source-level debugger for Goal is discussed as well. In addition, an empirical evaluation of the automated testing framework implementation for Goal is presented based on the work of almost 200 novice agent programmers.