An empirical evaluation of ant build maintenance using Formiga

Abstract

As a software project evolves, so does its build system. Significant effort is necessary to maintain the build system to cope with this evolution, in part because changes to source code often require parallel changes in the build system. Our tool, Formiga, is a build maintenance and dependency discovery tool for the Ant build system. Formiga's primary uses are to automate build changes as the source code is updated, to identify the build dependencies within a software project, and to assist with build refactoring. Formiga is implemented as an IDE plugin, which allows it to recognize when project resources are updated and automatically update the build system accordingly. A controlled experiment was conducted to assess Formiga's ability to assist developers with build maintenance. Subjects responded to scenarios in which various forms of build maintenance and/or knowledge about deliverables and their contents were requested. Subjects completed eleven different build maintenance tasks in both an experimental condition using Formiga and a control condition using only conventional IDE services. The study used a balanced design relative to both task order and use of Formiga. This design also ensured that order balancing was not confounded with the subjects' level of Ant expertise. Formiga was shown to significantly reduce the time required to perform build maintenance while increasing the correctness with which it can be performed for both novice and experienced developers.