A Project Management Approach to Using Simulation for Cost Estimation on Large, Complex Software Development Projects Carolyn Mizell, NASA, Kennedy Space Center Linda Malone, University of Central Florida Abstract: An engineering manager can use this article to develop an approach to estimating cost and scheduling for large, complex software development projects that considers the inherent difficulties of developing accurate estimates for such projects. None of the approaches or tools available today can estimate the true cost of software with any high degree of accuracy early in a project. This article provides an approach that uses a software development process simulation model that considers and conveys the level of uncertainty that exists when developing an initial estimate. A NASA project will be analyzed using simulation and data from the Software Engineering Laboratory to show the benefits of such an approach. Keywords: Software Development, Project Management, Cost Hslimation, Discrete Event Simulation EMJ Focus Areas: Program and Project Management, Systems Hiigiiiecriiig, Quantitative Methods & Models Cprojectsschedule ost and estimation for large software development is historically inaccurate. Popular estimating models have been shown to be only within 25% of actual costi for 50% of the time (Ferens and Christensen. 1998). Using a simulation tool, this article presents an estimation approach that illustrates the effects of normal working dynamics on the cost of a large software development project throughout the project's evolution. Simulation models can be used to communicate the uncertainty and complexity of the development process and can provide a check on other estimating methods that may be used. The ability to obtain an accurate estimate of an entire project prior to its start is, unfortunately, unrealistic. And yet cost and schedule estimates are necessary as management commits to funding such projects or bidding on a job. Simulation models are typically used to analyze the effects of process changes, and not for developing initial cost and schedule estimations. This article will describe how simulation models can be used for this new purpose and will show the benefits of using simulation as an estimation tool. The tool will illustrate the difficulties management faces in forecasting budgets at the beginning of a project and may encourage more realistic approaches to budgetary planning, including phased funding. The task of cost estimation for project managers of software development projects becomes more and more daunting as the size and complexity of the project increases. Complex software development projects are likened to pure research and development projects, with all of the inherent difficulties of managing and planning for work that is innovative and unique and that has uncertain requirements (Abdel-Hamid and Madnick, 1991). Planning requires that an estimate be developed at a time in the project when the values of key parameters such as product size and staff capabilities are unknown. This makes it unrealistic to expect an accurate estimate. Even if the values of key variables couid be known with certainty at the beginning of a project, software activities are labor intensive and prone to all the complex and dynamic factors that afTect human performance therefore, software development is not a deterministic activity and an estimate will require adjustments throughout the project until all the variables are known. The initial estimate for a project is the most difficult and least accurate, since there is less data available. Different tools and techniques for developing software project estimates exist, but none are guaranteed to give an accurate estimate. Often, though, the initial (and highly uncertain) estimate becomes the official estimate for the entire project and is used to judge whether the project is successful. Trying to obtain a precise estimate at a very early stage in a project has led to the use of techniques that do not depict the uncertainty and complexity of the factors. Human nature prefers a single number for an estimate as opposed to a range of numbers, even though a range estimate will have a much higher chance of including an accurate value (Boehm and Fairly, 2000). Much of the research work carried out in the software cost estimation field has been devoted to algorithmic models such as COCOMO, and yet methods that rely on expert judgment are still the most commonly used approaches (Agarwal and Kumar, 2001). Expert judgment approaches rely on experience on past projects and published industry averages. Average data does not tell the whole story, and although past projects may seem similar, they will not have the same development costs, since estimates based on past experience do not account for changes in environments, politics, or organizations (Abdel-Hamid and Madnick, 1991). In addition, historical data and experts' memories of the past can be tainted. Even though expert judgment is the most often used technique, empirical software estimation models such as COCOMO are still widely used. These tools provide rigor to the estimating process, but the portability of these tools to environments different from tbat for which the tool was developed comes into question. The developer of COCOMO, Barry Boehm, admits that COCOMO is not right for every development environment (Boehm, Abts et al., 2000). In essence, none of the approaches or tools available today can estimate the true cost of software with any high degree of accuracy early in a project. Managers should be given a technique that identifies risk and uncertainty based on the seemingly Refereed management tool manuscript. Accepted by Associate Editor Componation. 28 Engineering Management Journal Voi. 19 No. 4 December 2007