When NFR Templates Pay Back? A Study on Evolution of Catalog of NFR Templates

Abstract

[Context] Failures in management of non-functional requirements (NFRs) (e.g., incomplete or ambiguous NFRs) are frequently identified as one of the root causes of software failures. Recent studies confirm that using a catalog of NFR templates for requirements elicitation positively impacts the quality of requirements. However, practitioners are afraid of templates as the return on investment in this technique is still unknown. [Aim] Our aim was to investigate how the usefulness of catalog of NFR templates and its maintenance costs change over time. [Method] Using 41 industrial projects with 2,231 NFRs, we simulated 10,000 different random evolutions of a catalog of NFR templates. It allowed us to examine the distribution of catalog value, maintenance effort, catalog utilization over a sequence of projects (a counterpart of elapsing time). [Results] From the performed analysis it follows that after considering about 40 projects we can expect catalog value of 75% or more and maintenance effort of 10% or less. Then one could expect about 400 templates in the catalog, but only about 10% of them would be used by a single project (on average). [Conclusions] Usefulness and maintenance costs of catalog of NFR templates depend on the number of projects used to develop it. A catalog of high usefulness and low maintenance effort need to be developed from about 40 projects. Since high variability of studied projects, this number in practice might be lower. From the perspective of a large or medium software company it seems not a big problem.