Evaluation of a process for architectural assumption management in software development

Abstract

Context: Architectural assumption management is critical to the success of software development projects. In this paper, we propose an Architectural Assumption Management (AAM) process, comprised of four AAM activities: Architectural Assumption Making, Description, Evaluation, and Maintenance. Objective: Evaluating the AAM process in architectural assumption management, regarding the ease of understanding and the effort of conducting the AAM process, as well as the effectiveness of using the AAM process to make architectural assumptions explicit and to identify and reduce invalid architectural assumptions. Method: An explanatory study with 88 first-year master students in software engineering, and an exploratory study with five practitioners from five companies. Results: (1) the ease of understanding the AAM process is moderate for first-year master students but easy for practitioners; (2) the effort of conducting the AAM process is moderate for first-year master students; (3) Making and Evaluation took the students more time than Description and Maintenance; (4) the practitioners considered Evaluation as the most time consuming activity; (5) the AAM process can help to make architectural assumptions explicit and to identify and reduce invalid architectural assumptions in projects. The majority of the students and practitioners agreed that Architectural Assumption Evaluation is the most helpful activity for all these three aspects. For other activities, there are different opinions about their helpfulness; and (6) there are various factors identified that can impact the aforementioned results. Being aware of and properly adjusting these factors can facilitate the application of the AAM process in projects. Conclusions: The AAM process aims at systematically managing architectural assumptions in software development. The results of the case studies provide preliminary empirical evidence for the evaluation of the AAM process.