Performance and Robustness Engineering and the Role of Automated Software Development

Abstract

Performance engineering aims to demonstrate that the software being developed will meet the performance needs. The goal of robustness engineering is to prove that the system will work correctly in the presence or after occurence of faults or stress conditions. From this point of view robustness engineering complements performance engineering to cover the full range of conditions to which a system1 may be exposed. Performance and robustness properties need to be continuously monitored during the development process to ensure that the system will meet the user’s needs at the end. This paper will discuss aspects and problems of performance and robustness engineering. Also, it presents an approach, called “ISG” (Instantaneous System and Software Generation)2 which allows to continuously derive performance and robustness properties from the system-under-development. In case of ISG figures are derived from the real system right from the beginning. Therefore deviations from the desired functional, performance and robustness envelope can be corrected at an early stage. The capability for getting an immediate3 feedback from the system is obtained by the automated generation of the software from system engineering inputs. ISG builds the software by construction rules. This reduces the manual effort and allows for an immediate and representative feedback right after provision of inputs by the user. Due to automation the system can easily be instrumented on a user’s demand without requiring any additional programming effort. ISG automatically stimulates the system and exposes it to stress tests and fault injection, and records coverage and performance figures. By incremental development a smooth transition from the first idea to the final version is possible at low risk. The ISG approach has been applied to the domain of realtime, distributed, fault-tolerant systems and shall be extended towards other application domains in future such as databases and graphical user interfaces.