Measuring maintainability of OO-software - validating the IT-CISQ quality model

Abstract

The Consortium for IT Software Quality (IT-CISQ) standard claims to provide valid measures as well as an assessment method that are suitable for properly measuring software quality. We implemented the measures for maintainability as specified by the IT-CISQ standard and wanted to find out whether both – the IT-CISQ defined assessment method and the IT-CISQ measures – are suitable for determining the maintainability of object-oriented systems. We identified a reference study that classifies the maintainability of eight open-source Java projects. This study follows a comprehensive measurement and assessment process and therefore can be used for validating the IT-CISQ approach. Due to the missing consideration of project size metrics, the IT-CISQ assessment method is not capable of properly determining the quality of projects. Even considering size metrics does not substantially enhance the result, which is an indicator that the measures proposed by IT-CISQ do not properly measure maintainability. Finally, our benchmarking approach was applied. It sets the measurements in relation to 26 projects that constitute the benchmark base. Despite a lack of statistical significance, the benchmarking results show a better correlation with the ranking published by the reference study. As our benchmarking approach is well validated, we can conclude that the measures proposed by IT-CISQ have to be considerably enhanced, i.e., additional measures have to be added to be able to determine the maintainability of object-oriented software projects.