A systematic process to define expert-driven software metrics thresholds

Abstract

Software metrics are usually used for quantification, not giving the necessary support for decision making. To increase their usefulness, it is necessary to give them meaning through the definition of significant thresholds. Despite its importance, the state of the art on threshold derivation is mostly based on data-driven approaches. This paper presents a systematic approach to define thresholds for metrics in the absence of data and based on eliciting knowledge from experts. The proposed approach is based on identifying context factors that influence the thresholds for a given metric and is supported by fuzzy logic concepts to model the crisp value (i.e., collected data) into a linguistic variable (i.e., interpreted information). We present context factors elicited from three experts for the metrics code coverage, static code analysis warnings count and defect count. Further, we present cases on how to implement the proposed approach. As a result, we conclude that the approach is promising.