Numerical analysis of consensus measures within groups

Abstract

Measuring the consensus for a group of ordinal-type responses is of practical importance in decision making. Many consensus measures appear in the literature, but they sometimes provide inconsistent results. Therefore, it is crucial to compare these consensus measures, and analyze their relationships. In this study, we targeted five consensus measures: Φe (from entropy), Φ1 (from absolute deviation), Φ2 (from variance), Φ3 (from skewness), and Φmv (from conditional probability). We generated 316,251 probability distributions, and analyzed the relationships among their consensus values. Our results showed that Φ1, Φe, Φ2, and Φ3 tended to provide consistent results, and the ordering Φ1 ≤ Φe ≤ Φ2 ≤ Φ3 held at a high probability. Although Φmv had a positive correlation with Φ1, Φe, Φ2, and Φ3, it had a much lower tolerance for even a small proportion of extreme opposite opinions than Φ1, Φe, Φ2, and Φ3 did.