Statistical Validation of Energy Efficiency Improvements through Analysis of Experimental Field Data: A Guide to Good Practice

Abstract

Often in the area of road transport solutions and intelligent transport systems, two or more alternative solutions or methods compete in terms of energy gains, time efficiency, or other aspects. Measurements collected from field trials are used to make a comparative assessment but are usually limited because of resource constraints. The present paper describes how statistical inference techniques can be used in a systematic way, in order to validate the superior performance of one method over the other. We adopt such an approach to study the performance of two alternative routing methods in terms of achievable energy savings, although the same methodology can be widely applied to other use cases as well. We specifically employ and describe three different techniques to achieve the intended comparison, namely paired sample tests, statistical testing of mean value in a normal population, and two-sample tests in normal populations with unknown yet equal variances. We reach conclusions on whether claims of outperformance of one routing method over the other can be supported by our collected experimental data and to what extent.