Overselling overall map accuracy misinforms about research reliability

Abstract

Context: Image classification is routine in a variety of disciplines, and analysts rely on accuracy metrics to evaluate the resulting maps. The most frequently used accuracy metric in Earth resource remote sensing is overall accuracy. However, the inherent properties of this accuracy metric make it inappropriate as the single metric for map assessment, particularly when a map contains imbalanced categories. Objectives: We discuss four noteworthy problems with overall accuracy. Under circumstances frequently encountered, overall accuracy is misleading or misinterpreted. Methods: Literature review, hypothetical examples, and mathematic equations are used to prove overall accuracy is a poor general indicator of map quality. Conclusions: Any research that involves classification techniques or a map product that is evaluated only with overall accuracy may be unreliable. It is necessary for map providers to publish the error matrix and its development procedure so that map users can computer whatever metrics as they wish.