Improving visual estimation through active feedback

Abstract

In field surveys, ecological researchers and practitioners routinely make quantitative judgements that are known to vary in quality. Feedback about judgement accuracy is crucial for improving estimation performance yet is not usually afforded to fieldworkers. One reason it is rare lies in the difficulty of obtaining 'true values' (e.g. percentage cover) to learn from. Often, the only information we can access is other people's estimates of the same thing. Group average estimates tend to be remarkably accurate. By extension, receiving feedback about group averages may improve the estimation performance of individuals, dispensing with the need for 'true values' to learn from. In experiment 1, we tested whether feedback using group averages might improve estimates of species abundance as much as feedback using true values. However, not all feedback approaches are effective. In experiment 2, we compared two feedback formats for presenting information about group estimates of percentage cover. In both experiments, we used a novel 4-point interval estimation approach to quantify uncertainty that is known to reduce overconfidence but is yet to be applied in ecology. Results from experiment 1 show that feedback about group averages improved performance (calibration and accuracy) almost as much as feedback about the truth, despite the fact that group averages were generally not close to true values. In contrast, group averages in experiment 2 were remarkably close to true values, but the only participants who improved their estimates were those who evaluated their own performance during the feedback session, using active feedback with a calibration component. Feedback reminds surveyors not to give over-precise estimates and to appropriately reflect uncertainty. It improves calibration and accuracy of abundance estimates and could reasonably improve estimates of other quantities. Drawing on the wisdom of crowds, group averages could be used as a proxy for true values in feedback procedures. However, the format for delivering feedback matters. Actively engaging participants by having them evaluate their own estimation performance appears critical to improving their subsequent judgements, compared with passive feedback. We advocate the introduction of feedback into the training of ecologists. © 2012 The Authors. Methods in Ecology and Evolution. © 2012 British Ecological Society.