What is our power to detect device effects in animal tracking studies?

Abstract

The use of bio-logging devices to track animal movement continues to grow as technological advances and device miniaturisation allow researchers to study animal behaviour in unprecedented detail. Balanced against the remarkable data that bio-loggers can provide is a need to understand the impact of devices on animal behaviour and welfare. Recent meta-analyses have demonstrated the impacts of device attachment on animal behaviour, but there is a concern about the frequency and clarity with which device effects are reported. One aspect lacking in many studies is assessment of the statistical power of tests of device effects, yet such information would assist the interpretation of results. We address this issue by providing an overview of the statistical power, as well as the Type M (magnitude) and Type S (sign) error rate, of tests of device effects within the avian tracking literature across a range of assumed effect sizes. The median power of statistical tests ranged from 9% to 65% across a range of assumed effect sizes corresponding to benchmark values for small, moderate and large effects (d = 0.2, 0.5, 0.8, respectively). Moreover, when using effect sizes derived from previous a meta-analysis (d = 0.1), median power was only 6%. When assuming smaller effect sizes, statistical tests were characterised by high Type M and Type S error rates, suggesting that statistically significant results of device effects will tend to exaggerate the size of such effects and may estimate the sign of an effect in the wrong direction. Well-designed tracking studies will reduce device effects to low levels and consequently issues associated with low power will be commonplace. Nevertheless, assessment of device effects remains important, particularly when embarking on novel tracking studies. We recommend that statistical tests of device effects are reported clearly and are routinely accompanied by assessment of statistical power, including Type M and Type S errors, based upon realistic external estimates of effect size. Reporting the statistical power can help avoid the pitfalls of overstating results from individual studies, shift the emphasis to accurate reporting of effect sizes and guide decisions about the ethical impacts of device attachment.