Hawthorne effect reporting in orthodontic randomized controlled trials: Truth or myth? Blessing or curse?

Abstract

Objective To investigate in 10 orthodontic journals how many randomized controlled trials (RCTs) considered the Hawthorne effect, and if considered, to determine whether it was related to the patients or the therapists involved in the trial and, finally, to discuss the Hawthorne effect in an educational way. Materials and methods A search was performed on the Medline database, via PubMed, for publication type randomized controlled trial' published for each journal between 1 August 2007 and 31 July 2017. The American Journal of Orthodontics and Dentofacial Orthopedics, Angle Orthodontist, Australian Orthodontic Journal, Dental Press Journal of Orthodontics, European Journal of Orthodontics, Journal of Orthodontics, Journal of Orofacial Orthopedics, Korean Journal of Orthodontics, Orthodontics and Craniofacial Research and Progress in Orthodontics were assessed. Two independent reviewers extracted the data and identified whether the Hawthorne effect was considered or discussed in the articles and whether the Hawthorne effect was related to the behaviour of the patients, the therapists, or both. Results The initial search generated 502 possible trials. After applying the inclusion and exclusion criteria, 290 RCTs were included and assessed. The Hawthorne effect was considered or discussed in 10 of 290 RCTs (3.4%), and all were related to the patients' and none to the therapists' behaviour. Conclusions The Hawthorne effect reported in orthodontic RCTs was suboptimal. The researchers' lack of knowledge about this phenomenon is evident, despite evidence that the Hawthorne effect may cause over-optimistic results or false-positive bias.