Evidence-b(i)ased practice: Selective and inadequate reporting in early childhood autism intervention research

Abstract

We conducted a multi-pronged investigation of different types of reporting bias in autism early childhood intervention research. First, we investigated the prevalence of reporting failures of completed trials registered on clinicaltrials.gov, and found that only 7% of registered trials were updated with results on the registration platform and only 64% had associated published reports. Next, we investigated the extent to which inadequate reporting prevents inclusion in meta-analytic summary estimates by identifying reports of studies that were eligible for inclusion in a prior meta-analysis, and found that 25% were excluded due to inadequate reporting. Finally, we investigated selective reporting practices by analyzing the protocols of the studies included in the meta-analysis which had been registered on any trial registry and coding their timing, completeness, and consistency. We found that 23% of studies were pre-registered, 71% were late-registered, and 5% were registered at an unclear date. Only 8% of registrations specified all of the necessary components. Evidence of selective reporting was common; 36% failed to report a registered outcome, 61% reported unregistered outcomes, 23% switched primary and secondary outcomes, and 43% had assessment timepoints that differed from registration specification. Given the inadequacy of registration and reporting practices, we offer practical recommendations to facilitate improvement for the field of autism research. Lay Abstract: When researchers fail to report their findings or only report some of their findings, it can make it difficult for clinicians to provide effective intervention recommendations. However, no one has examined whether this is a problem in studies of early childhood autism interventions. We studied how researchers that study early childhood autism interventions report their findings. We found that most researchers did not register their studies when they were supposed to (before the start of the study), and that many researchers did not provide all of the needed information in the registration. We also found that researchers frequently did not publish their findings when their studies were complete. When we looked at published reports, we found that many of the studies did not report enough information, and that many studies were reported differently from their registrations, suggesting that researchers were selectively reporting positive outcomes and ignoring or misrepresenting less positive outcomes. Because we found so much evidence that researchers are failing to report their findings quickly and correctly, we suggested some practical changes to make it better.