Stuttering as a trait or a state revisited: motor system involvement in persistent developmental stuttering

Abstract

This corrigendum reports an update to the meta-analysis reported in Belyk et al. (). The publicly-available program GingerALE contains the most widely adopted algorithm for meta-analyses by activation likelihood estimation (ALE) of functional magnetic resonance imaging (fMRI) experiments. This program was recently reported by its developers to contain long-standing implementation errors that may have affected the statistical thresholds of many published meta-analyses, including our own (Eickhoff et al.,). Recently, the BrainMap Development Team formally reported two long-standing implementation errors in the GingerALE software (Eickhoff et al.,). These errors affected published ALE analyses using False-Discovery Rate (FDR) corrections for multiple comparisons prior to May 11, 2015 (GingerALE versions prior to v.2.3.3) and cluster-wise Family-Wise Error (cFWE) corrections for multiple comparisons prior to April 26, 2016 (GingerALE versions prior to v2.3.6). The implementation errors in these versions may have caused statistical thresholds in the resultant ALE analyses to be more liberal than intended by the researchers, including in our own analysis (Belyk et al.,). Furthermore, subsequent research has demonstrated that voxel-wise FDR correction in the context of ALE has the undesirable properties of being simultaneously low in sensitivity to true effects and highly susceptible to false positives (Eickhoff et al.,). This view is supported by a broader theoretical position that voxel-wise FDR may be inappropriate for spatially smooth data, such as the data represented in ALE analyses (Chumbley & Friston,). In contrast, cluster-wise approaches to statistical thresholds provide a reasonable compromise between sensitivity and conservatism. Although cluster-wise thresholding does not permit inferences at the level of individual voxels, it is more appropriate for inferences at the level of topological features (i.e., at the level of activation clusters or anatomically defined brain areas), which may be better suited to the manner in which neuroimaging data are generally interpreted. In light of the commendable degree of transparency shown by the BrainMap Development Team, it is incumbent upon cognitive neuroscientists who have used the GingerALE versions in question to issue self-corrections where published analyses have been affected. To that end, we both report a corrigendum and provide an update to our original meta-analysis. Materials and methods We repeated our original meta-analysis of functional neuroimaging studies of persistent developmental stuttering with the most recent version of GingerALE. Briefly, the analysis used ALE to separately describe the neural correlates of having a propensity to stutter when speaking (i.e., the trait of being a person who stutters) and the behavior of stuttering (i.e., the state of currently exhibiting a stutter). Readers are referred to the original publication for methodological details (Belyk et al.,). Three changes were made from the original meta-analysis. First, we used the most recent version of the GingerALE software in which major implementation errors have been corrected (v2.3.6, retrieved August 25, 2016). Second, we applied a cFWE threshold of P < 0.05 (calculated from an initial cluster-forming threshold of uncorrected P < 0.001) in lieu of the previously used voxel-wise FDR threshold. Third, we took the opportunity to update the dataset by searching for relevant articles published since our first analysis. We searched PubMed for articles published between July 1, 2013 and August 19, 2016 using the same search terms reported in Belyk et al. (). By applying the same inclusion criteria as in the original article, we added one new study to the re-analysis of positive associations of state stuttering (Toyomura et al.,). Results Only a small number of the most robust effects from the original analysis retained significance (Fig. and Table). Trait stuttering was associated with increased activity in the orofacial premotor cortex and Rolandic operculum, and with decreased activity in the left orofacial pre/primary motor cortex. State stuttering was associated with increased activity in the right orofacial primary motor cortex, and was not associated with decreased activity in any brain area. (Figure presented.) Results of the ALE re-analysis. Axial slices in neurological convention showing regions consistently reported for (A) trait over-activations (red) and trait under-activations (blue), and (B) state over-activations (green). (C) Sagittal view highlighting trait over-activations in orofacial premotor cortex, and state over-activation in orofacial primary motor cortex. Peak coordinates and extent of activation cover the approximate somatotopic representations of the laryngeal and lip muscles. M1, primary motor cortex; PMC, premotor cortex; RO, Rolandic operculum. [Colour figure can be viewed at wileyonlinelibrary.com]. Brain regions positively or negatively associated with trait and state stuttering. After each anatomical name in the brain region column is the Brodmann number for that region. The columns labeled as x, y, and z contain the Talairach coordinates for the peak likelihood within each cluster. The mm3 column lists the total volume of each cluster (Table presented.) Discussion We have reported an update to “Stuttering as a trait or a state: An ALE meta-analysis of neuroimaging studies” (Belyk et al.,) in light of the discovery of implementation errors in GingerALE software that may have led to overly liberal statistical thresholds in our analyses. In the updated analysis, only the most robust findings from the original meta-analysis retained significance. Importantly, the re-analysis is consistent with the interpretation of the data discussed in the original article and further suggests that the most robust neural correlates of persistent developmental stuttering are found within the motor areas that control the orofacial muscles. We reiterate the view of Eickhoff et al. () that the implementation errors in previous versions of the GingerALE software do not invalidate the results of earlier meta-analyses that have used this software. Rather, earlier analyses are valid, but are more liberal than intended by the researchers. We therefore encourage readers to treat the original and updated meta-analyses as a complementary pair, with the more liberal analysis emphasizing statistical power at the risk false positives, and the more conservative analysis reducing the risk of false positives at the cost of statistical power. Although it is possible that the clusters that were not replicated in the re-analysis were false positives, this is not necessarily the case, since the more conservative analysis may have failed to detect some true effects. Eickhoff et al. () reported the influence of both sample size and effect size (estimated as the proportion of experiments that contribute to each cluster) on statistical power. From their simulations, we estimate that the clusters that retained significance in the updated analysis had statistical power ranging from approximately 0.55–0.80 (i.e., from proportion “effect sizes” of 0.38–0.44 with 9-11 total experiments). The clusters that were significant in the original analysis, but that did not retain significance in the updated analysis, had statistical power that ranged widely, from approximately 0.15–0.85 (i.e., from proportion “effect sizes” of 0.13–0.55). The upper limit of this range reflects one cluster (the supplementary motor area) that was reported in a large proportion of studies, but that did not reach significance in the re-analysis. Future meta-analyses may be better able to detect these effects as more published data become available.