On Unreplicable Inferences in Psychopathology Symptom Networks and the Importance of Unreliable Parameter Estimates

Abstract

We recently wrote an article comparing the conclusions that followed from two different approaches to quantifying the reliability and replicability of psychopathology symptom networks. Two commentaries on the article have raised five core criticisms, which are addressed in this response with supporting evidence. 1) We did not over-generalize about the replicability of symptom networks, but rather focused on interpreting the contradictory conclusions of the two sets of methods we examined. 2) We closely followed established recommendations when estimating and interpreting the networks. 3) We also closely followed the relevant tutorials, and used examples interpreted by experts in the field, to interpret the bootnet and NetworkComparisonTest results. 4) It is possible for statistical control to increase reliability, but that does not appear to be the case here. 5) Distinguishing between statistically significant versus substantive differences makes it clear that the differences between the networks affect the inferences we would make about symptom-level relationships (i.e., the basis of the purported utility of symptom networks). Ultimately, there is an important point of agreement between our article and the commentaries: All of these applied examples of cross-sectional symptom networks are demonstrating unreliable parameter estimates. While the commentaries propose that the resulting differences between networks are not genuine or meaningful because they are not statistically significant, we propose that the unreplicable inferences about the symptom-level relationships of interest fundamentally undermine the utility of the symptom networks.