The proportion of non-depressed subjects in a study sample strongly affects the results of psychometric analyses of depression symptoms

Abstract

Background Recent studies have uncovered a peculiar finding: that the strength and dimensionality of depression symptoms’ inter-relationships vary systematically across study samples with different average levels of depression severity. Our aim was to examine whether this phenomenon is driven by the proportion of non-affected subjects in the sample. Methods Cross-sectional data from the “Cohort Study on Substance Use Risk Factors” was analyzed. Self-reported depression symptoms were assessed via the Major Depressive Inventory. Symptom data were analyzed via polychoric correlations, principal component analysis, confirmatory factor analysis, Mokken scale analysis, and network analysis. Analyses were carried out across 22 subsamples containing increasingly higher proportions of non-depressed participants. Results were examined as a function of the proportion of non-depressed participants. Results A strong influence of the proportion of non-depressed participants was uncovered: the higher the proportion, the stronger the symptom correlations, higher their tendency towards unidimensionality, better their scalability, and higher the network edge strengths. Comparing the depressed sample with the general population sample, the average symptom correlation increased from 0.29 to 0.51; variance explained by the first eigenvalue increased from 0.36 to 0.56; fit measures from confirmatory one-factor analysis increased from 0.81 to 0.97; the H coefficient of scalability increased from 0.26 to 0.48; and the median network edge increased from 0.00 to 0.07. Conclusions Results of psychometric analyses vary substantially as a function of the proportion of non-depressed participants in the sample being studied. This provides a possible explanation for the lack of reproducibility of previous psychometric studies.