What is the meaning of significance – healed or only slightly improved? A comparison of different evaluation methods to assess depressive symptoms in ambulant rehabilitants

Abstract

Background: To assess treatment success of depressive symptoms, different statistical methods are available. Effect sizes, percentage improvement, significant improvement, and clinical significant improvement are established methods to assess treatment effects based on “patient reported outcomes”. Aim of the study: How do treatment effects differ according to the used method? Methods: For N = 3018 patients treated in an ambulant psychiatric rehabilitation clinic, improvements were calculated based on effect sizes, percentage improvements, significant and clinical significant changes in Beck Depression-Inventory (BDI-II). Results: For the total sample, a sound medium effect of d = 0.62 was found, after exclusion of patients that were not depressed (euthymic) at time of admission, a large effect (d = 0.79) was calculated. The strongest informative value for symptom reduction of depression was found for the method of clinical significance, applying a cutoff score of 13 a total of 23% were classified as clinicalclinically significant and additional 19.3% as significantly improved. The method of percentagedpercentage improvement resulted in 30.4% of patients that achieved a value of at least 50%. Conclusions: Effect sizes give a rough overview over treatment success of a collective of patients, without consideration of the final state and the variability within the collective. The percentage improvement seems to be better suitable for examination of treatment response. With regard to clinical significance, results provide the most transparent information about significant improvements and change from dysfunctional to functional values.