Quantifying participant distress: Validity and applicability of a distress measure to evaluate harm in quantitative assessments

Abstract

Structured interviews remain a key approach to collect information from community members, particularly in development contexts. Such enumerated surveys often focus on potentially distressing topics including gender equality, social inclusion, wellbeing, and even socio-demographics. Researchers have an obligation to consider the ethics of survey processes and mitigate potential distress for participants. However, approaches to quantify and evaluate participant distress remain nascent outside of clinical practice. To support ethical considerations in quantitative survey deployment, we introduce a four-item formative measure to analyze interview ease, stress, privacy, and comprehension. We present the measure's conceptual and empirical development and examine the validity of the measure through data from Cambodia and Nepal (n = 4,674) using Partial Least Squares Structural Equation Modeling (PLS-SEM) for formative measurement model assessment. The measure is shown to have content and face validity, anticipated divergence with two reflective constructs, low collinearity, structural validity, and construct validity through known groups testing. As ethical considerations are increasingly recognized as important in research in both development and other research and evaluation contexts, tools to diagnose and analyze distress can support in mitigating negative impacts.