Collecting quantitative experimental data from a non-WEIRD population: challenges and practical recommendations from a field experiment in rural Sierra Leone

Abstract

Objective: Standardized pretest–posttest experimental designs with quantitative surveys are frequently applied to evaluate the effectiveness of health programs. However, this method is strongly informed by research on samples from Western, Educated, Industralized, Rich, and Democratic (WEIRD) societies and may not produce meaningful results in a distinct cultural, educational and socioeconomic context. Results: This paper reports several methodological challenges encountered along the research process of collecting quantitative survey data (i.e., during recruitment, obtaining informed consent, matching pretest–posttest data and data collection) for a mixed-methods field experiment on domestic handwashing in Sierra Leone. Ethical dilemmas of certain research practices are pointed out and potential solutions or alternatives are recommended for each challenge. Analysis of these challenges highlights the importance of reflecting on the aptness of research methodologies for non-WEIRD samples. While this is not to say that quantitative surveys are not suitable in a non-WEIRD context, their employment require considerable time for extensive pilot testing, involving local interviewers and participants in designing research projects and the modification of data collection strategies.