Thematic and textual analysis methods for developing social validity questionnaires in applied behavior analysis

Abstract

Social validity is often defined as the degree to which an intervention has value to the community that it effects, but it is seldom reported in the literature. Most social validity questionnaires are purposely created by the authors of the study and often lack a description of scale development process. The purpose of this study was to evaluate methods for the development of a Likert-type social validity scale. Caregivers of children who took part in a study on behavioral treatments for pediatric feeding disorders were part of an initial interview to inform scale development. We analyzed interviews using thematic (qualitative) analysis and textual (quantitative) analysis, and used the resulting themes to generate items for two social validity questionnaires. We examined the inter-rater reliability of the questionnaire development process and evaluated the content validity of the questionnaires resulting from each method. Textual analysis had higher inter-rater reliability for producing themes that could be converted to questionnaire items. The textual analysis method produced a questionnaire with content validity equal to that of the thematic analysis method. The study demonstrates the successful use of a quantitative approach to the development of social validity questionnaires for behavioral interventions.