Children’s Opinion of Retrospective Pre-Post ‘Then-Test’ Survey Validity

Abstract

Background: Over the past forty years there have been a number of studies conducted to compare traditional pre-post surveys (pretest-posttests; administered in two stages before and after an intervention) with retrospective pre-posts (thentests or pre-then-post-tests; administered after intervention only, with participants asked to reflect back to complete the ‘pre’ retrospectively). These previous studies have been with adult respondents and overwhelmingly quantitative.     Purpose: This paper examines children’s perspectives regarding traditional and retrospective pre-post self-report subjective surveys.   Setting: A school-based program run by a community services organisation in southeastern Melbourne.   Intervention: Both pre-post survey types were administered to sixty children attending a pro-social skills group run by a community services organisation in southeast suburban Melbourne.   Research Design:  Twenty children participated in eight small focus groups after completing the surveys. Each focus group was guided by three semi-structured questions, ran for 10-15 minutes, and had 2-3 participants. This research included an observation component as the researcher was present at the final session when the post surveys were completed. The research also utilised the quantitative findings from the surveys to check alignment with findings from the extant literature.   Data Collection and Analysis: Focus groups & qualitative analysis   Findings: The traditional and retrospective surveys confirm that the commonly recorded phenomenon of response shift in adults also occurs with children. Children comment that they prefer the retrospective test, identifying concerns that support theories discussed in the extant literature such as experience limitation, impression management, implicit theory of change, and memory recall.