Validity and Reliability of a Site-Level Assessment Questionnaire to Assess Nutrition and Physical Activity Practices in Schools

Abstract

Background: Schools are an ideal setting for policy, systems, and environmental approaches to obesity prevention. Although school health environment assessments exist for planning purposes, we developed and tested a comprehensive questionnaire that is suitable for both evaluation and planning. Methods: Reliability was measured by comparing data collected by school personnel from low-income elementary schools across California at two time points, an average of 2 months apart (n = 23). To assess convergent validity, school responses were compared with the responses completed by the research team (n = 28). A weighted kappa test statistic and percent agreement were calculated for each question and specific groups of questions (questionnaire section, item topic, and response type). Results: Test/retest reliability of the questionnaire yielded kappa statistics that ranged from-0.14 to 1.00 (interquartile range [IQR] 0.36). Percent agreement for reliability ranged from 34.78 to 100.00 (IQR 21.7). Kappa statistics for validity ranged from-0.14 to 1.00 (IQR 0.44). Percent agreement for validity ranged from 14.29 to 100.00 (IQR 39.2). Based on these findings, the tool was revised. Conclusions: Study findings indicate that the Site-Level Assessment Questionnaire as tested is a reliable and accurate instrument for use in low-income elementary schools. Revisions may have improved the validity and reliability. We therefore recommend either version for use to support low-income schools in their efforts to assess needs, evaluate progress, and create action plans; and to supply high-quality, aggregable data for large-scale analysis. Additional testing is recommended to validate the revised version, increase generalizability, and determine sensitivity to detect change over time.