Validity and reliability of Cybersecurity Judgment Questionnaire for middle and high school students: A validation study with Rasch analysis

Abstract

In cybersecurity practice, two methodological challenges exist, that is, how to improve validity and reliability of cybersecurity assessment as a weak hypothesis and how to assess and mitigate cybersecurity risks among the K-12 school population as the weakest link. To address these two challenges, the present study aimed to verify the validity and reliability of one newly developed questionnaire, Cybersecurity Judgment Questionnaire for Middle and High School Students, using the Rasch modeling. A total of 398 middle and high school students from the GenCyber Camps participated in this study. The results of the Rasch model analysis show that (1) the Cybersecurity Judgment Questionnaire was unidimensional, measuring one single construct of cybersecurity judgment; (2) the questionnaire had acceptable item construct validity and person construct validity; (3) the item reliability and person reliability of the questionnaire were also acceptable, and (4) a few items might need revisions to increase item difficulty and reduce redundancy, if this would be confirmed by more replication studies with more representative samples. Suggestions for improvement and administration of the questions are discussed.