What goes around comes around: an in-depth analysis of how respondents interpret ISP non-/compliance questionnaire items

Abstract

Purpose: Research on employee non-/compliance to information security policies suffers from inconsistent results and there is an ongoing discussion about the dominating survey research methodology and its potential effect on these results. This study aims to add to this discussion by investigating discrepancies between what the authors claim to measure (theoretical properties of variables) and what they actually measure (respondents’ interpretations of the operationalized variables). This study asks: How well do respondents’ interpretations of variables correspond to their theoretical definitions? What are the characteristics of any discrepancies between variable definitions and respondent interpretations? Design/methodology/approach: This study is based on in-depth interviews with 17 respondents from the Swedish public sector to understand how they interpret questionnaire measurement items operationalizing the variables Perceived Severity from Protection Motivation Theory and Attitude from Theory of Planned Behavior. Findings: The authors found that respondents’ interpretations in many cases differ substantially from the theoretical definitions. Overall, the authors found four principal ways in which respondents interpreted measurement items – referred to as property contextualization, extension, alteration and oscillation – each implying more or less (dis)alignment with the intended theoretical properties of the two variables examined. Originality/value: The qualitative method used proved vital to better understand respondents’ interpretations which, in turn, is key for improving self-reporting measurement instruments. To the best of the authors’ knowledge, this study is a first step toward understanding how precise and uniform definitions of variables’ theoretical properties can be operationalized into effective measurement items.