Unpacking the Quantifying and Qualifying Potential of Semi-Open Job Satisfaction Questions through Computer-Aided Sentiment Analysis

Abstract

Despite their suitability for mitigating survey biases and their potential for enhancing information richness, open and semi-open job satisfaction questions are rarely used in surveys. This is mostly due to the high costs associated with manual coding and difficulties that arise when validating text measures. Recently, advances in computer-aided text analysis have enabled researchers to rely less on manual coding to construct text measures. Yet, little is known about the validity of text measures generated by computer-aided text analysis software and only a handful of studies have attempted to demonstrate their added value. In light of this gap, drawing on a sample of 395 employees, we showed that the responses to a semi-open job satisfaction question can reliably and conveniently be converted into a text measure using two types of computer-aided sentiment analysis: SentimentR, and Linguistic Inquiry and Word Count (LIWC) 2015. Furthermore, the substantial convergence between the LIWC2015 and, in particular, SentimentR measure with a closed question measure of job satisfaction and logical associations with closed question measures of constructs that fall within and outside job satisfaction’s nomological network, suggest that a semi-open question has adequate convergent and discriminant validity. Finally, we illustrated that the responses to our semi-open question can be used to fine-tune the computer-aided sentiment analysis dictionaries and unravel antecedents of job satisfaction.