Using Machine Learning to Predict Inappropriate Respondents

Abstract

Results of an analysis of survey data which includes data from inappropriate respondents (respondents who do not devote an appropriate amount of attentional resources when answering questions or whose answers for two questions are contradictory) are un-trustworthy. To address this problem, an instructional manipulation check or directed questions scale can be used to identify such respondents. However, survey companies are not willing to use such tools for ethical reasons. In the present study, using eleven machine learning models and six exploratory variables, a prediction model which can judge whether a respondent is inappropriate is developed. The model shows that two explanatory variables, the maximum number of consecutive items on a scale to which a respondent answered with the same response option and response time, are effective for the prediction. The model can reduce the percentage of inappropriate respondents in the analyzed data, which leads to an improvement in the trustworthiness of the analysis results.