Designing an experiment on recognition of political fake news by social media users: factors of dropout

Abstract

Although social networking sites (SNS) offer functionalities for large-scale online research, user behavior and, in particular, scale and factors of their dropout from SNS-administered research have hardly been studied. In this paper we present an SNS-based experiment and survey tool and report the results of our investigation of user dropout from a research that uses this tool. This research is a pilot stage of a cross-country comparative study of political fake news recognition. At this stage Facebook and Vkontakte users from Russia have been recruited via SNS ad managing systems, asked to evaluate the truthfulness of the displayed news items and to answer a number of questions. We find that although we had to perform thousands of ad displays, among those who clicked the ad dropout rate was 60 and 65% in Vkontakte and Facebook respectively. 1,816 complete questionnaires were collected within a few days. More educated respondents, people living in or near megalopolises and those who agreed to grant access to their Vkontakte account data were significantly more inclined to complete the survey, but the major predictor of dropout was high individual speed – an indicator of low interest. Neither device type (mobile vs desktop) nor the number of questions per screen (one vs two) affected dropout. The number of leavers declined from the first to the last screens of our tool, but transition from the experiment to the survey and demographic questions produced clear peaks in the dropout curve.