Different types of data and the validity of democracy measures

Abstract

Different measures of democracy rely on different types of data. Some exclusively rely on observational data, others rely on judgement-based data in the form of in-house coded indicators or expert surveys. A third set of democracy measures combines information from indicators based on different types of data, some of them also data from representative surveys of the mass public. This article discusses the advantages and disadvantages of these different types of data for the measurement of electoral and liberal democracy. The discussion is based on the premise that the main priorities must be to establish a high degree of concept-measure consistency, i.e. indicators capture relevant aspects of the core concept of interest in a precise and unbiased manner, and to provide high coverage. The basic argument of the article is that no type of data is superior to others in all respects. The article draws on examples from extant datasets to illustrate the tradeoffs and it offers suggestions about how to reduce some of the potential drawbacks.