9Analysis and Evaluation
of E-Consultations
Angus Whyte
Napier University
Ann Macintosh
Napier University
ABSTRACT
E-democracy relates electronic delivery of the processes of democratic representation to practices
of communication, consultation, and participation in public decision-making. The evalua-
tion of e-democracy initiatives has not developed as quickly as public debate about the poten-
tial impacts. We review criteria for the political evaluation of e-consultations, technical
evaluation considerations, and social research methods that have been used to analyze e-con-
sultation outcomes. We argue that socio-technical evaluation needs to encompass a range of
qualitative and quantitative methods if interdependencies between the political, technical and
social conditions for success are to be better understood. We briefly compare field experiment
and case-study methodologies as a basis for evaluating e-consultation as a policy instrument
intended to enhance public participation in policy making. We suggest that case study
approaches are more suitable given the lack of previous research, the need for policy-makers to
understand the contexts that underpin successful e-consultation, and the needs for further
innovation.
Keywords: e-consultation, e-democracy, participation, evaluation, methodology
INTRODUCTION
Governments around the world are attempting to broaden democracy by providing an
effective conduit between themselves and civil society organizations and between citizens
themselves using innovative ICT to deliver more open and transparent democratic deci-
sion-making processes. In other words, developing e-democracy systems. A growing
number of publications indicate the emergence of e-democracy as a field of research and
practice, for example, Hague and Loader (2000) and the CACM (2001). In common
with these and others we define the term e-democracy as the use of ICT to support the
© 2003. e-Service Journal. All rights reserved. No copies of this work may be distributed
in print or electronically without express written permission from Indiana University Press.

10
e-Service Journal
democratic decision-making process, enabling democratic political participation (Mac-
intosh et al., 2002).
It is argued that democratic participation must involve both the means to be
informed and the mechanisms to take part in decision-making. Over the last decade
there has been a growing awareness of the need to consider new tools that enable a wider
audience to contribute to policy debates and make broader and deeper contributions—
enhancing deliberation so as to better inform and influence the policy process. Van Dijk
(2000) describes the potential of information and communication technology to deliver
e-democracy that addresses these joint perspectives of informing and participating. Gov-
ernments are increasingly responding to this agenda and leading developments. Much
research has been conducted on how to conduct written consultations and some coun-
tries have developed best practice guidelines for this (e.g., UK Cabinet Office, 2000).
The Report of the Government Commission on Swedish Democracy (2000) looks
beyond traditional consultation mechanisms and emphasizes the need to increase citi-
zens’ participation, to influence the development of society in the 21st Century.
The Organisation for Economic Co-operation and Development (OECD, 2001)
define three types of interaction, namely:
Information: is a one-way relationship in which government produces and
delivers information for use by citizens. It covers both “passive” access to
information upon demand and delivers information for use by citizens and
“active” measures by government to disseminate information to citizens.
Consultation: is a two-way relationship in which citizens provide feedback
to government. It is based on the prior definition of information. Govern-
ments define the issues for consultation, set the questions and manage the
process, while citizens are invited to contribute their views and opinions.
Active participation: a relationship based on partnership with government
in which citizens actively engage in defining the process and content of pol-
icy-making. It acknowledges equal standing for citizens in setting the
agenda, proposing policy options and shaping the policy dialogue—
although the responsibility for the final decision or policy formulation rests
with government.
These distinctions are useful because they indicate a scale of ‘engagement’ in policy-
making along which current e-democracy initiatives could be plotted. Thus the OECD
report concludes that “efforts to engage citizens in policy-making on a partnership basis
are rare, undertaken on a pilot basis only and confined to a very few OECD countries”
(ibid.) (for example, see Macintosh et al., 2002). It seems that much current e-democracy
work concerning citizen-representative dialogue that is initiated by administrations can
be categorized as e-consultation, which is our focus here.