Résumé
Cet article présente et compare deux approches origi-
nales de veille technologique basées sur un paradigme
antagoniste: une approche des sciences de la gestion
(prise de décision multicritère) versus une approche
participative (marché de prédictions). Elles sont toutes
deux destinées à soutenir la gestion d'un portefeuille
technologique ainsi que l'évaluation de nouvelles tech-
nologies dans le cadre d'une organisation active dans
les technologies de l'information. Pour évaluer la per-
tinence de notre recherche, nous avons réalisé plu-
sieurs expériences dans un environnement réel. Les
résultats ont montré que la rigueur des sciences de la
gestion combinée au côté participatif du Web 2.0 était
un atout dans le cadre de la veille technologique. De
plus, un cadre conceptuel a été établi pour comparer
les deux approches.

Mots clefs :
Veille technologique, multicritère, marché de prédic-
tions, Web 2.0



Abstract
This paper presents and compares two original ap-
proaches for technology assessment and foresight
based on opposite paradigm: a management science
approach (Multi-Criteria Decision-Making) versus a
participatory approach (Prediction Market). These
approaches are intended to support the management of
a technology portfolio and the assessment of new tech-
nology by an IT organization. In order to explore the
relevance of the research, we conducted several ex-
periments in real environments. The results demon-
strated that the rigor of management science and the
participation of the Web 2.0 approach are complemen-
tary strengths for technology foresight. Furthermore, a
framework has been established to compare the two
approaches.

Key-words:
Technology foresight, multi-criteria, prediction mar-
kets, Web 2.0

Comparison of Multi-
criteria and Prediction
Market Approaches for
Technology Foresight



Cédric Gaspoz
Information Systems Institute, University of Lausanne,
Lausanne, Switzerland
cedric.gaspoz@unil.ch

Jan Ondrus
ESSEC Business School, Paris, France
ondrus@essec.fr

Yves Pigneur
Information Systems Institute, University of Lausanne,
Lausanne, Switzerland
yves.pigneur@unil.ch

Cédric Gaspoz, Jan Ondrus, Yves Pigneur
2
Introduction
According to McKeen and Smith (2003), one of the criti-
cal issues in IT management is to “situate the challenges
facing the IT managers regarding emerging technology
…”. This requires companies to adopt a systematic proc-
ess to stay up-to-date and assess new technology for a
potential integration into modern organizations.
This paper focuses on two approaches that support the
assessment and foresight of new technology in order to
evaluate how businesses can take advantage of them.
Different management tools and techniques have been
proposed in the scientific community and the literature
(scenario planning, technology roadmap, ROI, real op-
tion) but few of them have been widely adopted by com-
panies.
In this paper, we present and compare two approaches we
designed and evaluated in two recent research projects. In
addition, we also propose a certain number of critical
success factors which makes one or the other approach
more appropriate to be used in certain corporate condi-
tions. The first completed research assumed that a man-
agement science approach, “multi-criteria decision-
making (MCDM)”, is well suited for technology fore-
sight. The second in-progress research investigates a par-
ticipatory approach, based on Web 2.0 tools, “prediction
market (PM)”. We used and validated both approaches
during the assessment mobile payment technologies.
In the next section, we present some work that has been
done in technology forecasting methods comparison.
Section 2 introduces the two explored approaches. In
Section 3, we describe the two designed artifacts, which
support our experiment detailed in Section 4. Section 5
summarizes the results obtained with both approaches. In
Section 6, we use a theoretical framework to compare the
two approaches and provide several key success factors.
Finally, we conclude and propose further research in Sec-
tion 7.
1. Related Work
Several authors studied the choice and the usage of tech-
nological forecasting methods in different types of or-
ganizations. Porter et al. (2003) introduce technology
futures analysis (FTA) as a field grouping all forms of
analyzing future technology and its consequences. After
presenting and classifying more than 50 methods, they
present two scoping issues of TFA: the content issues
(i.e., time horizon, geographical extent, level of detail)
and the process issues (e.g., participants, decision proc-
ess, study duration, resources available).
In his paper, Martino (2002) presents a review of recent
advances in technological forecasting based on eight
methods and shows the resulting possibilities from these
new approaches.
Presenting the implementation issues of technology intel-
ligence systems, Savioz et al. (2001) notes the importance
of the organization specificities in setting up such a sys-
tem.
Levary and Han (1995) identify six main factors affecting
technological forecasting and the choice of a method
(money available for development of technology, data
availability and validity, uncertainty surrounding the suc-
cess of technological development, similarity of proposed
and existing technologies and number of variables affect-
ing the development of technologies). They also studied
the prerequisites for use of specific technological fore-
casting methods.
Lichtenthaler (2005) conducted an exploratory case study
research in leading multinationals that identified the most
influential contingency factors for the selection of tech-
nology intelligence methods and assessment forms.
Lichtenthaler (2004) also presents the importance of the
type of coordination of the technology intelligence proc-
ess (structural, hybrid and informal) as well as the selec-
tion of information sources in the choice of a specific
method.
We found that none of this previous work elaborated a
comparison of selected approaches with their strength
and weaknesses related to their contextual implementa-
tion.
Leonard-Barton (1999) describes a dual methodology for
case studies about the same phenomenon, offering oppor-
tunities for complementary and synergistic data gathering
and analysis.
In this paper, we propose to establish a comparison
framework based on characteristics derived from past
research previously presented. This framework aims at
helping us to compare our two approaches and identify
their key success factors.
2. Presentation of the Approaches
The two selected approaches for our research differ on
many aspects. Before comparing them, we briefly de-
scribe their aim and context of usage.
2.1. MCDM: A Management Science
Approach
MCDM methods aim at supporting decisions in an effec-
tive way by analyzing a problem using either quantitative
(e.g., cost, weight) or qualitative (e.g., quality of service,
beauty) criteria simultaneously and concurrently. The
idea behind MCDM methods is not to find the optimal
solution (like a mathematical programming model) but
rather try to determine what solution is the closest to be
“optimal” in regards of several criteria or among existing
solutions. To collect the data, decision-makers (i.e., ex-
perts) need to express their preferences by evaluating the
alternatives and weighting the criteria.
Previous research indicates that MCDM methods are not
only used for decision-making but also for technology
foresight (Salo et al. 2003). Three distinct phases of the

Comparison of Multi-criteria and Prediction Market Approaches for Technology Foresight
3
decision have been characterized by Simon (1955). These
are intelligence, design, and choice. Bui (1984) argued
that MCDM methods usually focus on the two last
phases. In our case, the objective is to use MCDM meth-
ods for the intelligence phase of the decision process. The
idea is to examine the current environmental conditions
and unveil potential future issues before the establish-
ment of the decision.
2.2. Prediction Markets: An Emerging
Approach
Prediction markets are future trading platforms whose
contracts are ideas rather than goods or services. They
have been used in many different contexts and often pro-
duced more accurate forecasts than traditional methods
(Berg et al. 2003; Spann et al. 2003; Wolfers et al. 2004).
Still considered as an emerging approach, they enable
everybody to trade by aggregating the information dis-
seminated among all actors in a corporate crowd (e.g.,
employees, business partners). Furthermore, they allow
actors to trade based on their own assumptions, without
taking care on the hierarchy or other social pressures.
Hanson (1992) made the assumption that prediction mar-
kets should improve the progress of science based on the
absence of social, economical or political pressures.
Previous research (Gaspoz and Pigneur 2008) showed
that the information disseminated in the crowd was not
equal to the information reported among the hierarchy.
This difference was partially explained by the anonymity
of the traders on the prediction market and by the reward-
ing process, based on the best performances (i.e., the
quality of the information supplied).
3. Design of the Artifacts
In order to support our research, we designed two arti-
facts implementing the MCDM and “prediction market”
approaches. As research methodology, we adopted a de-
sign science paradigm and rigorously followed the rec-
ommendations prescribed by Hevner et al. (2004). We
developed iteratively and incrementally both artifacts
with build-and-evaluate loops. More details about the
artifact implementing MCDM methods can be found in
earlier work (Ondrus et al. 2006). Similarly, the predic-
tion market platform was also described in a previous
communication (Gaspoz and Pigneur 2008).
3.1. MCDM: A Group Decision Support
System
The requirements for a multi-actor multi-criteria analysis
are not easily fulfilled, as a great amount of data has to be
collected, computed, and visualized. Obviously, a digi-
talization of the processes is necessary. In other words,
we decided to use an IT artifact (i.e., a Group Decision
Support System, GDSS) integrated with the processes of
an MCDM approach. As none of the existing MCDM
tools surveyed encompassed the features needed, we de-
signed a new and original prototype with unique charac-
teristics required for our research. We concentrated our
efforts on the development of an interactive user interface
in order to improve data collection, computation, and
visualization.
Our prototype, PylaDESS, implements side-by-side two
formal MCDM methods: ELECTRE I (Benayoun et al.
1966) and the Weighted Sum Model (WSM) of Fishburn
(1967). To collect the data, we selected an interactive
process based on the “Pack of Card” technique proposed
by Simos (1990) and later improved by Pictet and
Bollinger (2003). We programmed this technique in Py-
laDESS in order to facilitate data collection. Experts can
evaluate technologies using a five value scale (i.e., weak
(1), fair (2), average (3), good (4), excellent (5)) for each
criterion they estimate as relevant.
To improve the visualization and analysis of the data, we
implemented many different data cross-analysis modules.
All of these features make PylaDESS a unique MCDM
tool to support multi-actor and multi-criteria analysis.
The iterative and incremental development of the IT arti-
fact was done in laboratory and its testing was organized
in a real environment. The design iterations allowed us to
better manage the different constraints encountered dur-
ing the analysis. In total, three distinctive iterations have
been conducted. First, the artifact has been used in back-
office for manual data input and computation the data.
During the second design iteration, the artifact has been
used in front of the experts to collect the data with card
game and give a real-time feedback of the results com-
puted. The third iteration consisted of using the artifact as
a group support system in roundtable setting. During
each of these iterations, numerous improvements have
been done in order to adapt the artifact for each context
with its constraints.
3.2. PM: e-Trading Market
To develop the prediction market platform, we conducted
three design iterations of the build-and-evaluate loop. We
also used the three Steps for Designing a Virtual Stock
Market from Spann and Skiera (2003) to determine the
requirements of our artifact.
The multiple evaluations of our artifact and the refine-
ments of our design led us to formulate five propositions
to design a prediction market for R&D portfolio man-
agement (Gaspoz and Pigneur 2008). These propositions
were used to design the platform for our current experi-
ment presented in Section 4.
The main specifications instantiated are the use of a spe-
cific ontology in order to allow each trader to acquire the
same comprehension of contracts and claims, coupled
with participative discussions between the participants.
We also implemented an IPO mechanism allowing any
actor to propose new technologies on the market, without
requiring a review process or preliminary validation of
his proposition.

Cédric Gaspoz, Jan Ondrus, Yves Pigneur
4
Due to the fact that most of the participants are not confi-
dent with trading mechanisms and concepts, we removed
almost all financial concepts from the interface in order
to reduce the trader's learning curve.
In order to increase the motivation of the participants, we
designed an experiment which alternates between group
and individual trading sessions. Group sessions are essen-
tial as it allows us to quickly obtain an evaluation of the
technologies because of the high volume of transactions
on the market.
Finally, in line with recommendations of several re-
searchers (Hanson 2003, Pennock 2004, Spann and
Skiera 2003), we implemented an automatic market
maker, allowing the traders to buy or sell when new in-
formation is available. Thus, the market aggregates more
information compared to a double auction market were
the traders have to wait for a corresponding offer to make
the deal.
3.3. Comparison of the Artifacts
The designed artifacts are quite different in their nature.
PylaDESS is a standalone application coded with Python
programming language. It runs on most popular operating
systems (MS Windows, Mac OS X, and GNU/Linux). In
terms of specific algorithms to compute the data, it im-
plements two formal MCDM methods and produce visual
outcomes (i.e., rankings and outranking graphs). More-
over, there are different visualization modules to conduct
cross-data analysis. More details of PylaDESS features
can be found in (Ondrus et al. 2006).
The e-trading market architecture requires a web server
and an Internet connection. The user interface is based on
web standards such as HTML, which is compatible and
reachable with any computer using a web browser. It
supports buy and sell operations and displays current
trading information (e.g., price, volume). The trading
mechanisms and market maker were implemented with
Python scripts based on Hanson’s (2003) algorithms.
4. Settings of the Experiments
To explore our approaches for technology foresight, we
applied them in the field of mobile payments. Based on
previous research (Ondrus and Pigneur 2007), we se-
lected several possible alternatives for future technology
developments in the Swiss mobile payments market.
In order to conduct a foresight process, we assessed cur-
rent payment technologies and added possible future up-
coming technology. By mixing both current and future
technologies, we are able to estimate more precisely the
impacts of future trends based on the existing market
conditions.
For the technology alternatives, we selected three types of
cards: (i) SmartCards (chip-based), (ii) Contactless cards
(RFID-based), and (iii) Magnetic cards (with magnetic
strips). We also included two phone-based technologies,
one using a phone remote network (e.g. GSM, GPRS)
and another one based on phone proximity networks (e.g.
Bluetooth, Infrared). In a second phase we added an up-
coming technology, Near Field Communication (NFC).
This technology is a fusion of the mobile phone and the
contactless card. More precisely, the mobile phone can
act as a RFID tag or reader. More information about
RFID and NFC can be found in (Want 2008).
4.1. MCDM: Visiting Swiss Experts
During a first phase, we assessed the current technologies
present on the Swiss market. We started this phase in
November 2005 and finished it in May 2006. We selected
20 of the major companies involved in payments in Swit-
zerland and visited each of them once or twice; depend-
ing on how much time they could give us.
The structured interviews lasted in average between half
an hour and an hour, sometimes more. In general, we had
between one and three experts representing the compa-
nies. All selected experts were leaders of mobile pay-
ments projects in their respective companies.
During the interviews, we used our computerized “Pack
of cards” technique to elicit the preferences of the ex-
perts. The computerized process enabled direct input in
PylaDESS and a real-time feedback of the results.
The second phase of the research (i.e., NFC assessment),
consisted of a real-time group setting. This roundtable
aimed at inviting all the companies that participated dur-
ing the first phase of the project. 16 experts representing
14 different companies came to the roundtable in October
2006. This roundtable had two distinctive parts. The first
part consisted of a presentation of the previous results
obtained. During the second part, we distributed individ-
ual forms for each expert to evaluate NFC using the five-
value scale, as done before. After having inserted and
computed the data in PylaDESS, we immediately ex-
posed the results to the experts.
4.2. PM: Gathering the Crowd
We ran a prediction market based on the selected mobile
payment technologies with twenty-nine master students
in business information systems. Christiansen (2007)
showed that our crowd size is over the minimum thresh-
old of participation to ensure well-calibrated results. The
one-month experiment took place in May 2008. Twenty
students were active on the platform. We recorded 390
trades representing 6291 shares from four markets con-
taining thirteen claims. Six of these claims were directly
related to the technologies used in the MCDM approach.
The setup of the experiment did not require more than
three working days. This includes the setup of the mar-
kets and user accounts. Furthermore, a presentation of the
platform, its markets and claims was made in class. On
the students’ side, the investment is tightly linked to the
number of trades made during the month. This includes
the research of an investment opportunity based on in-

Comparison of Multi-criteria and Prediction Market Approaches for Technology Foresight
5
formation available to the trader, passing an order and
looking at the new portfolio worth.
The incentive to play on the prediction market was a
prize for the trader with the highest worth at the end of
the experiment. This incentive alone was not sufficient to
have a continuous trading volume on the market, so we
introduced two short-term contracts during the experi-
ment, resulting on trading peaks on the market.
Finally, to insure sufficient trades to extend the market
accuracy, we used two strategies. First, we presented all
markets and claims in details during the class, allowing
students to ask questions on the claims and on related
issues. We completed this presentation with on-line mate-
rial presenting each claim in detail, accompanied with
presentation videos. Second, we used a market-maker to
allow the traders to quickly get their information aggre-
gated on the market.
4.3. Comparison of the Settings
As can be seen in Table 1, the settings for both ap-
proaches differ on several aspects.

MCDM PM
Who Selected experts Students (crowd)
Where One or two individ-
ual interviews with
each company.
+ One roundtable for
all the experts to
meet, discuss the
results and evaluate
NFC
One group meeting
to start the market
and some trading
activities. Later, The
participants continue
to trade alone any-
time and anywhere.
When Nov. 05 – May 06
+ Oct. 06
May 08 (1 month)
How

Several months for
setup, trips, phone
calls, analysis
Few days for setup
and analysis
Table 1. Differences of experiments’ settings

A considerable effort is required for the MCDM ap-
proach compared to the PM approach, especially for the
data collection process. Each company and experts need
to be met individually. The experts need more support
during their elicitation of preferences than the traders,
who just buy or sell.
A multi-criteria analysis requires a relatively great
amount of data to collect. The best way to proceed is to
meet the experts in a face-to-face mode. The advantage of
this direct contact is a personalized assistance and inter-
action during the whole process. This should prevent
erroneous data sets.
In the prediction markets, the participation of the players
is self-organized. This facilitates the overall management
of the analysis. However, the success of the prediction
markets outcome depends on the good willing of the
players to participate and trade without the pressure of
the project managers.
5. Analysis of the results
5.1. MCDM: Ranking and Outranking
From the results obtained, it was quite clear that card
technologies were preferred to phones for payment pur-
poses. The general ranking obtained with the WSM
method shows that cards, especially smartcards and con-
tactless cards, were preferred with a high ranking.
Phone-based solutions remain in last positions of most
rankings. This could be explained as mobile phone-based
payment schemes are still in an early stage of develop-
ment. Our results show that there is still progress to be
made in terms of ease of use, cost, reliability, and
user/market acceptance (i.e., awareness). However,
phone-based schemes already perform well in terms of
flexibility and value proposition improvement. The three
national mobile network operators consider value propo-
sition improvement to be an important aspect, which ex-
plains why they believe that mobile phones have some
future as a payment instrument. Due to space limitation,
we could not describe results in more details. A complete
description of the results of the first phase can be found
in (Ondrus and Pigneur 2007).
During the second phase, the results showed that NFC is
well evaluated. Its ranking is high and comparable to
contactless and smartcards. It is clearly performing better
than the other mobile phone technology tested in the first
phase. A deeper analysis of the results is described in
(Ondrus and Pigneur 2008).
5.2. PM: Price of Contracts
Due to the fact that the students were relatively well in-
formed on this topic and made an intensive use of infor-
mation disseminated, the results are the expression of a
good consensus between the traders. We could observe
that after a period of important variations during the first
two weeks, the prices tended to reach a consensus at the
end of the experiment while the volume of trades stayed
at the same level.
On the Mobile Payment Technologies market, we can
distinguish two claims' groups. The first group composed
of NFC, smartcard and RFID was the most active in term
of trades and all technologies reached a “price” over
50%. The second group gathered claims with few trades
and probabilities under 50%.
Our results indicated that NFC could be considered as the
next successful technology in the mobile payment field.
The price history shows a regular adaptation to reach the
consensus of 57.2%. We also saw a convergence of smar-
tcards and RFID technologies to reach a probability just
above 50%.
On the other end the mobile phone proximity and remote
technologies had only few trades. The reason for this

Cédric Gaspoz, Jan Ondrus, Yves Pigneur
6
disinterest could be the lack of available information or
the lack of confidence from the traders. In any case, the
results of these two claims are not significant.
Finally, magnetic card made a low score, supported by
many trades. We can interpret this result as a clear sign of
the gentle eviction of this technology on the payment
market. Even if the magnetic strips are still available on
most of the cards, these cards also contain a chip, which
put them in the smartcards category.
5.3. Comparison of the results
The results of the prediction market are globally similar
to the ones obtained with the MCDM approach. Table 2
summarizes these results.

MCDM PM
1. SmartCard (3.8/5) 1. NFC (57.16%)
2. NFC (3.6/5) 2. SmartCard (52%)
3. Contactless Card (3.6/5) 3. Contactless Card (52%)
4. Magnetic (3.3/5) 4. Phone proximity (51.20%)
5. Phone proximity (2.7/5) 5. Phone remote (49.51%)
6. Phone remote (2.7/5) 6. Magnetic Card (47.01%)
Table 2. Summary of the results (ranking)

The similarity of the results obtained is essential, as we
want to compare both approaches. Unfortunately, due to
length limitations, we are not able to display more de-
tailed results with interpretations. Nonetheless, the main
purpose of the paper is a theoretical and practical com-
parison of the approaches and their key success factors of
applications in corporate contexts.
6. Comparison and Discussion
To compare our two approaches, we derived a framework
based on the contingency factors developed by Lichten-
thaler (2005) and the individual factors affecting techno-
logical forecasting from Levary and Han (1995).
Lichtenthaler found that the contingency factors influence
the choice of assessment forms and technology intelli-
gence methods used in multinationals. Levary and Han
designed a framework to define the most appropriate
forecasting method(s) for various combinations of the
degree/extent of individual factors affecting technological
forecasting.
The combination of the two groups of factors enables us
to embrace the technological foresight activity globally
and systematically from the organization characteristics
to the information collection through the assessment
process.
The resulting framework contains three main compo-
nents: the organizational factors, the assessment proper-
ties, and the data attributes (Figure 3).
By organizational factors, we mean all factors determin-
ing the environment of the assessment process. These
factors could be the resources availability, the organiza-
tion’s internal communication culture or the decision-
making style.
The assessment properties are the characteristics of the
assessment conducted in a given organization. These
properties could be the assessment’s goal, the time hori-
zon of the prediction or the uncertainty of the assessment
field.

Organizational
Factors
Data
Attributes
Assessment
Properties
Technology
Forecasting
Method

Figure 3. Framework of comparison

Finally the data attributes are the characteristics of the
data needed for the technology forecast like data quality
and availability. We also distinguish between exogenous
and endogenous data collection processes. In the exoge-
nous processes, we do not worry about the provenance of
the data and the channel used to collect them. The en-
dogenous processes imply that we integrate a data collec-
tion process in the method.
6.1. Organizational factors
These factors are specific for every organization. Even if
they are not directly related to the assessment made, they
will define its conditions and modalities. Often, they are
implicitly embedded in the choice of a method, excepted
for the resources. Time, human or financial resources
dictate more or less the conditions of the assessment. In
the case of limited resources, familiarity with the various
methods will play an important role in restricting the
choice of options.
The MCDM approach is well suited for organizations
with formal and less participatory decision-making proc-
esses. This approach relies mainly on some selected ex-
perts at the expense of the crowd. MCDM methods may
be difficult to implement in more participatory organiza-
tions, as the number of possible participant is limited for
practical reasons. Likewise, the experts need a good
knowledge of the method, both for the assessment and the
interpretation of the results.
To make an efficient use of prediction markets, the or-
ganization must have a participatory and informal deci-
sion-making style. We need to open the market to the
most players in order to aggregate more information. Due
to their design, prediction markets does not require in-

Comparison of Multi-criteria and Prediction Market Approaches for Technology Foresight
7
depth knowledge of the method. Participants just have
two possible actions: buy or sell. Furthermore, the results
are quite simple to interpret. Given the short implementa-
tion time of this method, it is well suited for fast moving
organizations or for organizations with limited resources.
A challenge is to get participants to actively and regularly
trade on their own. Otherwise, the results obtained might
not be significant.
In the MCDM approach, the actors involved are usually a
set of selected and relevant experts who are motivated to
participate in order to get access to the data and therefore
knowledge that would augment their expertise.
In prediction markets, the participants are anybody inter-
ested in technology but are not always experts (“the
crowd of Web 2.0”). They constitute a community of
players who are driven by the game and its financial prof-
its. As opposed to the MCDM approach, the prediction
markets can easily indicate if players are good by consid-
ering the value of their portfolio and their total profit.
6.2. Assessment properties
The main property is the goal, which specifies whether to
assess the current environment or to generate knowledge
about the future. Properties also describe the nature of the
information to be generated. Depending on the needs, we
might require a static or dynamic picture of the trend
studied.
The MCDM approach gives a posteriori results to support
the resolution of a decision problem. At a specific time,
the MCDM analysis draws a rather detailed picture of a
situation benefiting from the granularity provided by the
criteria. These criteria help explaining precisely the rea-
sons of the outcome.
On the contrary, prediction markets are excellent tools for
longitudinal studies due to the inherent nature of the data
collection process. However, they give the prediction
(i.e., the claim’s price) without further explanations. In
other words, MCDM methods are detailed snapshots
taken at certain times and prediction markets are movies
shot over a period of time, suitable for assessments re-
quiring frequent or permanent updates
6.3. Data attributes
In MCDM, the data collection process is endogenous
since experts elicit their preferences using criteria and
alternatives previously established. As a result, a double
risk of bias exists during the establishment of the criteria
and alternatives and during the elicitation of the prefer-
ences. As the method cannot identify any bias introduced
by experts, it may be necessary to couple MCDM with a
Delphi analysis to avoid having too large disparities.
In the case of prediction markets, the data collection
process is exogenous. Full interest is given to the assess-
ment. The rest of the process is left to the crowd. Predic-
tion markets are not affected by unreliable information,
due to the aggregation mechanism. Prediction markets are
well suited in cases when information is not available or
potentially unreliable.
6.4. Key Success Factors
Based on the comparison, we propose some key success
factors for MCDM and prediction markets applied in
technology foresight. Our recommendations should sup-
port further explorations of these approaches.
MCDM methods are well suited for situations when a
group of relevant experts want to confront their opinions
in order to unveil weak signals of technology trends. On
their side, prediction markets need a crowd ready to trade
and share their beliefs. Their actions generate a prediction
through an implicit data aggregation mechanism relying
on information disseminated among the crowd. This
works particularly well when the corporate crowd is fa-
miliar with the topic.
To setup an MCDM analysis, a facilitator should be hired
to meet each expert individually. Face-to-face meetings
are essential to share the results, as they are usually cen-
tralized in standalone software. Prediction markets only
need a facilitator who can setup a claim on the platform.
Then, traders can play anytime and anywhere using a web
browser. The major challenge of prediction markets is to
gather a motivated crowd, which trades regularly.
The efforts required for the MCDM approach are re-
warded with insurance that the set of data collected is
valid since the facilitator supervises the whole process.
To overcome this issue in prediction markets, the crowd
automatically regulates the market. Even if a trader intro-
duces a bias in the market by doing irrational actions, the
crowd would neutralize him/her by doing opposite ac-
tions. At some point, the defective trader will be evinced,
as his/her financial resources to trade would vanish.
MCDM methods are used when experts need to have a
precise explanation of the phenomenon. The criteria,
weights, and evaluations are useful indicators for unveil-
ing possible weak signals. In our case, the results were
rankings and outranking graphs. Looking at the data col-
lected, we could explain precisely how we reached these
outcomes. As a result, the establishment of a consensus
could be reached after several rounds of analysis (i.e.,
Delphi). Prediction markets’ outcome is by nature a con-
sensus of the crowd based on many rounds of trades. The
aggregated results provide a simple but powerful indica-
tion of the probability that an event would occur. In addi-
tion, one can analyze the evolution of the trends by just
looking at the history of price traded. However, it is much
harder to explain the behavior of the traders over time.
7. Conclusion
Despite similar results, both approaches revealed some
benefits and demonstrated their complementarity. On one
side, the MCDM approach brought an analytic explana-
tion of the phenomenon by a controlled and criteria-based
evaluation. On the other side, prediction markets provide

Cédric Gaspoz, Jan Ondrus, Yves Pigneur
8
a synthetic aggregation of numerous individual beliefs
that is constantly adjusted and made available for every-
one. Therefore, we could not claim that one is better than
the other. Interestingly, we found that the drawbacks
identified could partially be solved by opting the best
aspects of both approaches.
For example, we could take consecutive snapshots during
a given period of time to follow trends using a MCDM
approach. Moreover, after few rounds of analysis, we
could improve the data collection process by building an
online user interface which would support the elicitation
of the preferences without a face-to-face confrontation.
For prediction markets, the quality of the players could
be ensured by opening the markets only to a practice
community with its experts. Furthermore, the outcome of
prediction markets could be enhanced by requesting more
information about the actions of the players. The objec-
tive would be to monitor the behavior of the players in
order to confirm that they are not just following the trend
generated by the market.
In this paper, we presented two different promising ap-
proaches for technology foresight. We found that the
combined strengths of the MCDM approach and predic-
tion markets could be exploited for technology assess-
ment and foresight to improve IT investment decisions.
In order to compare our two approaches, we built a
framework that contains essential dimensions to differen-
tiate technology foresight methods. Using this framework
enabled us to derive several key success factors for each
of our approaches.
For further research, we propose to extend this research
by improving our current framework and compare other
technology foresight approaches.
References
Benayoun, R., Roy, B., and Sussmann, B. (1966)
“Manuel de référence du programme ELECTRE”.
Note de synthèse, formation n.25, Direction scienti-
fique SEMA, Paris.
Berg, J.E., and Rietz, T.A. (2003), "Prediction Markets as
Decision Support Systems," Information Systems
Frontiers (5:1) 2003, pp 79-93.
Bui, T. (1984), "Building Effective Multiple Criteria De-
cision Models: A Decision Support System Ap-
proach," Systems, Objectives, Solutions (4), pp 3-16.
Chien, C.F. (2002), "A portfolio-evaluation framework
for selecting R & D projects," R&D Management
(32:4), pp 359-368.
Christiansen, J.D. (2007), "Prediction Markets: Practical
Experiments in Small Markets and Behaviours Ob-
served" The Journal of Prediction Markets (1:1),
2007, pp. 17-41.
Cooper, R.G., Edgett, S.J., and Kleinschmidt, E.J. (1999),
"New product portfolio management: practices and
performance," Journal of Product Innovation Man-
agement (16:4), pp 333-351.
Fishburn, P. C. (1967) “Additive utilities with incomplete
product set: Applications to priorities and assign-
ments”. Operations Research Society of America
(ORSA), Baltimore, MD, USA.
Gaspoz, C., Pigneur, Y. (2008) “Preparing a Negotiated
R&D Portfolio with a Prediction Market”, In: Pro-
ceedings of the 41st Hawaii International Conference
on System Science (HICSS-41), IEEE Computer So-
ciety
Hanson, R. (1992), "Idea Futures: Encouraging an Hon-
est Consensus," Extropy (3:2) 1992.
Hanson, R. (2003) Combinatorial Information Market
Design, Information Systems Frontiers, (5:1), pp.
107-119.
Hevner, A.R., March, S.T., Park, J., and Ram, S. (2004)
"Design Science in Information Systems Research,"
MIS Quarterly (28:1), pp 75-105.
Leonard-Barton, D. (1990) "A Dual Methodology for
Case Studies: Synergistic Use of a Longitudinal Sin-
gle Site with Replicated Multiple Sites," Organiza-
tion Science (1:3), pp. 248-266
Levary, R. R. and Han, D. (1995) “Choosing a Techno-
logical Forecasting Method”, Industrial Manage-
ment, (37:1), pp. 14–18.
Lichtenthaler, E. (2004) “Technology intelligence proc-
esses in leading European and North American mul-
tinationals”, R&D Management, (34:2), pp. 121-135.
Lichtenthaler, E. (2005) “The choice of technology intel-
ligence methods in multinationals: towards a contin-
gency approach”, International Journal of Technol-
ogy Management, (32:3-4), pp. 388-407.
Martino, J. P. (2003) “A review of selected recent ad-
vances in technological forecasting”, Technological
Forecasting and Social Change, (70: 8), pp. 719-733.
McKeen, J., and Smith, H. (2003), Making IT Happen -
Critical Issues in IT Management, Wiley, pp. 366.
Ondrus, J., Pigneur, Y. (2008) “Near Field Communica-
tion: an assessment for future payment systems”, In-
formation Systems and E-Business Management,
forthcoming
Ondrus, J. and Pigneur, Y. (2007) "Cross-industry prefer-
ences for mobile payments development in Switzer-
land," Electronic Markets, (17:2), 2007, pp. 142-152.
Ondrus, J., Monzani, J.-S., and Pigneur, Y. (2006) “A
GDSS for Visualizing and Assessing a Technology
Environment“, In Proceedings of the 12th Americas
Conference on Information Systems (AMCIS 2006),
4-6 Aug 2006, AIS
Pennock, D. M. (2004) “A dynamic pari-mutuel market
for hedging, wagering, and information aggregation”,
In Proceedings of the 5th ACM conference on Elec-

Comparison of Multi-criteria and Prediction Market Approaches for Technology Foresight
9
tronic commerce, ACM Press, New York, USA, pp.
170-179.
Pictet, J. and Bollinger, D. (2003) Adjuger un marché au
mieux-disant. analyse multicritère, pratique et droit
des marchés publics. Presses polytechniques et uni-
versitaires romandes, Lausanne.
Porter, A. L. et al. (2004) “Technology futures analysis:
Toward integration of the field and new methods”,
Technological Forecasting and Social Change,
(71:3), pp. 287-303.
Salo, A., Gustafsson, T., and Ramanathan, R. (2003)
"Multicriteria methods for technology foresight,"
Journal of Forecasting (22:2), pp 235-255.
Savioz, P., A. Heer, and H.P. Tschirky, (2001) "Imple-
menting a Technology Intelligence System: key is-
sues", Portland International Conference on Man-
agement of Engineering and Technology (PICMET),
Portland.
Simon, H.A. (1955) A Behavioral Model of Rational
Choice. Quarterly Journal of Economics (6), 1955,
pp.99-118.
Simos, J. (1990) Evaluer l’impact sur l’environnement.
Presses polytechniques et universitaires romandes.
Spann, M., and Skiera, (2003, B. "Internet-Based Virtual
Stock Markets for Business Forecasting," Manage-
ment Science (49:10).
Want, R. (2008), “An introduction to RFID technology”
IEEE Pervasive Computing (5:1), pp. 25-33.
Wolfers, J., and Zitzewitz, E. (2004), "Prediction Mar-
kets," Journal of Economic Perspectives (18:2), pp
107-126.