Improving Explicit Preference Entry by Visualising Data
Similarities
Kris Jack and Florence Duclaye
Orange Labs
2 av. Pierre Marzin – 22307 Lannion Cedex - France
{kris.jack, florence.duclaye}@orange-ftgroup.com

ABSTRACT
The explicit entry of preferences into personalised systems,
such as recommenders, can be tiresome. Users often report
boredom and quickly lose patience. This research presents
and evaluates a new interface which allows users to build
up their profile in a visual environment. The interface
visualises data similarities using a radial tree format. These
data similarities are automatically found by the normalised
Google distance metric. A group of participants are
recruited to test the effectiveness of the new interface. In
one test condition, participants manipulate a graph that is
organised using the similarity metric while in the other
condition they work with an unorganised graph.
Participants enter 34% more positive preferences (those that
suggest a liking) when using the organised graph compared
to the unorganised graph. Rather than reporting boredom,
participants enjoy entering their preferences using the
interface. A perceived reduction in cognitive effort is also
reported when using the organised graph.
Author Keywords
Graphical User Interfaces, Preferences, Similarity Metric,
Recommender Systems.
ACM Classification Keywords
H5.1 [Models and Principles]: User/Machine Systems –
human factors, software psychology; H.5.2 [Information
Interfaces and Presentation]: User Interfaces – graphical
user interfaces (GUI).
INTRODUCTION
Recommender systems attempt to offer items to users that
they judge will be appreciated. The quality of these
recommendations, however, is largely constrained by the
data at the system’s disposal. These data typically describe
a user’s preferences [16], domain-specific ‘general
knowledge’ [1], the items that can be recommended [12] or
previous users’ opinions of items [2]. The acquisition of
such data is, therefore, of central importance.
A hybrid content-based and collaborative filtering
recommendation system is under construction that exploits
all four of these data types. The system will be
implemented in the domain of cinema, providing users with
movie-based recommendations. This article focuses on
how data that describe users’ preferences can be acquired.
In collecting user preferences, recommender systems often
ask users to enter some details about themselves such as
their interests and to mark their appreciation of a given
number of items. To reduce the burden of entering too
many details, user modelling techniques, such as
collaborative filtering [2], exploit commonalities among
individuals, and predict their interests based on few explicit
preference entries. It remains preferable, however, to allow
users to engage with the system and to be able to exert
control when they desire to do so. Such interaction can
correct incorrect system decisions and help the user to
better understand the system. Explicitly entering
preferences, however, can be boring and time-consuming,
being acceptable only when the rewards are worthwhile
[17].
This paper presents a new approach to creating interfaces
for collecting user preferences. The approach is two-fold.
In the first step, domain-specific ‘general knowledge’ is
derived in terms of item similarities. Rather than asking a
human designer to organise the data in the graph, a robust
similarity metric is applied that tends to place similar items
in close proximity to one another. In the second step, items
are visualised graphically for the user to interact with.
Users can navigate within the graph and explicitly change
their preferences for the items displayed. A user evaluation
is conducted to determine the usefulness of the interface.
Results are discussed with interest in the preferences that
are elicited and the users’ perceived cognitive effort.
BACKGROUND
The recommendation system under construction creates a
profile for each user of the system. Such profiles contain
different types of preferences. Preferences have a precise
notion in several fields, notably decision theory [5],
philosophy [7] and Artificial intelligence [20]. As such, it

is worth being explicit about the types of preferences that
are stored here. First, monadic preferences can be stored,
representing a user’s like or dislike of an item or item
attribute e.g. “I like Tim Burton”, “I dislike horror movies”
and “I love Kill Bill”. Five amplitudes of sentiment can be
expressed from love, like, neutral, dislike and hate. In
addition, dyadic preferences can be asserted such as “I like
comedies more than dramas”. The elicitation of monadic
preferences is focussed upon in this article.
One method of instantiating user profiles is to ask users to
explicitly comment on items and item attributes e.g. “I love
comedies”. Any interface that demands such an action
from a user can be referred to as an explicit preference
entry (EPE) interface. Many examples of EPE interfaces
can be found in recommendation system literature, from
questionnaire-based entry forms (e.g. [11]) to dialogue
systems (e.g. [9]). MineKey (www.minekey.com), a blog
recommender, and StumbleUpon (www.stumbleupon.com),
a web site recommender, ask users to indicate their
preferences with respect to general topics (e.g. sports,
hobbies, arts). MovieLens, similarly, asks users to give
ratings to a number of films (equivalent to expressing
appreciation) that are presented in a list. The advantage of
such EPE interfaces is that users are given control over the
creation of their profile. They can explicitly create their
preferences and, in the case when some form of user
modelling is in place, correct any mistakes that may have
been made by the system (e.g. Wall Street Journal article
entitled “If TiVo thinks you gay, here’s how to get it
straight”).
EPE interfaces, however, can make the preference entry
process tedious and completing large questionnaires that
ask seemingly repetitive questions can be off-putting to
users. While entering a rating for 5 movies may be
acceptable, entering ratings for 10, 15 or 20 movies may
stretch the patience of most users. A study by Swearington
and Rashmi [17] suggests that users are only willing to put
more effort into teaching a system about themselves if they
see significant increases in the system’s performance.
Users, above all, should enjoy entering their preferences.
This problem has been largely addressed by the use of
collaborative systems where users freely offer their
preferences [15], and sometimes pay to do so [19]. Such an
approach, however, often requires large networks of users
and a substantial time investment in order to collect useful
datasets.
As an alternative, EPE interfaces can be designed using
data visualisation techniques in order produce more user
friendly interfaces. The use of data visualisation is not new
in recommender systems. Both Music Plasma
(www.musicplasma.com) and Amaznode
(amaznode.fladdict.net/) have shown how recommendations
can be attractively visualised to the user. Such
recommendations are useful as users can quickly see how
closely items are related to one another by their proximity
on the visual representation. Similarly, data visualisation
has been used to display user profiles in recommendation
systems [10].
Recommendation systems are clearly constrained by the
data available to them. For example, to take the extreme
case of a user who asks for a recommendation when
nothing is known about them, the best that the system can
do is to appeal to the notion of consensus relevancy rather
than personal relevancy [13]. That is, the system can
produce recommendations that suit most people, but cannot
personalise them for the individual. As the system builds a
user profile, whether this is through implicit or explicit
means, it can produce more personalised recommendations.
Most recommendation systems, however, do not seek user
data at random. System designers must choose which
preferences are most useful to elicit. For example, in a
movie recommendation system, such as MovieLens, it is
more useful to ask users about their movie preferences from
a wide range of movies rather than just one type. If,
instead, users were asked only about horror movies then
their profiles may not be very informative. Similarly, if the
system only asked a user to rate actors who they liked and
did not ask them to rate those who they did not like then
their profiles would likely be biased. Example of both liked
and disliked actors are likely to lead to a more informative
user profile.
This article considers the use of data visualisation
techniques in producing a user-friendly EPE interface.
Using the interface, a user should be able to enter their
monadic preferences such as “I like Tim Burton” without
encountering the typical problem of boredom. The
proposed interface is now detailed before evaluating its
effectiveness.
VISUALISING DATA SIMILARITY IN AN EPE INTERFACE
EPE interfaces should encourage users to enter their
preferences and should be enjoyable to use. The
preferences that are entered can then be exploited by a
recommendation system in order to construct the user’s
profile. An EPE interface has been designed that visualises
the attributes of items in the domain of cinema. In this
case, these attributes are actors who star in movies. It
should be noted that while the system has been tested in the
domain of cinema, the EPE generation techniques that have
been selected are robust and applicable in several domains.
The visualisation of actors is completely automated. The
system takes a list of strings, in this case a list of actors’
names, as input. Using a similarity metric, the system
calculates how similar each actor is to all other actors in the
list. Actor similarities are then visualised as physical
distances on the computer screen with actors who are
similar to one another appearing in close proximity and
those who are dissimilar appearing further apart.
The selected similarity metric is first introduced before
presenting the data visualisation technique. The EPE

Table 1. The number of Google hits for actor pairs and
their normalized Google distances given in brackets.
Jackie Chan and Bruce Lee are closer to one another than
to Janne Fonda.
interface itself is then described. The interface allows users
to interact with the visual graph of actors in order to enter
their preferences.
Similarity Metric
The notion of similarity is subjective. Given two people,
for example Bruce Lee and Jackie Chan, a number of
similarities can be drawn. To name just a few, they are
both actors, they have both starred in several martial arts
movies and they were both raised in Hong Kong.
Similarity metrics can use such information to determine
how similar two items are based upon their features [2],
deriving numeric values of similarity. While there is a clear
notion of similarity between these actors, basing a metric
upon such data will also find that all other actors who
starred in several martial arts movies and were raised in
Hong Kong are similar to Bruce Lee and Jackie Chan.
Besides, coming into the possession of such detailed actor
descriptions can be very difficult.
It is possible, however, to exploit human cognition in order
to instantiate semantic similarities. Traditionally, domain
experts have been employed to determine how similar items
are to one another. Unfortunately, the volume of data
processed in a recommendation system is often too large to
wait for hand produced results. Alternatively, data in
collaborative networks can be exploited. Flixster
(www.flixster.com), for example, allows users to write
about their favourite movies. In analysing the comments,
actor similarities can be derived. Unfortunately, this
approach creates very few actor relationships and is, of
course, domain specific, requiring an active community of
users.
A similarity metric has been selected that is not domain
specific, does not require data descriptions and can process
large volumes of data. This metric is typically referred to
as the Normalised Google Distance metric [4]. It is
accessible through the Measures of Semantic Relatedness
server API [18] and tends to perform well under diverse
conditions. The Normalised Google Distance between two
items, i1 and i2, is:
1 2
1 2 1 2
1 2
max{log ( ),log ( )} log ( , )
( , )
log min{log ( ),log ( )}
f i f i f i id i i
M f i f i
−
=
−

where M is the total number of Google pages searched, f(i1)
and f(i2) are the number of hits for i1 and i2 respectively, and
f(i1,i2) is the number of hits for the co-occurrence of i1 and
i2.
The workings of the Normalised Google Distance metric
are intuitively appealing. In essence, the more often two
items appear together on the same web pages, the more
similar they are considered to be. To illustrate, consider the
working of the metric in finding the similarity between the
three actors Jackie Chan, Bruce Lee and Jane Fonda. Jackie
Chan and Bruce Lee are clearly more similar to one another
than to Jane Fonda. This is confirmed using the
Normalised Google Distance metric (Table 1). While all
three actor names appear in roughly the same number of
web pages, estimated by Google, the co-occurrence of actor
names in the same pages is much higher when comparing
Jackie Chan with Bruce Lee as opposed to Jane Fonda.
That is, web pages that contain reference to Jackie Chan
also contain reference to Bruce Lee more often than to Jane
Fonda.
Actors Jackie Chan Bruce Lee Jane Fonda
Jackie
Chan
2,420,000
(0.0)
965,000
(0.09)
145,000
(0.26)
Bruce Lee 965,000
(0.09)
2,630,000
(0.0)
46,700
(0.37)
Jane
Fonda
145,000
(0.26)
46,700
(0.37)
1,930,000
(0.0)
This similarity metric is applied to all items in the system,
producing a form of domain specific ‘general knowledge’.
For example, Jackie Chan is more like Bruce Lee than Jane
Fonda. These data are all generated in a one-off session
prior to being used in the EPE interface. It should be
stressed that although all of the results presented here are
based on actor similarities, this does not prevent the metric
from being used in other domains.
Data Visualisation Technique
The similarities between items in the system are visualised
in a graph. The more similar two items are, the closer their
proximity tends to be in the graph. The data are visualised
using the Prefuse toolkit for java (www.prefuse.org)
through the radial tree layout method [e.g. 8]. A photo of
each actor, accompanied by their name, is displayed as a
node in the graph. The radial tree can be focussed upon any
node in the graph by implementing the smooth transition
animation from Yee et al. [21]. User studies have found
this form of visualisation particularly attractive for
navigating throughout data.
This form of data visualisation has also been used
extensively in understanding social networks, due to the
connections that are drawn between nodes [6]. In this case,
these connections are drawn to symbolise a close similarity
between the nodes. In order to present all nodes in a single
radial tree, first, all nodes are connected to their two most
similar nodes, in terms of similarity. Only the two nearest
neighbours are connected as the readability of the graph is
improved when fewer connections are drawn. To connect
nodes that remain unconnected from the main tree, the
system then connects the closest unconnected pair of nodes
repeatedly until all nodes are connected. Note that a single

node can have more than two connections. For example,
even though only the connections between Jackie Chan and
his two most similar actors are drawn, he may be the most
similar actor to a fourth actor. When the fourth actor’s
connection is drawn with Jackie Chan, Jackie Chan will
have three connections, two being with his two most similar
actors and one being with the fourth actor. Actors that are
considered to be similar to many other actors thus appear in
the graph with many connections.
The EPE Interface
An EPE interface has been designed that allows user to
navigate throughout the graph of actors and to indicate their
preferences (Figure 1). As there are 500 actors in the
screen, not all of their images can be displayed to the user
at the same time. To deal with this problem of ‘screen real-
estate’, there are two viewing modes for the graph; zoomed
in and zoomed out. When zoomed out, the user can view
the graph in entirety but cannot see the individual actors.
By zooming in, the can see individual actor photos
accompanied by their names. In total, users can perform
five functions:
1. zoom in by right clicking on a graph area (in zoomed
out mode) and zoom out by right clicking on a graph
area that does not contain an actor (in zoomed in mode);
2. pan within the graph by left clicking on a graph area and
dragging in the direction to pan;
3. search for an actor by typing the actor’s name in the
search box. When the user starts to type a name, the
mode changes to zoomed out mode and all actor nodes
who’s names match the string are enlarged;
4. change the preference towards an actor (like, dislike,
neutral, no preference) by righting clicking on the
actor’s node (in zoomed in mode);
5. re-organise the graph to centre upon one actor by double
left clicking on another actor node.
No preferences are registered for any actors when the
interface is first displayed? As such, all nodes appear with
a white backing. Node backing colours are used to indicate
that a preference has been registered for a particular actor
and a message also appears in text in the bottom left of the
screen.
USER EVALUATION
Introduction
An empirical study was conducted that made use of the
EPE interface. While considering if the interface was easy
and enjoyable to use, the similarity metric was also
evaluated by constructing two versions of the graph. The
first version was constructed using the similarity metric as
described in the previous section. The second version,
however, was constructed by randomizing the positions of
the actors in the first version. Users were asked to enter
their preferences using the two graph versions, allowing the
differences between the two to be discovered. The results
of the study allow a broad number of conclusions to be
reached in terms of EPE interfaces. A within-subjects
design was followed.
Figure 1. The EPE interface instantiated with 500 actors. a) Zoomed out mode. Actors appear as small nodes in the radial
tree. The title reads “Who are your favourite actors?” On the bottom right, a search bar allows users to search for actors.
Four actors match the search and are enlarged in the graph; b) Search bar. Shows the string “u” being searched and indicates
that there are 4 actors that contain the string “u” in the graph; c) Zoomed in mode. Each node is made up of an actor’s photo
with their name below. The node is coloured depending upon the user’s preference. In this case, the central node, Uma
Thurman, appears in yellow signifying a like preference. Grey lines are drawn between nodes to indicate similarity between
actors. The search bar, although not shown, also appears in zoomed in mode.
a)
b)
c)

Materials
The study was conducted within the field of
cinematography, where participants were asked to declare
their monadic preferences with respect to actors. They
could indicate that they like, dislike or are neutral towards
each actor. Cinematography has received much attention
throughout recent years in recommendation systems both
for its commercial appeal and for the availability of data
(e.g. www.amazon.com and [11]). Orange Labs, similarly,
has a commercial interest in providing Home Services such
as Video On Demand for its customers.
Three graphs were generated for use in the study: an
organised graph (where actors were positioned according
to their similarity); an unorganised graph (which was a
randomised form of the organised graph) and a
demonstration graph (which was a different randomised
form of the organised graph). Each graph was displayed in
different tests using the EPE interface. Each graph
contained the 500 most frequently starring actors from an
in-house database of films. This quantity of actors allowed
a manageable 125,000 similarities to be calculated, while
still providing a substantial amount of choice for
participants. The similarity between each pair of actors was
calculated using the Normalised Google Distance.
Participants
28 participants were recruited. All were experienced
computer users who had received a university-level
education. There were 14 male and 14 female participants.
Procedure
Participants were given a sheet of instructions and asked to
complete a small number of tasks with the demonstration
graph in order to familiarise themselves with the EPE
interface. This included practicing the five functions
available to them. The study leader was available to answer
any questions that arose. All questions that were asked
were answerable by referring to the instructions. The
familiarisation process took around 10 minutes on average.
Once familiarised with the EPE interface, participants were
given two tasks to complete. One task was completed with
the organised graph and the other with the unorganised
graph. These tasks were referred to as task 1 and task 2 so
as not to influence the participant. Here, they will be
referred to as the organised graph task and the unorganised
graph task for clarity.
In both tasks, the participant was asked to enter as many
preferences as they could within the period of five minutes.
All of the actions that participants performed with the EPE
interface were recorded in log files. Participants were
divided equally in two experimental conditions to avoid
carry-over effects. In one condition, the organised graph
task was followed by unorganised graph task and in the
other condition the unorganised graph task was followed by
the organised graph task. While completing the tasks, the
study leader left the room to allow the participant to
concentrate. On completing the two tasks, participants
were asked to complete a questionnaire. Both groups were
given the same questionnaire to complete.
Hypothesis
It is hypothesised that participants will find it easier to
declare their preferences for actors in the organised graph
task than in the unorganised graph task, within the same
time period.
Measurements
The ease of declaring preferences is measured by analysing
the differences between the preferences that are entered in
the two tasks. Both the quantity of preferences and the
distribution of tastes (e.g. likes, dislikes and neutrals)
entered are considered. The ease of declaring preferences is
also measured by asking participants for their subjective
opinions of how easily they found declaring their
preferences for actors in the two tasks.
RESULTS
All 28 participants successfully performed the two tasks
and completed their questionnaires. The quantity of
preferences and the distribution of tastes that were elicited
are presented alongside the subjective ease with which
participants believed that they entered preferences. In
addition, participants reported in their questionnaires a
number of differences between the graphs in the two tasks
and how they found the EPE interface.
Preference Elicitation
The quantity of preferences and distribution of tastes that
were entered in the two tasks differed. This difference was
significant with respect to the number of actors that were
liked in the two tasks (mean = 36.11, SD = 22.29 for
organised, vs. mean = 26.89, SD = 16.05 for unorganised,
student t-test = 0.00003) (Figure 2). This translates to a
34% increase in the number of actors that participants
indicated that they liked in the organised graph task
compared to the unorganised graph task.
While no significant difference was found in terms of the
overall number of preferences that participants entered in
the two tasks (mean = 53.04, SD = 34.69 for organised, vs.
mean = 46.43, SD = 26.74 for unorganised, student t-test =
0.01950), more preferences tended to be entered in the
organised graph task. No significant differences were
found in the number of actors that were disliked (mean =
6.18, SD = 3.88 for organised, vs. mean = 7.11, SD = 5.67
for unorganised, student t-test = 0.16418) nor those that
elicited neutral preferences (mean = 10.75, SD = 16.61 for
organised, vs. mean = 7.11, SD = 5.67 for unorganised,
student t-test = 0.13711) between the two tasks. There was
a slight tendency, however, for more dislikes and neutral
preferences to be entered in the case of the unorganised
graph task.

Figure 2. Mean difference between the numbers of
participants’ preference entries in the two tasks.
Table 2. Mean responses to perceived ease of preference
entry statements. Cronbach’s alpha reflects how well the
responses are correlated. Note that the means for the
second question are reversed (e.g. 3.64 is equivalent to 2.36
and 3.11 is equivalent to 2.89).
Preference Elicitat ion
0
10
20
30
40
50
60
All Like Dislike Neut ral
N
o
.
P
re
fe
re
n
c
e
s
E
n
te
re
d
Organised
Unorganised
Perceived Ease of Entering Preferences
To enter a preference, participants had to right click upon
actor nodes. In both tasks, participants could search for an
actor using the search facility or zoom in on the graph and
pan around. Neither one of these techniques were promoted
over the other, leaving it to the participant to decide how
they wanted to complete the tasks.
In the questionnaire, participants were asked to comment
upon how much effort it took them to declare their
preferences in the two different tasks. Two related
statements were marked on a 5-point Likert scale, where 1
is equivalent to strongly disagree and 5 is equivalent to
strongly agree. The first statement took the positive form “I
found it easy to enter my preferences” while the second
statement took the negative form “Entering my preference
demanded too much effort”. Both positive and negative
forms were used to improve the quality of the results.

Mean Ease of Preference
Entry Statements Organised Unorganised
I found it easy to enter my
preferences. 3.96 3.36
Entering my preferences
demanded too much effort
(mean reversed) 3.64 3.11
Cronbach’s alpha = 0.80

In asking these questions, participants reported that it was
easier to enter their preferences and that it demanded less
effort to do so in the organised graph task than in the
unorganised graph task (Table 2). The results indicate that
participants found it easy to enter their preferences in the
organised graph task whereas there was neither agreement
nor disagreement in the case of the unorganised graph task.
It should be kept in mind that these tasks were simply
named task 1 and task 2 for the participants and that no
notion of organisation in the graphs had been suggested.
Perceived Differences between Graphs
In the questionnaire, participants were also asked to share
their experiences by being asked a number of open
questions. In particular, if they indicated that they found
any differences between the graphs in the two tasks, then
they were asked what kind of differences they perceived.
The majority of participants, 22 out of 28 (79%), indicated
that they found differences between the graphs in the two
tasks. All of these participants though that the organisation
of the actors different in the two differed tasks. Several
comments were volunteered with respect to these
differences:
• the organised graph had been hand designed so that it
was easier to navigate;
• the organised graph rearranged itself based on the
actors that the participant said that they liked;
• the organised graph arranged the participant’s favourite
actors together;
• there were more connections between actors in the
organised graph than in the unorganised graph;
• the organised graph arranged actors together who:
• co-starred in the same films;
• shared the same nationality;
• shared the same degree of celebrity;
• were similar to one another.
It is revealing to note that participants made use of the word
similar in describing actors positioning the organised graph.
One participant went as far to suggest that a human-
designed ontology was responsible for the organisational
structure.
Appreciation of the EPE Interface
Using the same 5-point Likert scale, participants indicated
that they enjoyed using the EPE interface and that they
would be happy to use it again (mean = 4.11, SD = 0.99).
Participants did, however, have some concerns with the
interface. These included the facility to:
• change preferences. Some participants did not like
having to click twice to register a dislike and three
times to enter a neutral preference.
• zoom in. Some participants would have preferred a
precise indication of the region into which they could
zoom into before zooming.
• zoom out. Some participants felt lost when zoomed in
on the actors as they were not sure of where they were
with respect to the entire map

DISCUSSION
The results confirm that participants enter more preferences
for actors who they like (positive preferences) in the
organised graph task than the unorganised graph task in the
same time period (Figure 2). This increase in positive
preferences is estimated to be around 34%, a substantial
difference for systems that are powered by such knowledge.
The effect is localised to positive preferences even though
participants were asked to give as many preferences as they
could in terms of like, dislike and neutral. Such a bias also
appeared in the questionnaires evidenced by participants
making various comments about their favourite actors but
none about disliked actors. Perhaps this result reflects
people’s desire to search out actors who they like over those
who they dislike. Perhaps, however, the distribution of the
data was such that most of the actors were actors who tend
to please.
It may be expected that the substantial increase in explicitly
given positive preferences would lead to a perceived
increase in cognitive effort on behalf of the participant.
The results, however, demonstrate the contrary; participants
perceive that they can find familiar actors more easily and
that they expend less cognitive effort in the process (Table
2). From the outset of both tasks, it is likely that
participants spend the same amount of effort finding an
actor. Once that actor is found, however, it is easier to find
similar actors in the organised graph than it is in the
unorganised graph, as they tend to be within close
proximity. This would likely account for the perceived
decrease in cognitive effort.
Participants clearly enjoyed using the EPE interface from
their questionnaire comments. They noted that they felt
more like they were playing a game at times and not at all
bored. This EPE interface clearly succeeds in offering an
alternative to standard list-based EPE interfaces.
Participants did note, however, a few concerns with the
interface. For example, they found that right clicking on a
node to change the preference was sometimes problematic.
This could be addressed by displaying three icons on each
node, one for each level of appreciation and asking users to
click the appropriate one. Participants also suggested some
improvements to the zooming facility. When the user is
zoomed out, and wants to zoom in on an area, it would be
useful to indicate the region that the user would zoom in on
if they right clicked (similar the broken rectangle in Figure
1). At present, the interface zooms in on the cursor point,
but this is not evident for the user. Similarly, an animated
zoom would help users to orientate themselves when
changing the zoom mode. When zoomed in, it would also
be useful to know at which region of the map the user was
zoomed in to. This could be indicated by showing a
miniature version of the complete map in zoomed out
mode, perhaps in the top right hand corner of the screen,
with the current region indicated.
It should be noted that while taking 5 minutes to enter
preferences was acceptable for this study, it would still
likely be too much to ask of regular users. It is much more
acceptable if the EPE interface is always available to the
user when they want use it. Such an interface should not be
consigned to a one-off preference entry situation, such as
when a user begins to use a system but should be accessible
at any time, becoming more of an interactive tool.
In entering preferences, participants perceived a number of
differences between the graphs in the two tasks, even
though they were only exposed to them for 5 minutes each.
The notion of actor similarity was evoked in describing the
organised graph task while no such comments were made
about the unorganised graph. Several participants offered
reasons as to how they perceived the differences between
the two graphs, noting that similar actors often appeared
next to one another in the organised graph. Participants
were more sensitive to the graph’s organisation when they
could identify relationships between the actors, such as
actors who share the same nationality or who often co-star
in the same films. These comments validate the use of
Normalised Google Distance metric as a measure of
similarity in this domain.
The fact that participants commented on the notion of
similarity is encouraging. When a user sees that two actors
are connected together, they note that the system makes a
connection between them. This can go some way to
understanding the system’s logic [1], an important step in
building trust in the system [14]. It is intended that the
hybrid recommendation system under construction makes
use of such similarities in making recommendations.
Typically, a user will enter a number of preferences, using
an EPE interface like the one described here, and will then
request recommendations. Rather than surrendering all of
the control over the system, however, the user can take
control by making precise or imprecise requests. For
example, if the user asks for a movie with an actor like
Jackie Chan then the system can propose actors who are
similar to Jackie Chan, based on the similarity metric. It is
expected that the reply to this request will be better
understood if the user has already been introduced to the
system’s notion of similarity in the preference elicitation
phrase. The user’s understanding of system logic should
therefore be improved.
The measure of semantic relatedness server [18] has only
recently become available, allowing unlimited public access
to Google-based similarity metrics (as opposed to the
restricted access APIs that exist). The results reported here
make a contribution as to how users perceive the workings
of the similarity metric within the context of actors,
confirming that users perceive the notion of similarity. In
one case, a user went as far as to suggest that the organised
graph was based on a human-designed ontology.
Chen and Dumais [3] also tested the impact of data
reorganisation on users. They compared the traditional list
output of a search engine with an automatically categorised
output and discovered that users found information 50%

faster using the categorised version. They claim that the
categorised results allowed users to focus on items in the
categories rather than having to browse through the results
sequentially. Such factors may also be at work in the
visualisation of data in the EPE interface. Pu and Chen [14]
also found that an organisation-based presentation of
recommendations increased trust in recommendations over
a list-based view. Users also reported that they spent less
cognitive effort when using the organised view, as found
with the EPE interface.
CONCLUSION
A new EPE interface is described that can takes data and
organises it based on a robust similarity metric. Data
similarities are then visualised into a pleasing tree-based
graph. Users can navigate through the graph and explicitly
enter their preferences for different items. Users enter 34%
more positive preferences when the graph is organised with
the similarity metric compared to when it is left
unorganised. The similarity metric thus goes some way to
replacing what is traditionally in the domain of human-
design decisions. Not only do users find more items that
they like in the organised graph but they report a reduction
in cognitive effort. The new interface is a robust and
flexible alternative to existing EPE interfaces.
REFERENCES
1. Burke, R. Knowledge-based Recommender Systems.
Encyclopedia of Library and Information Systems, 69
(32).
2. Candillier, L., Meyer, F. and Boullé, M., Comparing
State-of-the-Art Collaborative Filtering Systems. in
Machine Learning and Data Mining in Pattern
Recognition, 5th International Conference, MLDM
2007, (Leipzig, Germany, 2007), Springer.
3. Chen, H. and Dumais, S., Bringing order to the Web:
automatically categorizing search results. in CHI '00:
Proceedings of the SIGCHI conference on Human
factors in computing systems, (2000), ACM, 145-152.
4. Cilibrasi, R. and Vitanyi, P.M.B. The Google similarity
distance. Ieee Transactions on Knowledge and Data
Engineering, 19 (3). 370-383.
5. Fishburn, P.C. Utility Theory for Decision Making.
Wiley, New York, NY, USA, 1970.
6. Freeman, L. Visualizing Social Networks. Journal of
Social Structure, 1 (1).
7. Hansson, S.O. Preference Logic. in Gabbay, D. and
Guenthner, F. eds. Handbook of Philosophical Logic,
2001, 319-393.
8. Herman, I., Melançon, G. and Marshall, M.S. Graph
Visualization and Navigation in Information
Visualization: A Survey. IEEE Transactions on
Visualization and Computer Graphics, 6 (1). 43.
9. Krulwich, B. LIFESTYLE FINDER: Intelligent User
Profiling Using Large-Scale Demographic Data AI
Magazine, 1997, 37-45.
10.Middleton, S.E., Shadbolt, N.R. and De Roure, D.C.,
Capturing Interest Through Inference and Visualization:
Ontological User Profiling in Recommendation
Systems. in K-CAP'03, (Sanibel, Florida, USA, 2003).
11.Miller, B., Albert, I., Lam, S., Konstan, J. and Riedl, J.,
MovieLens unplugged: experiences with an occasionally
connected recommender system. in IUI '03: Proceedings
of the 8th international conference on Intelligent user
interfaces, (2003), ACM, 263-266.
12.Pazzani, M. and Billsus, D. Learning and Revising User
Profiles: The Identification of Interesting Web Sites.
Machine Learning, 27 (3). 313-331.
13.Pitkow, J., Schutze, H., Cass, T., Cooley, R., Turnbull,
D., Edmonds, A., Adar, E. and Breuel, T. Personalized
search. Commun. ACM, 45 (9). 55.
14.Pu, P. and Chen, L., Trust building with explanation
interfaces. in IUI '06: Proceedings of the 11th
international conference on Intelligent user interfaces,
(2006), ACM, 100.
15.Rashmi, S., Design Strategies for Recommender
Systems. in UIE Web App Summit, (2006).
16.Shi, X. and Yu, Z., An optimal recommendation system
for content personalization management [DTV
applications]. in International Conference on Consumer
Electronics, 2005. ICCE05, (2005), 121-122.
17.Swearingen, K. and Rashmi, S. Interaction design for
recommender systems. in Designing Interactive
Systems, 2002.
18.Veksler, V.D., Grintsvayg, A., Lindsey, R. and Gray,
W.D., A proxy for all your semantic needs. in
CogSci2007, 29th Annual Meeting of the Cognitive
Science Society, (Nashville, TN, 2007).
19.von Ahn, L., Liu, R. and Blum, M., Peekaboom: A
Game for Locating Objects in Images. in ACM CHI06,
(2006).
20.Wellman, M.P. and Doyle, J., Preferential Semantics for
Goals. in Ninth National Conference on Artificial
Intelligence (AAAI-91), (Anaheim, California, USA,
1991), AAAI Press/MIT Press, 698-703.
21.Yee, K.-P., Fisher, D., Dhamija, R. and Hearst, M.,
Animated exploration of dynamic graphs with radial
layout. in Information Visualization, 2001. INFOVIS
2001. IEEE Symposium on, (2001), 50.