Visualising Social Bookmarks
J. Klerkx and E. Duval
fjoris.klerkx, erik.duvalg@cs.kuleuven.be
Katholieke Universiteit Leuven, Belgium
Abstract
Social bookmarking tools are very popular nowadays. In most
tools, users tag the bookmarks to describe them. Therefore, it is of-
ten hard for users to discover implicit structures between tags, users
and bookmarks. We think that this is essential for both end users to
discover new bookmarks that could be of interest to them, and for
researchers who want to study how people use social information re-
trieval tools. In this work, a cluster map visualisation technique is cus-
tomized to enable users to explore social bookmarks in the del.icio.us
and the CALIBRATE system. The design of our visualisation aims to
automatically identify tag and community structures, and visualises
these structures in order to increase the users awareness of them.
1 Introduction
The ability to store or bookmark web pages by describing them by tags or
terms, has been one of the most important features of browsers since the
beginning of the Web [Hammond et al., 2005]. Social bookmarking tools
became possible when the process of keeping bookmarks migrated from the
browser to the Web.
Tagging is the process of assigning meaning to online items, such as web
pages, images, videos, etc. by labelling them with personalised keywords
that are shared among users [Guy and Tonkin, 2006]. The purpose of web-
based social bookmarking tools is to tag the content of other users, mainly
for the benet of the tagger, although the bookmarks and tags are generally
1

2public, and users can establish networking opportunities [Hammond et al.,
2005].
Social networks are an important factor for nding and spreading infor-
mation because a big part of learning is social [Wenger, 1996]. For example,
talking with your neighbour about how to build a garden house, may pro-
vide new insights about this process. Wenger introduced the idea of the
\community of practice" which means that people can satisfy their need for
information more eciently if they are embedded in a community with sim-
ilar interests and problems [Wenger, 1998]. \Weak ties" are relationships
between people that don't know each other very well. These weak ties are
important to allow information to spread from one closely-knit community
to another [Granovetter, 1983].
Implicit relationships between users, tags and content are therefore very
useful in Web2.0-style social information retrieval systems. However, those
relationships are not always clear in the traditional ways of accessing social
bookmarks. Our research therefore focusses on using information visuali-
sation techniques
 that enable insight to analysts in those implicit relationships, and
 that may oer end users new ways to nd content and information
that could be of interest to them but that would not have been found
through explicit searches.
We will start this paper in section 2 with an overview of existing so-
cial bookmark tools and the traditional ways of accessing the information
within those systems. Section 3 looks at the data of two existing social
bookmarking systems in more detail. A number of requirements for open-
ing the view on social bookmark spaces, are discussed in section 4. Section
5 elaborates on a prototype application with a cluster map visualisation for
exploring social bookmarks. This prototype has been used to explain some
typical usage scenarios in section 6. Implementation notes are presented in
section 7. Section 8 discusses the prototype and presents evaluations of the
methodologies. We conclude this work in section 9.
2 Social Bookmark Tools
There are quite a few social bookmarking initiatives, like CiteULike [Ci-
teULike, 2008]), Connotea [Connotea, 2008]), del.icio.us [delicious, 2008]),

3Furl [Furl, 2008]), BlogMarks [BlogMarks, 2008]), etc. A thorough general
review can be found in (Hammond et al., 2005). A number of ways are
usually available for browsing personal bookmarks:
1. First of all, one can browse through scrollable pages of bookmarks.
2. Secondly, one can perform a \tag query" of the collection by clicking
on a tag [Millen et al., 2007].
3. Exploratory search activities are also often supported in social book-
mark applications. Some examples are browsing bookmarks by time
or by popularity like on del.icio.us.
Tagging is an unstructured bottom-up approach of classifying content,
in contrast to a top-down structured approach based on taxonomies, the-
sauri or ontologies [Weinberger, 2007]. The semantic structures that result
from a tagging approach are often referred to as a folksonomy. Tags gen-
erally produce a
at namespace, rather than the hierarchical structures
that taxonomies or other formal classication systems provide [Hammond
et al., 2005]. However, there can also be rich implicit structures between
tags, bookmarks and users that are not immediately clear. For instance,
there is a clear relationship between two separate tags that have been used
to classify the same bookmarks. Two users that use exactly the same tags
for dierent bookmarks is another example. This means that they are in-
terested in the same topics, and therefore they might be interested in the
bookmarks of the other.
On top of that, some researchers claim that social bookmarking ap-
plications become harder to navigate as the amount of tags increases an
they thus become less meaningful [Chi and Mytkowicz, 2007]. The reason
for their claim is that e.g. del.icio.us is moving closer and closer to the
proverbial \needle in a haystack" where any single tag references too many
documents to be considered useful.
An important goal of our research is thus to provide understanding in
bookmarks, tags, users and the relationships between them. In related
work, \tag clouds" are often used to visualise the tag structures of one or
more users. Tag Clouds are visual presentations of a set of words, typically
a set of tags, in which attributes of the text such as text, weight or color
can be used to represent features (e.g. frequency) of the associated terms
[Halvey and Keane, 2007]. When a user clicks on a tag, the user obtains an

4ordered list of tag-described resources, as well a list of related tags [Hassan-
Montero and Herrero-Solana, 2006]. Usually, the tags are displayed in
alphabetical order.
Many other visualisation tools of tag, bookmark and network structures
were created for del.icio.us and others. Notable such tools include HubLog
[HubLog, 2008]) that enables graphical browsing of del.icio.us tags in a
mind mapping way. Extisp.icio.us Text [Extisp.icio.us, 2008]) provides a
random textual scattering of user tags, sized according to the number of
times that they have been used. Revealicious [Revealicious, 2008]) is a set of
2D graphic visualisation techniques that enables the user to browse, search
and select tags and bookmarks. Vizster [Heer and Boyd, 2005]) is a tool
that is designed for visualising the online social network Friendster, as a
browseable network of social relations. Vizster is very useful for sociological
research but does not take tag structures into account.
All these initiatives are based on visualising the tag structures or the
community structures but they do not take both structures into account
for the visualisation at the same time, to make implicit community and tag
structures more apparent, e.g. for analysts that are studying folksonomies.
The remainder of this paper investigates how to use information visualisa-
tion techniques in such ways to make those structures more apparent and
to make bookmarks accessible.
Two social bookmark tools were chosen as the basis of our visualisation
design: del.icio.us and the CALIBRATE portal. These tools are described
in the next section, together with the rationale for choosing them.
3 Information Characteristics
3.1 del.icio.us
Del.icio.us [delicious, 2008] is probably the most well-known social book-
marking tool, designed to store and share bookmarks on the web instead of
saving them in the browser. We chose del.icio.us as a source of data as it is
highly popular with many users, lots of data and with a very easy API to
access this data. Those characteristics makes del.icio.us a very good refer-
ence point: if the visualisation techniques prove to be eective and ecient
for exploring del.icio.us, they might be valuable for other social bookmark
tools as well.
A number of steps are involved in the process of collecting del.icio.us
data:

3.1 del.icio.us 5
1. We start from one or more del.icio.us user names and collect their
bookmarks, and the tags of those users that describe those book-
marks.
2. Del.icio.us enables users to create a network of other users, so that
they can access the bookmarks of these users. In the second step,
we collect the tags and bookmarks for all the users that are in the
network of the users identied in step 1.
3. Del.icio.us also enables users to be a fan of other users. User A \is
a fan of" user B if user A has added user B to his network. This
relation is not necessarily bi-directional. In this third step, the same
data is again collected for all the fans of the chosen user(s) in step 1.
This process of collecting data takes place at runtime but can also be
performed beforehand. However, the del.icio.us API does have some limits:
 Only those bookmarks, tags and users in the network can be used
that are explicitly made public by the user that added them.
 The del.icio.us API does not allow to submit a web page and query
and retrieve information on the users that tagged that particular web
page. If this would be allowed, we could for instance take a number of
random webpages, ask which users tagged them, and afterwards visu-
alise the implicit social structures that possibly exist between those
users. What can be retrieved for specic web pages is the number of
users that tagged the web page and added it to their collection. We
could use this number to visualise the popularity of a web page if we
consider a higher number as more popular.
 The usage throttling and abuse monitoring software at the del.icio.us
website limits the amount of queries that can be performed in order to
prevent the harvesting of all the data of del.icio.us [del.icio.us, 2008].
On top of that, a maximum of a hundred bookmarks per user can be
retrieved in one query.
Summarised, the available information characteristics on bookmarks
are:
 the URL, e.g \www.google.com",
 tags that describe the web page, e.g \search, search engine, web",

3.2 CALIBRATE portal 6
 title of the bookmark, e.g. \Google search engine", and
 number of users that tagged the bookmarks, e.g \saved by 22974
people".
The characteristics of a particular user are:
 the del.icio.us username, e.g \jkofmsk"
 the users in his network, and
 fans of the given user.
3.2 CALIBRATE portal
Applying social tagging and bookmarking to an educational setting, CAL-
IBRATE users are teachers who create a social bookmark and add tags
on learning material, for their own use and to share them with their stu-
dents or other teachers. This work is done in the context of the CALI-
BRATE project [CALIBRATE, 2008], which makes K-12 digital learning
resources available to 78 schools in Hungary, Austria, Estonia, Czech Re-
public, Lithuania and Poland in dierent curriculum areas. Schools can
access material in dierent languages through a portal that is connected to
a federation of learning resource repositories (GLOBE) [Globe, 2008] in the
pilot countries. Within this portal, a personal bookmarking and tagging
tool has been available to all users since the beginning of 2007. The data of
these users, their tags and their bookmarked learning resources are taken
into account for our visualisation where it is used to visualise the implicit
structures between users, tags and learning resources.
Learning material is described by Learning Object Metadata (IEEE
LOM [IEEE, 2002]). A goal of LOM is to enable sharing of descriptions of
learning resources between resource discovery systems, which should lead
to a reduction in the cost of providing services based on high quality re-
source descriptions [Duval and Hodgins, 2003]. The availability of LOM
works to our advantage because it contains more metadata than the avail-
able metadata in traditional social bookmark systems like del.icio.us. This
metadata can be taken into account while visualising them to further enable
insight in the collection. Compared to the features that were available in
del.icio.us, we use the following extra elements that describe the resources
and are important in the exploration process:

7 authors; e.g. while searching for material on the semantic web, a
learner might prefer material created by Tim Berners-Lee, rather than
an unknown author.
 language; e.g material in Spanish is not interesting for non-Spanish
speaking learners.
 unique identier; e.g. important for actually retrieving the material
if it is interesting.
 creation date; e.g. material might be out-of-date and therefore less
interesting than newer created material.
On top of this, we have access to information about the users in CALI-
BRATE, like the languages they speak and their country of residence. This
information will for instance be used to lter out material that cannot be
understood by the user.
The information from CALIBRATE and del.icio.us is summarised in
Table 1. The data itself for this analysis came from a period of ten months,
January 1 to October 31 2007 and includes real data. There were 142 teach-
ers, hereafter referred as users, who made 1022 posts of favourite learning
resources. These bookmarks covered 682 dierent learning resources or
items. There were 1029 multilingual tags recorded in the system, some of
which were reused by users.
4 Prototype Requirements
Our research therefore focusses on using information visualisation tech-
niques
 that provides insight to analysts in the implicit relationships between
bookmarks, tags and users;
 that may oer end users new ways to nd content and information
that could be of interest to them but that would not have been found
through explicit searches.
This section discusses a number of requirements for applications that are
designed with the goal of opening up the view on social bookmark spaces.
A prototype application that uses information visualisation techniques, has
been designed with these requirements in mind.

8Table 1: Overview data features in del.icio.us and CALIBRATE portal

91. Hypertext makes a dynamic organisation of information, i.e. nodes
with content, possible through links or connections. Navigation be-
tween the information is possible by following those links. Many rich
hypertext systems are criticised since its users may suer from the
\lost-in-hyperspace" syndrome which means that they cannot iden-
tify where they are, or that they cannot remember what they have
covered already [Theng and Thimbleby, 1998]. Much work has been
done to address this problem, like providing breadcrumbs that show
the trail of links leading to the current page [Blustein et al., 2005].
A number of visualisations of hypermedia systems have been devel-
oped to address this problem by using a novel paradigm instead of
the hypertext paradigm [Mukherjea, 1999]. Most of them have been
based on the node and link graph diagram metaphor like the \Navi-
gational view builder " [Mukherjea and Foley, 1995] where web pages
are represented as nodes.
Social bookmarking tools like del.icio.us consider both users and book-
marks as pieces of content. Tags are presented as links between those
users and bookmarks and can therefore be used to navigate through
the social bookmark space. Applications with the goal of opening
up the view on social bookmark spaces should be careful not to cre-
ate the same lost-in-hypertext syndrome that can be found in tradi-
tional hypertext systems. In our design, we use a novel paradigm,
called a cluster map visualisation, for browsing and exploring social
bookmarks. This paradigm and the reason for choosing it, will be
explained in section 5.2.
2. Secondly, we want the visualisation system to be able to automatically
identify implicit tag and community structures. Such a structure is
formed when two or more tags or users describe or share common
bookmarks. After the identication of these structures, they should
be visualised for the users, so that these users can become aware of
them. We think that this can lead to the discovery of interesting
material for those users, that could not have been found through
explicit searches.
3. We consider visualisation as a technology. This means that visuali-
sation has to be eective and ecient or that it should do what it is
supposed to do by using a minimal amount of resources [Van Wijk,
2006]. Information exploration should be a joyous experience, but

10
many commentators talk of information overload and anxiety (Wur-
man, 1989). A key challenge for information visualisation researchers,
often suggested by the community, is to make their systems usable
by common computer users [Mukherjea, 1999]. Visualisation appli-
cations do not always t in the normal work
ow of users. If we
therefore want that our visualisation design is useful, ecient and
eective, users should be able to explore the social bookmarks in a
playful manner in a fun and engaging space.
4. Human-computer information retrieval (HCIR) has emerged in re-
search as the study of information retrieval techniques that brings
human intelligence into the search process [Marchionini, 2006]. To
achieve this, users should be able to interactively create a selection
of bookmarks and get details on them when needed [Shneiderman,
1996].
5 Social Bookmark Visualisation
Figure 1 shows that our visualisation consists of three parts:
1. a selection widget that presents lists of the users and tags in the
currently loaded data (section 5.1),
2. a cluster map visualisation (section 5.2), and
3. a lter pane with integrated results list (section 5.3).
These panes are synchronised with each other. We discuss these panes in
detail in the next paragraphs. Typical usage scenarios will be described in
section 6 where we focus on exploring and discovering new bookmarks, by
identifying and visualising those implicit structures that are e.g. formed
when two or more tags or users describe or share common bookmarks.
5.1 Selection Widget
Users of social bookmarking tools most frequently browse the bookmark
space by other people and by tags [Millen and Feinberg, 2006]. Providing
easy access to the people and tag space is therefore important as we want
to create a visualisation that is both eective and ecient while explor-
ing the social bookmark space. End users of e.g. del.icio.us know how to

5.1 Selection Widget 11
Figure 1: Cluster Map visualisation

5.1 Selection Widget 12
nd interesting items within lists. Therefore, the selection widget presents
overview lists of both users and tags, that are currently loaded in the sys-
tem. Alternatives to list presentations are e.g. tag clouds [Halvey and
Keane, 2007]. However, our main goal is not to nd interesting tags on
itself but to provide understanding of relationships between users, tags and
bookmarks. A simple list presentation is sucient as a starting point for
selecting interesting tags and users for this goal.
Figure 2(a) shows that each item is presented as a node in the list. A
node consist of the particular tag or user and the number of bookmarks that
are described with that tag or user. In CALIBRATE, more information is
available about the resources. Therefore, Figure 2(b) not only shows the
tags and users but also countries and languages. Each node also contains
a check-box that can be used to create a selection of items that should be
added to the visualisation. To optimise browsing of tags, users, countries
and languages, a lter widget has been built that is visible on top of the
selection widget and that can be used to nd items faster. Figure 2(a) shows
for instance only those items in the lists that start with \web" because the
word \web" has been typed in the widget.
In our design, we have chosen to follow the philosophy of \start with
what you know, then grow" [Heer et al., 2005]. This means that, by default,
nothing is visualised in the cluster map but the bookmarks of the current
user. The user can thus start from his own bookmarks, and afterwards use
checkboxes of tags, users, languages or countries, to include the correspond-
ing items in the visualisation. In this way, the initial visualisation carries
less perceptual and computational burden to start with, because the user
does not need to gure out complex structures that could be visual if more
information was added to the visualisation. An example of this can be seen
in Figure 8 which shows a rather complex cluster map of 7 users and one
tag \java". When a user would see this visualisation from the start, he
would need a some time to make sense of the patterns.
The same philosophy is also adopted in related research like Vizster
which is a visualization system for playful end-user exploration and nav-
igation of large-scale online social networks [Heer et al., 2005]. HubLog
[HubLog, 2008]) enables graphical browsing of del.icio.us tags in a mind
mapping way and starts from one tag that a user should ll in. The
del.icio.us network explorer [explorer, 2008] starts with one user and enables
the exploration of that users' network.

5.1 Selection Widget 13
(a) del.icio.us data (b) CALIBRATE data
Figure 2: A view of the selection widget. When a user clicks a checkbox,
the item is added to the cluster map visualisation. (a) shows the lists for
del.icio.us, (b) for the CALIBRATE portal.

5.2 Cluster Map Visualisation 14
5.2 Cluster Map Visualisation
Visualisation should show the data: it should reveal what the data is
about [Tufte, 2001]. Our visualisation of the social bookmark space should
therefore be expressive. Clear visualisation of the triple users, items and
tags should be supported to create an eective navigational and exploration
tool. Many visualisations have been developed for web resources (see [Fluit
et al., 2005] and [Dodge and Kitchin, 2001] for an extensive overview).
Especially in the context of social bookmark tools, a number of graph vi-
sualisations have been created (see section 2). Another metaphor that has
frequently been used for accessing social bookmark tools is the landscape
metaphore like the \Islands of Music" where tags of music songs are repre-
sented as islands on a map [Pampalk, 2006]. Similar tags are located closer
to each other.
A cluster map [Fluit et al., 2005] is a technique that is very expres-
sive in visualising classes and item that belong to those classes. One can
also consider Items like tags, users, language and countries as classes where
bookmarks belong to one or more of those classes. Those classes and rela-
tionships between them need to be easy to detect. A cluster map makes use
Venn diagrams which makes it easy to detect those relationships between
those classes [Venn, 1880]. Therefore, we decided to customise a cluster
map for visualising social bookmark spaces with the goal of validating the
use of this technique for exploring those spaces.
A cluster map visualises the objects of a number of selected classes.
Those classes and their relationships are easy to detect. It is immediately
apparent which items belong to one or multiple classes, which classes over-
lap and which do not. A bookmark is represented as a small circle in the
visualisation. Each bookmark belongs to the collection of one or more users.
In Figure 3, two users are shown, each with a hundred bookmarks in their
collection. A small icon of a human, followed by a user name identies a
user. Those two users have six bookmarks in common which is represented
by the label \6 / 100 ". This is represented in the visualisation by the
smaller common cluster of bookmarks in the middle. By using the selec-
tion widget that has been described in the previous paragraph, users can
select which users and tags are drawn on the cluster map.
Bookmarks can be clustered by the users that have them in their col-
lection, or by the tags that describe them. This can be seen in Figure 4,
where the tag \email" is shown. Users \lisamac" and \jgarber" have one
bookmark in common which is tagged by \email". The user \jgarber" has

5.2 Cluster Map Visualisation 15
Figure 3: Cluster Map showing 2 users with 100 bookmarks and 6 of them
in common. An icon followed by a username identies a user.
Figure 4: Cluster Map visualisation, showing 200 bookmarks with 2 users
and 1 tag \email". 20 bookmarks of user \jgarber" and 23 bookmarks of
users \lisamac" are about \webdesign". The selection of bookmarks on
webdesign has been created by using the lter pane (section 6(a)).

5.3 Filter pane with Integrated Result List 16
3 extra bookmarks tagged with \email" that are not in the collection of the
other user.
An important feature of social bookmarking tools is the popularity of
bookmarks. For instance, if a web page on the topic of \java" is added by
10.000 people, and another web page on the same topic is only added by a
hundred people, one might trust the rst web page more than the latter.
The rapid and accurate identication of popular bookmarks is therefore an
important requirement in the design of our visualisation. Color is an impor-
tant and frequently-used feature for information encoding [Healey, 1996a]
because well chosen colors allow for rapid and accurate identication of
individual data elements by users [Healey, 1996b] [Few, 2004]. In Figure 4,
two main colors are used to represent bookmarks yellow and grey. A color-
saturation sequence is used for the yellow color to identify the popularity
of the bookmarks among the users. This means that a bookmark that is
presented with a brighter yellow color, is more popular among users.
Such a set of selected bookmarks can be created by clicking on a user,
a tag, a country or a language in the selection widget, or by performing a
keyword query in a search box. A selection of bookmarks is color-encoded
in the visualisation:
 grey bookmarks are items that were ltered out by the users because
they were less interesting, and
 yellow bookmarks are items that are in their interest.
The yellow bookmarks in Figure 4, belong to a selection of bookmarks
that were tagged with the tag \webdesign". The yellow bookmark be-
tween \lisamac" and \email" therefore belongs to the collection of the users
\lisamac' and \jgarber" and is described by the tags \email" and \webde-
sign". This selection was created by clicking on this tag in the list of tags
in the selection widget, which can be seen in Figure 2(a) where this tag is
highlighted.
Looking at CALIBRATE data, we can visualise bookmarks belonging
to countries. Figure 5 shows that people from Poland and Hungary tagged
respectively 167 and 264 learning resources in total of which there are 29
in common.
5.3 Filter pane with Integrated Result List
In the previous paragraph, we already mentioned that users should be able
to interactively create a selection of bookmarks and get details on them

5.3 Filter pane with Integrated Result List 17
Figure 5: A view of shared learning resources: of all the 167 learning
resources that were bookmarked and tagged by users from Poland, 29 are
shared with the 264 learning resources that were tagged by users from
Hungary.
when needed [Shneiderman, 1996]. In this way, they can zoom in on po-
tentially relevant bookmarks and continuously keep an overview of how
the additional search criteria restrict the remaining number of bookmarks.
Therefore we integrated a lter pane with a number of controls that can be
used to lter out less interesting bookmarks or add more interesting ones.
Figure 6(a) shows controls that can currently be used:
1. The rst one is a search box where a typical keyword query like
\javascript" can be performed.
2. The second one is a data visualisation slider [Eick, 1994] that can be
used to indicate an interval of the number of people that tagged a
bookmark.
These combined controls allow users to quickly nd popular bookmarks
on a particular topic. This is a basic use-case that is available in most
social bookmarking tools like del.icio.us and Dogear [Millen et al., 2006].
Other controls could be added to the lter pane as well. In the case of
CALIBRATE, we could e.g. choose to add a drop box for the language of
the resources. Users would than be able to lter out languages they do not
master. The results of the combination of these lters are shown in two
places:
1. the clustermap visualisation, where selected resources that do
match the criteria in the lter pane are represented as yellow circles.
The resources in Figure 5 that are represented in yellow match the

18
criteria in the search pane (Figure 6(a)) where the user added the
keyword \javascript" to include only results on this topic, and also
changed the slider to include only the objects that were tagged by at
least 2 users and at the most 4 users.
The complexity of the visualisation can be high when many tags are
added with a lot of overlaps between those tags. An important aspect
of using the lter controls, is therefore, that they can be used to reduce
this complexity.
2. the result list(Figure 6(b)), that shows detailed metadata about
the bookmarks that are selected in the cluster map, or bookmarks
that match the search terms when a query was performed. In the
case of the data from del.icio.us, the metadata cover the title, the
location, user(s) that added the bookmark and tag(s) that describe
the bookmark. On top of those, the metadata covers the language
and the country of the users in the case of the CALIBRATE por-
tal. A user can interact with the detailed information by clicking
on e.g. a tag that describes a bookmark. If this tag is not already
drawn on the cluster map, the visualisation automatically updates
itself and the tag classication is shown, together with the possible
relationships between this tag and currently visualised bookmarks.
The results should preferably be ordered by relevance because users
always want to nd the most relevant items. Therefore, we would e.g.
need relevance feedback from users. However, capturing and incor-
porating this feedback is has been of less priority because our main
goal has been to visualise the relationships between users, tags and
bookmarks. The results are therefore ordered alphabetically in the
result list for the time being. Ordering by relevance has been moved
to future work.
6 Exploring Social Bookmarks
We mentioned in the introduction that the use of information visualisation
techniques oers end users new ways to nd content and information that
could be of interest to them. On top of that, it is important that these
techniques enable insight in the implicit relationships as they are not always
clear in the traditional social bookmark tools (section 4); insight that could
enable them to nd interesting material that would not have been found

6.1 Exploration Ways 19
(a) Filter Pane (b) Result view
Figure 6: (a) shows the lter pane with a search box, a slider for the number
of users that tagged a bookmark and radio buttons for indicating on which
data set to search. (b) shows the result view with detailed metadata about
bookmarks.
through explicit searches. For example, if a user nds a relationship with
another user, he can nd material that has been tagged by that user that is
interesting to him. As an alternative to exploring social bookmarks on the
del.icio.us and CALIBRATE portal, we oer a novel access paradigm that
tries to enable a user to explore social bookmarks in a playful manner in a
fun and engaging space by creating a cluster map visualisation. The cluster
map visualisation enables users to identify tag and community structures.
The following sections will explain how users can become aware of them by
interpreting the visualisation.
6.1 Exploration Ways
Users can explore the social bookmark space by using the cluster map
visualisation in a number of ways.
1. First of all, the basic exploratory functionality on the del.icio.us web-
site is supported because this is the basic functionality that typical
end users of social bookmark tools expect to be able to do: One
 can look at one's own bookmarks,
 see which tags are used to tag a web page, either by the user
himself or by others in his network, and
 see all other users in one's own network that have saved the
same bookmark. Note that this means that a user will not see

6.1 Exploration Ways 20
del.icio.us users that added the same bookmarks but are not
in the same network. The reason for this is the usage throttling
and abuse monitoring software at the del.icio.us website (section
3.1).
We chose to let a user start from an egocentric point-of-view, with
a visualisation of one's own bookmarks, much along the lines of the
philosophy of start with what you know, then grow [Heer et al., 2005].
Users will see those bookmarks from the start, which will give them a
more comfortable feeling because they are already familiar with those
bookmarks and tags. They can than select a number of bookmarks
in the visualisation or perform a keyword query in the lter pane, af-
ter which detailed metadata about the resulting bookmarks is shown
in the result list. Only those bookmarks that are also currently vi-
sualised, are added to the result list because both the visualisation
and the result list are continuously synchronised with each other or
tightly coupled. This is needed to have comprehensible and consistent
aordances to guide the end users [Ahlberg and Shneiderman, 1994].
The metadata of the results contains all the tags that describe the
bookmarks, i.e. not only the tags of the user itself. The metadata
also contains all the users that added the specic bookmarks. The
user can click on those tags and users, and by doing this add them to
the cluster map, where the layout of the bookmarks updates itself to
represent the new sub-clusters, like in the example of Figure 4 where
the tag \email" is added.
2. The rst way of exploring is not sucient for eciently exploring so-
cial bookmarks. Users should also be able to explore all tags that are
in the data to nd interesting bookmarks. A second way of exploring
is therefore by browsing the selection widget to nd those tags. This
can be compared with sifting through a list or cloud of tags on the
del.icio.us website. Users can order the tag list alphabetically or by
the number of bookmarks they describe. When ordering them in the
latter way, one can see which tags are used the most. By adding tags
to the visualisation, the corresponding bookmarks, possibly tagged by
dierent users, can be explored in the cluster map. Moreover, tags
often denote concepts, so relationships between those concepts may
also be discovered. An example of this, is shown in Figure 7 where
users can see the relationships between the concepts \programming",

6.1 Exploration Ways 21
\dev" which stands for development, and \java". However, it's the
responsibility of the users to interprete the visualised relationships.
\Coee" is for example related with the coee brand \java" but \cof-
fee" is not related with the concepts of \programming" and \dev".
3. Highly interactive applications support human-computer information
retrieval (HCIR) [Marchionini, 2006]. Users should for instance be
able to easily see all metadata, associated tags and users of a book-
mark in the visualisation. This can easily be done by clicking upon
the bookmark in the visualisation after which this information is also
shown in the result list.
4. When a user nds an interesting bookmark, he should be able to
keep it for further reference [Shneiderman, 1996]. Implementing and
maintaining a system with user information and the history of his
actions is not in the scope of our design because we can use the
del.icio.us system for this purpose. Users can easily post newly found
bookmarks to del.icio.us by clicking upon them in the visualisation.
For the same reason, Users that are found to have the same interests,
can also be easily added to the del.icio.us network of the current user
by clicking on the corresponding nodes in the visualisation. This
is only possible when the user that is currently exploring the social
bookmark space, has a user account on del.icio.us.
5. Only a fraction of data is loaded from the start as we follow the
philosophy of \start with what you know, then grow" (section 5.1). To
enable further exploration, one can click the nodes in the visualisation
that represent users after which the network and the fan data of those
users are loaded into the system so that they become available for
exploring. The reason that we do not add the data of all users that
saved a bookmark automatically from the start is twofold:
 the data is loaded incrementally which reduces the volume of
data that has to be retrieved [Chaudhuri and Dayal, 1997].
 the usage throttling and abuse monitoring software at the del.icio.us
website, that we discussed in section 3.1, prevents us of adding
this information from the start. In the case of CALIBRATE, we
do have this information and the data is therefore immediately
loaded.

6.1 Exploration Ways 22
Figure 7: Cluster Map visualisation, showing the relationships between a
number of concepts, denoted by tags.

6.2 Example 1 23
6.2 Example 1
The following relationships exist between the users in the underlying del.icio.us
data:
 \jkofmsk" has two users in his network: \cougare" and \mmeire".
 \cougare" has two users in his network: \phOng" and \vuorikari".
 \phOng" on his turn has two users in his network: \pieterjelle" and
\woumpousse".
While exploring the bookmarks and tags of these users, one might want to
nd out if there are implicit relationships between the non-related users of
them. One can easily nd tags that are frequently used by ordering the
list of tags by popularity. Figure 8 shows the visualisation of the users
above and the tag \java" that is mostly used in this data. Showing this
tag leads to the uncovering of an implicit community of users that are
not originally in the same network of users, and thus probably are not
acquainted. However, they all seem to be interested in the concept of the
\java" programming language. By adding those other users to one's own
network, one can create a sort of community-of-practice [Wenger, 1998]
with a similar interest: programming in java. Establishing this community
may prove useful in the future while searching for new material on this
topic. This kind of visualisation therefore oers the opportunity to nd
content and information that could be of interest.
This example also shows that semantic closeness results in geomet-
ric closeness. If two items share many objects, they are semantically
close [Fluit et al., 2005]. The user \vuorikari" did not tag a bookmark
with \java". Therefore this user is drawn further away from the item that
represents this tag \java".
6.3 Example 2
Examining the social bookmark space of the CALIBRATE portal with the
cluster map visualisation allows users to get an impression of the resources
that are \used" across country and language barriers. Some researchers
in ARIADNE [Vuorikari et al., 2007], are interested in learning resources
that \travel well". In the context of this work, we may dene a learning
resource as \travels well" if it satises one or more of the following criteria:

6.3 Example 2 24
Figure 8: A Cluster Map visualisation showing a network of users, formed
by common bookmarks that are all tagged by \java".

25
 an item that is bookmarked by a user is of a dierent language than
user's mother tongue,
 an item has tags in (a) dierent language(s) than that of the item
language,
 an item is from a dierent country than where the user is from.
Travel well resources may be interesting for e.g repository owners or re-
searchers, in seeing what kind of resources users from other countries book-
mark. An example of this is shown in Figure 9 that gives an overview
of the hungarian learning resources that travel well: from 226 hungarian
resources, that are represented by the green color:
1. 5 are shared by users from both Poland and Estonia,
2. 15 are shared by Estonian users that are not shared by Polish users,
3. 20 are shared by Polish users that are not shared by Estonian users,
and
4. 1 is shared by German users.
These hungarian resources that \travel well", mainly cover an advanced
music education area, which contains hardly any explanatory text, but
mostly images and animations, which could explain why they are popular
in the other countries as well.
7 Prototype Implementation
Our prototype for visually searching and analysing del.icio.us and CALI-
BRATE social bookmarks is created with our open and extensible informa-
tion visualisation framework that we created as part of our research on the
use of information visualisation techniques for
exible and ecient access
to learning repositories [Klerkx et al., 2005]. Mapping collections of objects
into an interactive visual form is the core functionality of our framework.
We want to make it easy to add:
 new visualisation techniques like tree-maps, hyperbolic trees, node-
graphs, sheye-views, etc., by plugging in existing visualisation com-
ponents into our framework; and

26
Figure 9: A view of hungarian learning resources that travel well in both
Estonia as in Poland.

27
 new data sources, possibly delivered in various formats and structured
according to various metadata schemes.
Our framework therefore consists of a software architecture that is designed
to be ecient, scalable and of high-performance, with a structured API. For
the cluster map visualisation, we plugged in the Aduna Cluster Map soft-
ware [Aduna, 2008].This is a library in JavaTMthat contains functionality
for creating visualisations of collections of hierarchically classied objects.
By integrating this library in our framework, the Aduna visualisation tech-
nique is available for our prototype, described in this paper, but also for
other case studies that have been, or will be developed with our framework.
Moreover, this integration demonstrates the open and extensible nature of
our framework.
With our framework, we can for instance easily visualise the bookmarks
with e.g. a tree-map where the bookmarks are classied per user in a
exible and ecient manner, or with any of the other techniques that our
framework provides. A tree-map is a visualisation of hierarchical structure
that makes 100% use of the available display space. It maps the complete
hierarchy onto a rectangular region in a space-lling manner [Shneiderman
and Johnson, 1991]. Such a tree-map may be benecial by providing an
extra overview when exploring bookmarks by visualising all of them at
once. Performing a keyword query in the lter pane on the topic \web"
gives users an impression of the distribution of resources on that topic over
all users. This can be seen in Figure 10. It makes sense to further explore
those alternatives in subsequent future work.
The Contextualized Attention Metadata(CAM) framework [Wolpers
et al., 2006], is integrated into our information visualisation framework.
CAM can be used to capture the attention users spend on content in an
application [Najjar, 2008]. We gather user data of the prototype to evaluate
the eectiveness and usefulness of our design, and adapt the design where
needed. For instance, if we would notice from those logs that users never
use the lter widget (section 5.1), this could mean that the functionality is:
 not clear to the users and should be explained by e.g adding a help-
function,
 not found by the users in the user interface, so a better place needs
to be found for it,
 not found useful by the users so we would need to nd a new way of
ltering items.

28
Figure 10: Tree Map visualisation: blue rectangles represent the users,
yellow rectangles represent the bookmarks that match the selection made
by the keyword web in the search box and grey rectangles are bookmarks
that do not match.

29
8 Discussion
We strongly believe in rapid prototyping development, where software sys-
tems are delivered incrementally and requirements analysis continues through-
out the process, interleaved with implementation and evolution [Luqi, 1989].
It enables us to uncover problems like usability issues in the early stage of
development. During development, we have involved a number of users to
sollicit feedback. The results are described in the next paragraph. A user
study with a rather complete prototype version of our visualisation has
been conducted afterwards to validate the used techniques. These results
are discussed in section 8.2.
8.1 Evaluation 1
A fairly small number of test users (4  1) is enough to nd most usability
problems [Nielsen, 1992b]. Four users, both del.icio.us users and repository
owners in the CALIBRATE project, were involved to locate problems dur-
ing the design of our visualisation. We found out a number of things while
using the \think aloud" method during interviews with those users. The
\think aloud" method is a method for user testing where the users verbalise
their thoughts while using the system that is being tested [Nielsen, 1992a].
In this way, those users reveal their view on the system and possibly, their
misconceptions. We learned the following issues during those sessions:
 Portal Integration. Our third design requirement (section 4) stated
that we want our visualisation to be useful, ecient and eective.
Among other things, this means that our visualisation needs to be
tightly coupled with the normal work
ow of users. A browser ap-
plication is normally used to access del.icio.us and CALIBRATE for
exploring the bookmark space, but our visualisation is not directly
integrated in those portals. If users nd interesting bookmarks with
our visualisation, it should be easy for them to save and tag those
bookmarks within e.g. del.icio.us. Therefore, we integrated the func-
tionality to directly post newly found bookmarks to del.icio.us but
this is not enough. People want to be able to export a number of
bookmarks at once in order to save time. Adding users to one's own
network on del.icio.us is a requested functionality that has recently
been integrated.
 Zooming. This functionality could help some users while exploring

8.1 Evaluation 1 30
the social bookmark space to keep a good overview of the visualisa-
tion. This functionality will be added to the general framework (see
section 7) so that it will be available for this application but also for
other case studies that have been created with it. It still needs to
be further investigated if this would actually help in keeping a good
overview. On top of that, we need to make sure that this does not
cause the \lost-in-hyperspace" syndrome in the visualisation that we
want to avoid (section 4).
 Implementation issues. There were some issues with the visualisa-
tion prototype. For instance, we found out that some users in the edu-
cational CALIBRATE portal bookmarked some resources and tagged
them by concatenating keywords like for instance \water, pollution,
water pollution", which has been used as one tag. Our prototype or
the CALIBRATE portal needs to be adapted to be able to cope with
this.
 Learning Curve. We noticed during the think-aloud sessions that
the users did not directly use all available functionality because they
didn't know how to use it, or did not know it was there. Some users
therefore suggested to include a help-function or create a small video-
demo to keep this learning curve as small as possible.
 Complexity. The more tags and users are added, the more complex
a cluster map visualisation can become with overlaps between social
bookmarks and dierent categories. This would certainly be a prob-
lem if the cluster map would be a static image. However, the user is
given continuously control over the items that are currently visualised
and therefore the user can decide to remove items from the visual-
isation at any point during the exploration of the social bookmark
space.
 Timeline. Visualisation of social bookmarks through time would
enable users to learn about the evolution of their bookmarks, tags
and network. This would re
ect how users' interests and focus de-
velop over time. One idea to add time to our visualisation of social
bookmark space is to represent time by moving graphics. One ex-
isting example of this is Gapminder where the authors use animated
scattergrams to show statistics on world health [Rosling and Rosling,
2006].

8.2 Evaluation 2 31
8.2 Evaluation 2
The second evaluation that has been performed has been a subjective eval-
uation by using a web application: subjects were asked to ll out an online
post-experimental questionnaire. First of all, they had the possibility to
watch a short demo that explained the purpose of our prototype. After
this viewing this movie, subjects were asked to explore their del.icio.us
bookmark space with our prototype. The questions in the survey were
stipulated in such way to ensure that the users needed to perform a num-
ber of tasks. The goals of this evaluation were threefold: (i) to assess the
eectiveness of our approach to enable exploration of social bookmarks by
visualising the relationships between users, tags and bookmarks; (ii) to
assess the subjective acceptance of a visualisation tool for the purpose of
exploring social bookmarks; (iii) to nd out possible usability issues of the
prototype.
8.2.1 Survey Statements
In total, there were 15 statements:
 11 statements to measure the eectiveness of visually exploring social
bookmarks, and the subjective acceptance of this approach. To mea-
sure the subjects agreement to those statements, we used the Likert
scaling method [Likert, 1932] which is a popular scaling method. All
items were rated on a 1-to-5 rating scale: strongly disagree - disagree
- undecided - agree - strongly agree.
 4 open questions where users could ll in their opinion, suggestions,
and usability issues.
8.2.2 Findings
Table 2 presents the responses of all subjects to the statements above. The
mean for the level of expressiveness was higher than 4 with a standard devi-
ation of 0.52, meaning that all subjects found the cluster map visualisation
expressive for eectively exploring social bookmarks.
Finding items by using the search box has a value of 4.44 with a standard
deviation of 0.72, which means that all users found this a valuable feature
for exploring their bookmarks. We believe that this is related to the fact
that people are used to using keyword queries for searching bookmarks in
their own contexts.

8.2 Evaluation 2 32
Sorting tags by frequency or name with a mean of 2.55 was perceived
as not easy. From the general remarks that we received as answers to our
open questions, we learned that users thought that it would be a valuable
and interesting feature while exploring but it should be made more clear in
the interface by e.g specic buttons for this functionality.
Except for the previous statement, all others that measured the criterion
of eective exploration have values higher than 3. We can therefore assess
that our design has been found eective. The statement that measured the
experienced eciency of exploring the social bookmark space, has a value
of 4.11 with a standard deviation of 0.92. This means that users found
our design fast and ecient. All statements that measured the subjective
acceptance of our visual design have values over 3.22.
We can therefore conclude this section that users considered the use
of information visualisation techniques an eective and ecient means to
explore social bookmark spaces.
8.2.3 Recommendations
From the open questions that we asked users we received a number of recom-
mendations for further enhancement of the design. These are summarised
below:
 The items move in the visualisation, when e.g adding new items to
the visualisation. The reason for this, is that the visualisation algo-
rithm tries to recalculate the layout of the cluster map whenever new
items are added. The advantage of this is that users always see a
visualisation with the most optimal layout for the current elements
in the visualisation. However, this makes it sometimes dicult for
users to get a grip on what is visualised because the continuity is lost
for them. One way to solve this, would be to only recalculate the
layout when the users chooses so. This is of course something that
has to be further investigated.
 Adding an initial ranking on the tags in the tag lists so that users
will not have to bother to do that.
 Adding \clear"-buttons to the search box and the lter widget of the
tag lists (section 5.3 and 5.1).
 The dierence between users and tags should become more clear in
the visualisation. For now, users are identied by an icon and a

33
Table 2: Eective Exploration and Subjective acceptance.
username but we could for instance use dierent geometric shapes for
users and tags.
9 Conclusions
In this work, we have investigated how we can eectively and eciently
provide visual access to a collection of social bookmarks? Therefore, we
have presented the use of information visualisation techniques
 that enables insight to analysts in the implicit relationships between
users, tags and bookmarks; and
 that oers end users new ways to nd content and information that

REFERENCES 34
could be of interest to them but that would not have been found
through explicit searches.
For this purpose, we have created a tightly-coupled visualisation prototype
that uses a cluster map visualisation for representing bookmarks, users
and tags (Section 5). This prototype uses real data from del.icio.us and
CALIBRATE. We chose del.icio.us as a source of data as it is highly popular
with many users, lots of data and with a very easy API to access this data.
Those characteristics makes del.icio.us a very good reference point (section
3.1) for other social bookmark tools. Two evaluations have been conducted
to measure the eectiveness and the eciency of our design (Section 8).
From those evaluations, we learned that the use of information visualisation
techniques prove useful in opening up the view on social bookmark spaces.
Other possible data sources that can be used in the future, are folksonomies
like Flickr [Flickr, 2008] and YouTube [YouTube, 2008], where respectively
photographs and videos are described by tags.
References
[Aduna, 2008] Aduna (2008). The aduna clustermap library.
http://www.aduna-software.org/projects/display/CLUSTERMAP/.
Last retrieved: April 2008.
[Ahlberg and Shneiderman, 1994] Ahlberg, C. and Shneiderman, B.
(1994). Visual information seeking: tight coupling of dynamic query
lters with stareld displays. In CHI '94: Proceedings of the SIGCHI
conference on Human factors in computing systems, pages 313{317, New
York, NY, USA. ACM.
[BlogMarks, 2008] BlogMarks (2008). Blogmarks.
http://www.blogmarks.net. last retrieved: April 2008.
[Blustein et al., 2005] Blustein, J., Ahmed, I., and Instone, K. (2005). An
evaluation of look-ahead breadcrumbs for the www. In HYPERTEXT
'05: Proceedings of the sixteenth ACM conference on Hypertext and hy-
permedia, pages 202{204, New York, NY, USA. ACM.
[CALIBRATE, 2008] CALIBRATE (2008). The calibrate project.
http://calibrate.eun.org. last retrieved: april 2008.

REFERENCES 35
[Chaudhuri and Dayal, 1997] Chaudhuri, S. and Dayal, U. (1997). An over-
view of data warehousing and olap technology. SIGMOD Rec., 26(1):65{
74.
[Chi and Mytkowicz, 2007] Chi, E. and Mytkowicz, T. (2007). Understand-
ing navigability of social tagging systems. In Proceedings of the SIGCHI
conference on Human Factors in Computing Systems (CHI'07), pages
(San Jose, USA, April, 2007), New York, NY. ACM Press.
[CiteULike, 2008] CiteULike (2008). Citeulike. http://www.citeulike.org.
last retrieved: April 2008.
[Connotea, 2008] Connotea (2008). Connotea. http://www.connotea.org/.
last retrieved: April 2008.
[del.icio.us, 2008] del.icio.us (2008). The del.icio.us abuse monitor.
http://del.icio.us/help/json/url. last retrieved: April 2008.
[delicious, 2008] delicious (2008). The del.icio.us social bookmarking tool.
http://del.icio.us. last retrieved: April 2008.
[Dodge and Kitchin, 2001] Dodge, M. and Kitchin, R. (2001). Atlas of Cy-
berspace. Number ISBN 978-0201745757. Addison-Wesley Professional.
[Duval and Hodgins, 2003] Duval, E. and Hodgins, W. (2003). A lom re-
search agenda. In Hencsey, G., White, B., Chen, Y., Kovacs, L., and
Lawrence, S., editors, Proceedings of the 12th International World Wide
Web Conference, pages 659{667, New York. ACM Press.
[Eick, 1994] Eick, S. G. (1994). Data visualization sliders. In UIST '94:
Proceedings of the 7th annual ACM symposium on User interface soft-
ware and technology, pages 119{120, New York, NY, USA. ACM.
[explorer, 2008] explorer, N. (2008). The del.icio.us network explorer.
http://www.twoantennas.com/projects/delicious-network-explorer/.
last retrieved: April 2008.
[Extisp.icio.us, 2008] Extisp.icio.us (2008). Extisp.icio.us text.
http://kevan.org/extispicious. last retrieved: April 2008.
[Few, 2004] Few, S. (2004). Show Me the Numbers: Designing Tables and
Graphs to Enlighten. Number ISBN 0970601999. Analytics Press.

REFERENCES 36
[Flickr, 2008] Flickr (2008). Flickr. http://
ickr.com/. last retrieved: April
2008.
[Fluit et al., 2005] Fluit, C., Sabou, M., and van Harmelen, F. (2005).
Ontology-based information visualisation: Towards semantic web appli-
cations. In Geroimenko, V., editor, Visualising the Semantic Web (2nd
edition), pages 36{48. Springer Verlag.
[Furl, 2008] Furl (2008). Furl. http://www.furl.net. last retrieved: April
2008.
[Globe, 2008] Globe (2008). The globe consortium. http://globe-
info.org/globe/go. last retrieved: April 2008.
[Granovetter, 1983] Granovetter, M. (1983). The strength of weak ties: A
network theory revisited. Sociological Theory, 1:201{233.
[Guy and Tonkin, 2006] Guy, M. and Tonkin, E. (2006). Folksonomies:
Tydying up tags? D-Lib Magazine, 12(1).
[Halvey and Keane, 2007] Halvey, M. J. and Keane, M. T. (2007). An
assessment of tag presentation techniques. In WWW '07: Proceedings of
the 16th international conference on World Wide Web, pages 1313{1314,
New York, NY, USA. ACM. ISBN: 978-1-59593-654-7.
[Hammond et al., 2005] Hammond, T., Hannay, T., Lund, B., and Scott,
J. (2005). Social bookmarking tools (i). D-Lib Magazine, 11(4). ISSN
1082-9873.
[Hassan-Montero and Herrero-Solana, 2006] Hassan-Montero, Y. and
Herrero-Solana, V. (2006). Improving tag-clouds as visual information
retrieval interfaces. In Proceedings of International Conference on
Multidisciplinary Information Sciences and Technologies (InScit2006).
ISBN: 84-611-3104-5.
[Healey, 1996a] Healey, C. G. (1996a). Choosing eective colours for data
visualization. In VIS '96: Proceedings of the 7th conference on Visu-
alization '96, pages 263{270, Los Alamitos, CA, USA. IEEE Computer
Society Press.
[Healey, 1996b] Healey, C. G. (1996b). Eective visualization of large mul-
tidimensional datasets. PhD thesis, The University of British Columbia
(Canada). Adviser-Kellog S. Booth and Adviser-James T. Enns.

REFERENCES 37
[Heer and Boyd, 2005] Heer, J. and Boyd, D. (2005). Vizster: Visualizing
online social networks. In INFOVIS '05: Proceedings of the Proceed-
ings of the 2005 IEEE Symposium on Information Visualization, page 5,
Washington, DC, USA. IEEE Computer Society. ISBN ISSN:1522-404x
, 0-7803-9464-x.
[Heer et al., 2005] Heer, J., Card, S. K., and Landay, J. A. (2005). prefuse:
a toolkit for interactive information visualization. In CHI '05: Proceed-
ings of the SIGCHI conference on Human factors in computing systems,
pages 421{430, New York, NY, USA. ACM.
[HubLog, 2008] HubLog (2008). Hublog.
http://hublog.hubmed.org/archives/001049.html. last retrieved:
April 2008.
[IEEE, 2002] IEEE (2002). IEEE Standard for Learning Object Metadata.
The Institute of Electrical and Electronics Engineers, Inc., 3 Park Av-
enue, New York, NY 10016-5997, USA.
[Klerkx et al., 2005] Klerkx, J., Meire, M., S.Ternier, Verbert, K., and Du-
val, E. (2005). Information visualisation: Towards an extensible frame-
work for accessing learning object repositories. In Kommers, P. and
Richards, G., editors, Proceedings of World Conference on Educational
Multimedia, Hypermedia & Telecommunications, pages 4281{4287, Mon-
treal, Canada.
[Likert, 1932] Likert, R. (1932). A technique for the measurement of atti-
tudes. Archives of Psychology, 22(140):1{55.
[Luqi, 1989] Luqi (1989). Software evolution through rapid prototyping.
Computer, 22(5):13{25. ISSN 0018-9162.
[Marchionini, 2006] Marchionini, G. (2006). Toward human-
computer information retrieval. http://www.asis.org/Bulletin/Jun-
06/marchionini.html.
[Millen et al., 2007] Millen, D., Yang, M., Whittaker, S., and Feinberg,
J. (2007). Social bookmarking and exploratory search. In Bannon, L.,
Wagner, I., Gutwin, C., Harper, R., and Schmidt, K., editors, Proceedings
of the Tenth European Conference on Computer Supported Cooperative
Work, ECSCW, pages 21{40, Limerick, Ireland. Springer.

REFERENCES 38
[Millen and Feinberg, 2006] Millen, D. R. and Feinberg, J. (2006). Using
social tagging to improve social navigation. In Workshop on the Social
Navigation and Community based Adaptation Technologies.
[Millen et al., 2006] Millen, D. R., Feinberg, J., and Kerr, B. (2006). Do-
gear: Social bookmarking in the enterprise. In CHI '06: Proceedings of
the SIGCHI conference on Human Factors in computing systems, pages
111{120, New York, NY, USA. ACM.
[Mukherjea, 1999] Mukherjea, S. (1999). Information visualization for hy-
permedia systems. ACM Comput. Surv., page 6.
[Mukherjea and Foley, 1995] Mukherjea, S. and Foley, J. D. (1995). Visu-
alizing the world-wide web with the navigational view builder. Comput.
Netw. ISDN Syst., 27(6):1075{1087.
[Najjar, 2008] Najjar, J. (2008). Tracking Contextualized Attention Meta-
data, pages 93{117. Faculteit Ingenieurswetenschappen, Katholieke Uni-
versiteit Leuven.
[Nielsen, 1992a] Nielsen, J. (1992a). Evaluating the thinking-aloud tech-
nique for use by computer scientists. Advances in human-computer in-
teraction, 3:69{82. ISBN: 0-89391-751-6.
[Nielsen, 1992b] Nielsen, J. (1992b). The usability engineering life cycle.
Computer, 25(3):12{22.
[Pampalk, 2006] Pampalk, E. (2006). Computational Models of Music Sim-
ilarity and their Application in Music Information Retrieval. PhD thesis,
Vienna University of Technology, Vienna, Austria.
[Revealicious, 2008] Revealicious (2008). Revealicious.
http://www.ivy.fr/revealicious/. last retrieved: April 2008.
[Rosling and Rosling, 2006] Rosling, H. and Rosling, O. (2006). New soft-
ware brings statistics beyond the eye, pages 522{530. Organisation for
Economic Co-operation and Development.
[Shneiderman, 1996] Shneiderman, B. (1996). The eyes have it: A task
by data type taxonomy for information visualizations. In VL '96: Pro-
ceedings of the 1996 IEEE Symposium on Visual Languages, page 336,
Washington, DC, USA. IEEE Computer Society.

REFERENCES 39
[Shneiderman and Johnson, 1991] Shneiderman, B. and Johnson, B.
(1991). Tree-maps: A space lling approach to the visualization of hier-
archical information structuress. In Proceedings of Proceedings of IEEE
Information Visualization '91, pages 175{282. UM Computer Science
Department; CS-TR-2657.
[Theng and Thimbleby, 1998] Theng, Y. L. and Thimbleby, H. (1998). Ad-
dressing design and usability issues in hypertext and on the world wide
web by re-examining the "lost in hyperspace" problem. JUCS, 4(11):839{
855.
[Tufte, 2001] Tufte, E. R. (2001). The Visual Display of Quantitative In-
formation. Graphics Press.
[Van Wijk, 2006] Van Wijk, J. (2006). Views on visualization. IEEE Trans-
actions on Visualization and Computer Graphics, 12(4):421{433.
[Venn, 1880] Venn, J. (1880). On the diagrammatic and mechanical rep-
resentation of propositions and reasonings. Philosophical Magazine and
Journal of Science S. 5., 9(59).
[Vuorikari et al., 2007] Vuorikari, R., Ochoa, X., and Duval, E. (2007).
Analysis of user behavior on multilingual tagging of learning resources. In
In Proceedings of the 1st Workshop on Social Information Retrieval for
Technology Enhanced Learning, SIRTEL'07. CEUR,, volume 307, pages
6{17.
[Weinberger, 2007] Weinberger, D. (2007). Everything Is Miscellaneous:
The Power of the New Digital Disorder. Times Books.
[Wenger, 1996] Wenger, E. (1996). Communities of practice: The social
fabric of a learning organization. HealthCare Forum Journal, 39(4):20{
26.
[Wenger, 1998] Wenger, E. (1998). Communities of Practice: learning,
meaning, and identity. PhD thesis, Cambridge University Press, New
York.
[Wolpers et al., 2006] Wolpers, M., Gunnar, M., Najjar, J., and Duval, E.
(2006). Attention metadata in knowledge and learning management.
Proc. 6th International Conference on Knowledge Management (I-Know
2006), Journal of Universal Computer Science (ISSN 0948-695x).

REFERENCES 40
[YouTube, 2008] YouTube (2008). Youtube. www.youtube.com. last re-
trieved: April 2008.