Proceedings of the ED-MEDIA 2004, pp. 4373-4379, 2004
User Behavior in Learning Objects Repositories: An Empirical Analysis


Jehad Najjar, Stefaan Ternier and Erik Duval
Computer Science Department, K.U.Leuven
Celestijnenlaan 200 A, B-3001 Leuven, Belgium
{Jehad.Najjar, Stefaan.Ternier, Erik.Duval}@cs.kuleuven.ac.be


Abstract: This paper investigates the ways in which users interact with Learning Objects
Repositories (LORs) such as ARIADNE, MERLOT and SMETE when searching for relevant
learning objects. More specifically, we focus on the ways users search the ARIADNE
Knowledge Pool System (KPS). Our analysis is based on the log files of queries. We
investigate user behavior and compare it with the behavior of the indexers who introduce
learning objects into the KPS. We discuss lessons learned from this analysis and suggest
guidelines for the development of successful application profiles and toolsets.



1. Introduction

Over the last few years, many efforts have been spent to define metadata models. The purpose of these
models is to support the indexation and search of educational materials [learning objects] in Learning Object
Repositories [LORs]. Learning objects are stored in LORs to be used for different kinds of educational purposes and
by different end users. Understanding the end user behavior helps us to improve the repositories and their associated
tools. By doing this, we will be able to serve the users in a more efficient ways. Similar studies have been done in
the context of digital libraries in (Jones, et al., 1998) and in the context of Internet browsing in (Cockburn &
McKenzie, 2000). In these studies the end user behavior has been studied by analyzing the transactions’ log files.
On the other hand, in the context of LORs, this important issue has not been yet extensively investigated (Duval &
Hodgins, 2003).
In ARIADNE [Alliance of Remote Instructional Authoring & Distribution Network for Europe] learning
objects and metadata have been collected for over 7 years in the Knowledge Pool System [KPS]. At the time of
writing, the KPS contains nearly 4,750 metadata instances for learning objects from different science types,
languages, contexts and granularities; introduced by more than 100 indexers. These metadata instances have been
collected in seven Local Knowledge Pool Systems [LKPs] distributed over Europe. Metadata have been produced in
nine languages of the multilingual ARIADNE community. Moreover, many searchers from around the world use
the ARIADNE query tool (Neven, et al., 2003) to search for their relevant materials.
In this paper, we present a statistical analysis of ARIADNE query log files. In the first phase we analyzed
readily available data on 4,723 queries launched by about 390 users in six ARIADNE LKPs [Genoa, Galati,
Grenoble-UJF, Lausanne-EPFL, Lausanne-UNIL and Leuven-CS] over different time periods. Moreover, we
compare the behavior of the ARIADNE searchers with the behavior of the ARIADNE indexers investigated in our
previous work (Najjar, et al., 2003). Differences and similarities of the behavior of both searchers and indexers are
obtained. Hereby, the degree to which indexers and searchers agree when index or search learning objects in the
KPS can be determined. Hence, this work will allow us to enhance the use of ARIADNE metadata and provide
guidelines for developing, evaluating application profiles and associated tools.
The paper is structured as follows: in section 2, data collection is introduced. In section 3, we discuss the
results derived from investigating searchers queries in the ARIADNE KPS. Discussion is illustrated in section 4.
Conclusions and future work are drawn in section 5.

Proceedings of the ED-MEDIA 2004, pp. 4373-4379, 2004
2. Data Collection

The ARIADNE Indexation- and Query tool shown in figure 1 has been used to introduce learning objects
of different granularities, science types and disciplines into the ARIADNE KPS.


Figure 1: Indexation- and Query tool for ARIADNE Figure 2: Media types of indexed learning objects

Currently, the KPS contains a large number of learning objects used for different educational purposes. For
example, PowerPoint slides, zip files, and html documents [see figure 2].
User activities have been logged in each of the ARIADNE LKPs. Data logged includes:
• User IP address
• Query Date
• Metadata elements used to form the query such as: title, science type, main concept, etc.
• Conditional operator such as: =, <=, >=, contains, starts with, ends with, etc.
• Values provided for those selected elements such as: metadata standards in practice, exact sciences, learning
technologies, etc.
• Number of learning objects that satisfy information provided.

The data were collected over different time periods, related to about 390 different users in six ARIADNE
LKPs:

• Galati [University of Galati, Romania]
• Genoa [University of Genoa, Italy]
• Grenoble-UJF [Université Joseph FOURIER à Grenoble, France]
• Lausanne-EPFL[École Polytechnique Fédérale de Lausanne, Switzerland]
• Lausanne-UNIL [Université de Lausanne, Switzerland]
• Leuven-CS [CS Dept., K.U.Leuven, Belgium]
The distribution of the logged queries is shown in table 1. The total number of queries included in this
study is 4,723.
Table 1: User queries classified by LKPs and time period
LKP Name Grenoble-UJF Leuven-CS Lausanne-EPFL Genoa Galati Lausanne-UNIL
Start Date Feb. 03 Apr. 03 Feb. 02 Jan. 03 Apr. 02 Feb. 03
End Date Oct. 03 Oct. 03 Dec. 03 Nov. 03 Aug. 03 Oct. 03
No. of Queries 1787 1146 857 423 409 101
N = 4723 Queries