ORI GIN AL PA PER
Metrics-based evaluation of learning object reusability
Javier Sanz-Rodriguez • Juan Manuel Dodero •
Salvador Sanchez-Alonso
Published online: 15 August 2010
 Springer Science+Business Media, LLC 2010
Abstract This paper aims to help in the selection of reusable educational materials from
repositories on the web, developing an indicator of the reusability of learning objects. For
this purpose, our research will be carried out in three stages. The first, based on previous
studies in this area, will determine those aspects that influence reusability. The second will
define a set of metrics that measure those aspects using metadata. The third will propose
different methods of aggregation in order to obtain a single resulting value and evaluate the
efficiency of the model by analyzing a significant set of learning objects obtained from the
eLera and Merlot repositories. The results obtained suggest that the proposed indicator
could provide useful information when searching for learning objects in repositories. This
reusability measurement could constitute an indicator of quality, which would allow search
results to be ordered, with those with the greatest possibility of being reused taking pri-
ority. Furthermore, the proposed reusability indicator could be calculated automatically or
in an assisted way if metadata elements satisfy the minimum quality requisites identified.
Keywords Reusability  Learning objects  Metadata  Metrics
1 Introduction
As with the development of open source software in projects such as Linux or Apache, in
education, there is a trend towards the development of quality open educational resources,
with suitable user rights that enable users to reuse them and modify them to fit their
J. Sanz-Rodriguez (&)
University Carlos III of Madrid, Av. Universidad 30, 28911 Leganes, Madrid, Spain
e-mail: javier.sanz.rodriguez@uc3m.es
J. M. Dodero
University of Cadiz, C/Chile, s/n, 1003 Cadiz, Spain
e-mail: juanma.dodero@uca.es
S. Sanchez-Alonso
University of Alcala de Henares, Ctra. Barcelona km. 33600, 28871 Alcala de Henares, Madrid, Spain
e-mail: salvador.sanchez@uah.es
123
Software Qual J (2011) 19:121–140
DOI 10.1007/s11219-010-9108-5

context. While they represent tremendous opportunities, open education programs also face
novel challenges and new anxieties. Perhaps the most obvious is the quality assurance of
the open materials. The task of manually reviewing materials is laborious and the quantity
of educational resources is enormous and growing by the day, so we need novel modes of
reviewing, assessing and sharing evaluations (Kelty et al. 2008).
These open educational resources include learning objects, which differ in that their
definition contains no explicit mention of their open nature and in that they are associated
with further technological characteristics such as whether they are digital, modular, self-
contained or reusable (Friesen 2009). However, this concept is controversial and currently
different definitions coexist—(IEEE 2002; Polsani 2003; Wiley 2002)—which may be
synthesized as follows: any educational material that is independent, self-contained, dig-
ital, identified by metadata and that may be reused in different educational contexts.
The concept of reusability constitutes the main reason behind the technologies associated
with learning objects. This is due to the fact that developing quality educational materials is
costly in terms of time and resources, which is why being able to reuse already-existing quality
materials will generate pedagogical and economic profits (Campbell 2003; Koper 2003).
While reusing learning objects is an empirical and observable fact, Sicilia (2004) affirms
that reusability is an intrinsic attribute of the object, which provides a priori measure of
quality, which may be proven by posteriori reuse data. This concept of reusability may be
defined as the degree to which a learning object can work efficiently for different users in
different digital environments and in different educational contexts over time. It should
always be borne in mind that there are different technical, educational and social factors that
will affect reuse (Palmer and Richardson 2004). In most situations, it will be necessary to
carry out certain types of modification—modularization, adaptation and aggregation—in
order to be able to reuse the learning object (Zimmermann et al. 2007).
Currently, most initiatives taken to improve reuse have attempted to define standards by
which learning objects may be used in different platforms without interoperability prob-
lems (Duval 2004). However, we find ourselves in a situation in which the potential
benefits of reuse remain out of reach; there are insufficient studies of reusability indicators
and the design criteria to guarantee it is lacking (Sicilia 2004).
Ochoa and Duval (2008, 2009) carried out a quantitative analysis of the reuse of
learning objects in real world settings. The scope of the study includes objects of differing
granularity and different types of repository. It concluded that only 20% of objects stored
in repositories are actually reutilized. In addition, the problems that the reuse of learning
objects must overcome are similar to those of other shared resources in repositories, such
as images, software libraries or APIs. For this reason, the reuse of learning objects is not
intrinsically any easier or more difficult than that of other types of component. Ochoa and
Duval affirm that the theories used for the reuse of other types of component may be used
in the reuse of learning objects. They also indicate that, although the reuse of educational
materials is currently on going, even without a technological framework that favors it, an
effort must be made to overcome these deficiencies in order to increase the degree of reuse.
In the current situation, any search for learning objects in a repository could return an
enormous list of results. With no quality indicator to shine a light on this information,
looking for learning objects can become a waste of time and effort (Kumar et al. 2005). As
happens with any search engine, it is desirable to have a filtering process so that the
information supplied satisfies the needs of the user in the best way possible.
This lack of information is the motivation behind this paper, the aim of which is to
propose an automatic or assisted method for estimating reusability which, by using the
122 Software Qual J (2011) 19:121–140
123