Lorcan Coyle · Juan Ye · Emerson Loureiro · Stephen Knox · Simon
Dobson · Paddy Nixon
A Proposed Approach to Evaluate the Accuracy of
Tag-based Location Systems
Received: June 11 2007 / Accepted: July 3 2007
Abstract Location detection systems that use tags are
a popular means of determining a user’s location. These
systems are characterised as requiring the user to carry
an identity tag that is detected by sensors, which typi-
cally use some form of triangulation to determine loca-
tion. Although estimates for precision for these systems
are published by the respective manufacturers the cus-
tomer experience can vary widely. This paper proposes
an evaluation framework which will allow different sys-
tems to be compared more directly. This framework is
specifically targeted at evaluating the experiences of tag-
ging humans, which can cause particular difficulties due
to the fact that many tag-based systems use communi-
cation frequencies that cannot pass easily through the
human body.
Keywords Evaluation Frameworks · Location-based
Systems
1 Introduction
Location based systems are becoming more commonly
applied to a number of problems in ubiquitous comput-
ing and beyond [11,5]. When dealing with these systems
it is important to have a gauge of their accuracy. High-
tower and Borriello performed an evaluation of location
systems and defined the quality of location systems by
looking at the percentage of readings that fall within
a certain distance of the true value [6]. The distances
denote the accuracy, or grain size, of the position in-
formation GPS can provide and the percentages denote
precision, or how often we can expect to get that ac-
curacy. Dobson et al. derive three factors for accuracy
This work is partially supported by Science Foundation Ire-
land under grant numbers 05/RFP/CMS0062 “Towards a se-
mantics of pervasive computing” and 04/RPI/1544 “Secure
and predictable pervasive computing”
L. Coyle
Systems Research Group, School of Computer Science and In-
formatics, UCD Dublin, Ireland;E-mail: lorcan.coyle@ucd.ie
of location information: precision, which captures inac-
curacy due to the resolution of sensor readings; decay,
which captures inaccuracy due to the staleness of sensor
readings; and confidence, which measures the degree of
belief on a sensor reading [2]. We use a combination of
these definitions: using Hightower and Borriello’s defi-
nition of accuracy and precision to calculate values for
Dobson et al’s precision component of location accuracy.
When we use the term precision in this paper, we are
using Dobson et al’s meaning of the word.
This paper describes an evaluation framework for cal-
culating the precision of a tag-based location system.
Other papers have evaluated location based technology
individually, but these evaluations are usually performed
by the stakeholders, and the evaluation criteria are not
always transparent. Additionally, it is often unclear whe-
ther the published precisions are determined from eval-
uations using tags alone or from tagged human subjects.
Our work focuses on human subjects due to the inherent
interference caused by the human body on the communi-
cations mechanisms used by the tested location systems.
While the examples described in this paper focus on hu-
man tagging, the framework itself is agnostic to what is
being tagged. The important point to take is that eval-
uations using this framework should take the intrinsic
physical properties of what is being tagged into account
when determining system precision.
Section 2 briefly surveys a number of tag-based loca-
tion systems that could be evaluated using this frame-
work. Section 3 describes the evaluation framework and
Section 4 describes an exemplar evaluation that is under-
way to test the precision of an installation of a tag-based
Location based system. Finally, Section 6 concludes the
paper, lays out the hypotheses underpinning our evalu-
ation, predicts some results and outlines some ongoing
research related to this paper.

2 Survey of Tag-based Location Detection
Systems
There are a number of popular tag-based location detec-
tion systems today. The most widely used is the GPS
system, which allows a user to be positioned anywhere
on the surface of the planet with a high degree of ac-
curacy (approximately 3-10m precision). This is beyond
the scope of this paper since its precision has been cal-
culate effectively and its limitations are well understood.
The most prominent of these limitations is the neces-
sity to have clear line-of-sight to at least four satellites.
The framework described in this paper is better suited
to local location positioning systems offering a sub-room
granularity.
Ubisense allows precise local positioning by tracking
a tag (the Ubitag), which is attached to a person. The tag
maintains radio contact with an installation of sensors.
These sensors use ultra-wideband (UWB) technology to
detect and react to the position of Ubitags. Ubisense
uses both Time Difference Of Arrival and Angle of Ar-
rival to calculate location. In a typical open environment,
Steggles and Gschwind claim that location accuracy of
about 15cm can be achieved across 95% of readings [10].
The Active Badge location system uses diffuse in-
frared technology to communicate between the tag and
sensor network [12] and offers room-level precision; the
Cricket system uses ultrasonic signals to calculate po-
sitional data, at a precision of approximately 1m (4sq
feet) [9]. When a cricket tag receives signals from mul-
tiple beacons in the infrastructure it triangulates its po-
sition. When receiving signals from only a single beacon
it can still provide location based on proximity to that
beacon. Many of these systems use radio signal strength
information (RSSI) to calculate location: these include
SpotON [7]; LANDMARC, which uses active RFID [8];
RADAR, which uses signals from existing WiFi infras-
tructure [1]; and Feldmann et al.’s work using Bluetooth
[3].
Each of these systems has some common factors that
affect any evaluation of their precision. We address these
factors in Section 4; our evaluation framework will at-
tempt to deal with these factors and allow evaluations
of multiple systems to be compared directly against each
other.
3 Evaluation Framework
Any evaluation of a location-based system must take into
account a number of factors that affect readings from
location sensors. Some of these relate to the technology
used and others relate to tag-based systems in general:
– Tag State: the tag state may have an impact on ac-
curacy. To conserve power, many tags will enter an
idle state after a period of inactivity. When perform-
ing evaluations this feature should be disabled. Also,
powered tags should be fitted with fresh batteries to
ensure that they operate at peak efficiency.
– Interference: depending on the tag communication
system, if more than one tag is put in the same gen-
eral area, they could interfere with each other. This
is typically a feature when dealing with passive RFID
tags.
– Sensor Configuration: The space with the highest
precision will typically be that which is well covered
by sensors. The physical configuration of these sen-
sors is critical, e.g., when dealing with Ubisense, the
best configuration is to place a sensor in each corner
of a room. In this way, the space with the highest
precision is in the middle of the room, i.e., is visi-
ble to all sensors. Correspondingly, the worst reading
area is the wall or the corner of the room that is out
of coverage of the sensors. A fair evaluation should
compare systems at a range of areas from where they
should be most accurate to where they will perform
worst.
– Environment: the environment in which the tag is
located will have an important impact on the de-
tected precision. The presence of metallic structures
or electronic equipment could lead to interference
with the location detection hardware when using UWB
systems, and the presence of ultrasound noise may in-
terfere with the cricket system. Any evaluation should
attempt to minimise the effect of these obstacles.
– Height When dealing with location detection sys-
tems care should be taken if 2D coordinates are re-
quired when dealing with systems that offer 3D co-
ordinates. Depending on the application, often the
height dimension is ignored and only x and y coordi-
nates are used. However, this may cause problems if
there is a difference in precision at different heights.
If this is the case, care must be taken when position-
ing the tag on the human body. If evaluating a 3D
location based system it would be possible to test
the extent of this problem by positioning the tag at
a different heights and comparing the generated pre-
cisions.
– Frequency: The frequency of sensor readings is an
important consideration when gathering evaluation
data. Generally this can be altered but as the fre-
quency is increased, the cost in terms of power usage
increases. An evaluation should use a frequency that
will be usable in real world conditions.
The factor that is usually ignored in typical evalua-
tions is the human body itself. If the location position-
ing system is explicitly for use in tracking people then
its evaluation should use human test subjects. This is
especially important when evaluating systems that use
communications that have difficulty penetrating water
(since people are composed of mostly water). This is an
extension of the environment factor except that it is not
useful to remove the human from the evaluation if track-
ing people is the target application.