EvAAL, evaluating AAL systems through competitive benchmarking, the experience of the 1st competition

Abstract

As Ambient Assisted Living (AAL) emerges as a need for our ageing societies, many barriers are still in place against its wide adoption. One of the main issues related to the creation of an AAL market is the lack of consensus around well established technologies which should effectively cover real needs of the population. EvAAL (Evaluating AAL Systems Through Competitive Benchmarking) is a newborn initiative aimed at evaluating solutions related to Ambient Assisted Living by organizing annual international competitions. Its main objectives are the creation of a community of stakeholders around AAL and the creation of metrics and benchmarks for both innovative prototypes and commercial solutions. EvAAL focuses not only on comparison of algorithms or specific hardware issues, but also of user acceptance, deployment and installation effort, integrability, etc. In its first versions, the competition is focusing on specific technical aspects of AAL but aims, in the near future, at joining heterogeneous "ambient" technologies in a common evaluation framework. In July 2011, the first EvAAL competition took place in Valencia, Spain, on Indoor Localization and Tracking for AAL. This paper describes how EvAAL is designed, its principles and how it is internally organized, and goes though an evaluation of this structure though the experience gained during the first competition. © 2012 Springer-Verlag.