Design for a system of multimodal interconnected ADL recognition services

Abstract

As smart interconnected sensing devices are becoming increasingly ubiquitous, more applications are becoming possible by re-arranging and re-connecting sensing and sensor signal analysis in different pipelines. Naturally, this is best facilitated by extremely thin services that expose minimal functionality and are extremely flexible regarding the ways in which they can be re-arranged. On the other hand, this ability to re-use might be purely theoretical since there are established patterns in the ways processing pipelines are assembled. By adding privacy and technical requirements the re-usability of some functionalities is further restricted, making it even harder to justify the communication and security overheads of maintaining them as independent services. This creates a design space that each application must explore using its own requirements. In this article we focus on detecting activities of daily life (ADL) for medical applications and especially independent living applications, but our setting also offers itself to sharing devices with home automation and home security applications. By studying the methods and pipelines that dominate the audio and visual analysis literature, we observe that there are several multi-component sub-systems that can be encapsulated by a single service without substantial loss of re-usability. We then use this observation to propose a design for our ADL recognition application that satisfies our medical and privacy requirements, makes efficient use of processing and transmission resources, and is also consistent with home automation and home security extensions.