Drinking Gesture Recognition from Poorly Annotated Data: A Case Study

Abstract

In this chapter we present a case study on drinking gesture recognition from a dataset annotated by Experience Sampling (ES). The dataset contains 8825 “sensor events” and users reported 1808 “drink events” through experience sampling. We first show that the annotations obtained through ES do not reflect accurately true drinking events. We present then how we maximise the value of this dataset through two approaches aiming at improving the quality of the annotations post-hoc. Based on the work presented in Ciliberto et al. (2018), we extend the application of template-matching (Warping Longest Common Subsequence) to multiple sensor channels in order to spot a subset of events which are highly likely to be drinking gestures. We then propose an unsupervised approach which can perform drinking gesture recognition by combining K-Means clustering with WLCSS. Experimental results verify the effectiveness of the proposed method, with an improvement of the F1 score by 16% compared to standard K-Means using Euclidean distance.