MCoMat: A new performance metric for imbalanced multi-layer activity recognition dataset

Abstract

Existing performance metrics assess classifiers on single granularity layer. Having multi-layer labels is also possible such as activity recognition datasets. Semantic annotations could be given with multiple granularity layers in these datasets e.g., activity and the current step within that activity like: cooking and taking ingredients from fridge. Recognizing both layers is important i.e., remote monitoring of patients with dementia. To evaluate a classifier for both layers concurrently, a new performance metric is required. However, it is not easy to design as there are many underlying issues: the relation between the layers and the impact of class imbalance. This work proposes a new metric for evaluating multi-layer labeled dataset considering the mentioned factors and is applied on two datasets. It is found that it can assess the performance of a model classifying activities at two different granularity layers and give more insightful results i.e. reflecting performance for each layer.