Leveraging uncertainty to rethink loss functions and evaluation measures for egocentric action anticipation

Abstract

Current action anticipation approaches often neglect the intrinsic uncertainty of future predictions when loss functions or evaluation measures are designed. The uncertainty of future observations is especially relevant in the context of egocentric visual data, which is naturally exposed to a great deal of variability. Considering the problem of egocentric action anticipation, we investigate how loss functions and evaluation measures can be designed to explicitly take into account the natural multi-modality of future events. In particular, we discuss suitable measures to evaluate egocentric action anticipation and study how loss functions can be defined to incorporate the uncertainty arising from the prediction of future events. Experiments performed on the EPIC-KITCHENS dataset show that the proposed loss function allows improving the results of both egocentric action anticipation and recognition methods.