Deep-based openset classification technique and its application in novel food categories recognition

Abstract

Being able to accurately recognise food categories from input images has many possibly useful applications such as content-based recipe searching or automatic intake calories tracking. Convolutional neural networks has been successfully applied in a number of food recognition tasks. Despite its impressive predictive performance on closed datasets, there is currently no standard mechanism for distinguishing unknown object classes from the known ones leading to invalid classification attempts even on non-food images. In this paper, we study a technique for detecting whether input images are beyond the scope of CNN's knowledge. The idea is to model the final activation vectors of data from the known classes using a data description method namely the support vector data description. We can then reject network's prediction if the activation vector of the query image is too different from the known ones as generalised by the model. Experimental results on a subset of UECFOOD100 datasets demonstrated that the proposed method was able to accurately classify instances from the known classes while also being able to satisfactorily reject the prediction of novel food image compared to two commonly used baselines.