An independent approach to training classifiers on physiological data: An example using smiles

Abstract

Training neural network and other classifiers on physiological signals has challenges beyond more traditional datasets, as the training data includes data points which are not independent. Most obviously, more than one sample can come from a particular human subject. Standard cross-validation as implemented in many AI tools gives artificially high results as the common human subject is not considered. This is handled by some papers in the literature, by using leave-one-subject-out cross-validation. We argue that this is not sufficient, and introduce our independent approach, which is leave-one-subject-and-one-stimulus-out cross-validation. We demonstrate our approach using KNN, SVM and NN classifiers and their ensemble, using an extended example of physiological recordings from subjects observing genuine versus posed smiles, which are the two kinds of the nicest smiles and hard for people to differentiate reliably. We use three physiological signals, 20 video stimuli and 24 observers/participants, achieving 96.1% correct results, in a truly robust fashion.