SSIM based signature of facial micro-expressions

Abstract

Facial microexpressions (MEs) play a crucial role in the non verbal communication. Their automatic detection and recognition on a real video is a topic of great interest in different fields. However, the main difficulty in automatically capturing this kind of feature consists in its rapid temporal evolution, i.e. MEs occur in very few frames of video acquired by a conventional camera. In this paper a first study concerning the perceptual characteristics of ME is performed. The study is based on the observation that MEs are visible by a human observer, even though they are very rapid, and almost independently of the context. The Structural SIMilarity index (SSIM), which is a common perception-based metric, has been then used to detect a sort of fingerprint of MEs, that will be indicated as PES (Perceptual Expression Signature). The latter is able to efficiently guide the preprocessing step for MEs recognition procedure, as it allows for a fast video segmentation by providing only those frames where a ME occurs with high probability. Preliminary empirical studies on MEs in the wild have confirmed the feasibility of such an approach.