Audio visual speaker verification based on hybrid fusion of cross modal features

Abstract

In this paper, we propose hybrid fusion of audio and explicit correlation features for speaker identity verification applications. Experiments were performed with the GMM based speaker models with a hybrid fusion technique involving late fusion of explicit cross-modal fusion features, with implicit eigen lip and audio MFCC features. An evaluation of the system performance with different gender specific datasets from controlled VidTIMIT data base and opportunistic UCBN database shows a significant performance improvement. © Springer-Verlag Berlin Heidelberg 2007.