Bimodal Biometrics Using EEG-Voice Fusion at Score Level Based on Hidden Markov Models

Abstract

This paper presents an experiment on bimodal biometrics based on fusion at score level of electroencephalographic (EEG) and voice signals. The experiments described have been carried out using an open database with commands in Spanish, available to the academic community for research purposes. The accuracy of user identification systems is evaluated using Hidden Markov Models (HMM) as classifiers for both EEG and voice modalities. Feature extraction is implemented using Mel-Frecuency Cepstrum Coeficients (MFCC) and Wavelet analysis for voice and EEG, respectively. From the scores generated by each independent system, data fusion is proposed at a score level for multiple cases of weighted sums based on weighted arithmetic means, and for the score product case using a geometric mean scheme. Performance evaluation indicates a recognition rate of 90% in average. The results obtained confirm the expected tendency, validating the convenience and usefulness of multimodal user identification systems while setting the ground for future studies involving data fusion at different levels and using other classifiers.