Filter bank common spatial pattern and envelope-based features in multimodal EEG-fTCD brain-computer interfaces

Abstract

Brain-computer interfaces (BCIs) provide alternative means of communication and control for individuals with severe motor or speech impairments. Multimodal BCIs have been introduced recently to enhance the performance of BCIs utilizing single modality. In this paper, we aim to advance the state of the art in multimodal BCIs combining Electroencephalography (EEG) and functional transcranial Doppler ultrasound (fTCD) by introducing advanced analysis approaches that enhance system performance. Our EEG-fTCD BCIs employ two distinct paradigms to infer user intent: motor imagery (MI) and flickering mental rotation (MR)/word generation (WG) paradigms. In the MI paradigm, we introduce the use of Filter Bank Common Spatial Pattern (FBCSP) for the first time in an EEG-fTCD BCI, while in the flickering MR/WG paradigm, we extend FBCSP application to non-motor imagery tasks. Additionally, we extract previously unexplored time-series features from the envelope of fTCD signals, leveraging richer information from cerebral blood flow dynamics. Furthermore, we employ a Bayesian fusion framework that allows EEG and fTCD to contribute unequally to decision-making. The multimodal EEG-fTCD system achieved high classification accuracies across tasks in both paradigms. In the MI paradigm, accuracies of 94.53%, 94.9%, and 96.29% were achieved for left arm MI vs. baseline, right arm MI vs. baseline, and right arm MI vs. left arm MI, respectively – outperforming EEG-only accuracy by 3.87%, 3.80%, and 5.81%, respectively. In the MR/WG paradigm, the system achieved 95.27%, 85.93%, and 96.97% for MR vs. baseline, WG vs. baseline, and MR vs. WG, respectively, showing accuracy improvements of 2.28%, 4.95%, and 1.56%, respectively compared to EEG-only results. Overall, the proposed