Towards Speech Synthesis from Intracranial Signals

Abstract

Brain-computer interfaces (BCIs) are envisioned to enable individuals with severe disabilities to regain the ability to communicate. Early BCIs have provided users with the ability to type messages one letter at a time, providing an important, but slow, means of communication for locked-in patients. However, natural speech contains substantially more information than a textual representation and can convey many important markers of human communication in addition to the sequence of words. A BCI that directly synthesizes speech from neural signals could harness this full expressive power of speech. In this study with motor-intact patients undergoing glioma removal, we demonstrate that high-quality audio signals can be synthesized from intracranial signals using a method from the speech synthesis community called Unit Selection. The Unit Selection approach concatenates speech units of the user to form new audio output and thereby produces natural speech in the user's own voice.