Voice biometrics restore patient’s ability to generate speech

Despite suffering a brainstem stroke, “Ann” can speak in her own voice using a biometric avatar controlled by her mind.

Ann, whose identity is being obscured to discuss details of her experience, was paralyzed after the stroke and lost the ability to speak at the age of 30.

An implant developed by researchers at University of California, San Francisco and University of California, Berkeley uses voice and face biometrics to generate speech and expression data from Ann’s brain signals, according to a UCSF article.

Researchers implanted a thin rectangle of 253 electrodes on the surface of areas in her brain that were critical for speech, intercepting the signals that would travel to the muscles in Ann’s face, tongue and mouth. A cable of electrodes bridges from a port in Ann’s skull to software to recognize her unique signals for speech.

An AI algorithm trained on a recording of Ann’s 2005 wedding speech generates Ann’s new voice. It translates brain signals as a patient attempts to speak, rather than just thinking about something.

The AI uses subunits of speech – phonemes – meaning it only needed to be trained on 39 sounds to generate any word in English. Her avatar is animated on a graphical mesh using expression signals for happiness, sadness and surprise.

This is the first time that speech or facial expressions have been synthesized from brain signals, the article claims. The system can decode signals at a median of 78 words per minute with a median 25 percent word error rate, compared to Ann’s current communication device’s 14 words.

Researchers are now developing a wireless version. Such a development could open the technique to many more who live severe paralysis.

“Our goal is to restore a full, embodied way of communicating, which is the most natural way for us to talk with others,” says Edward Chang, MD, chair of neurological surgery at UCSF.

Article Topics

biometrics  |  face biometrics  |  healthcare  |  research and development  |  voice biometrics