Research into protections against speech analysis privacy threats maturing rapidly

Our voice reveals much more about us than we may realize: The biometric information of our speech contains information about our identity, but also emotional state, education level and even health.  Over the past years, speech analysis technology has been developing rapidly, with experts expressing concern that voice could become another frontier in privacy preservation.

Tom Bäckström, an associate professor at Aalto University in Finland, recently published a research paper on privacy in speech technology in the IEEE Xplore digital library. The paper argues that although the current technology may not be able to extract all this information just yet, researchers should develop protective measures before it becomes a threat to privacy.

The paper notes that speech signals can yield information on the speaker’s biology, including health and whether they are intoxicated, their psychological state, affiliations, relationships and physical environment.

Potential threats to privacy include insurance companies that can detect that someone has a health condition such as Parkinson’s, advertising that exploits people’s emotional state, or extracting information from voice conversations that could be used for harassment, stalking and extortion.

Attackers can obtain both the target information, or the message of the speech, or the speech signals, which contain additional information beyond the core message. Aside from their health and emotional state, the speech could potentially reveal whether the speaker is intoxicated, their socioeconomic background, gender, ethnicity, as well as affiliations.

Background sounds can also reveal not just the location but also the distance from the sensor and what kind of hardware or software is being used.

The paper lists three categories of attacks: motivated, opportunistic, and incidental.

In motivated attacks, hackers target specific individuals in two ways. They may extract private details — such as health information — from public speeches or recordings. Alternatively, they may scan databases of anonymous speakers to identify and reidentify their target, effectively unmasking someone whose identity was supposed to remain hidden by anonymizing their information.

The third category is incidental attacks, where systems accidentally expose information. A speech device might misinterpret a command and trigger functions that reveal private data to unauthorized users.

The research also classifies attack scenarios based on the extent to which the speech is associated with other data and the attacker has access to this additional information. The report describes attempts in the VoicePrivacy 2022 challenge to break anonymization and pseudonymization protections shielding different elements and amounts of identity information.

The research also proposes several measures to secure the privacy of speech, including filtering private information, securing processing through cryptography and implementing privacy-preserving architectures in which information is only stored locally or on a distributed network of devices.

One of the protection measures suggested is implementing privacy-by-design principles. Even if individual modules of a system can assure privacy, their interaction can still lead to information leaks. Privacy, therefore, must be ensured at a more basic level.

A biometric reference, which stores a user’s attributes such as identity, should be unlinkable, meaning that comparing two or more biometric references will not reveal whether they are from the same or different persons. It should also be irreversible, meaning that the biometric sample cannot be derived from the biometric reference.

Finally, the principle of renewability should ensure that multiple, unlinkable biometric references can be created from the same biometric samples and used to identify the person without revealing information about the biometric samples.

“Together, unlinkability and irreversibility, thus, minimize the amount of information leaked if biometric references are revealed,” the paper notes. “At the same time, renewability ensures that leaked references can be revoked and new ones created to replace them.”

Privacy issues related to speech technology are expected to gain more attention in the future, as the technology becomes widely adopted.

Biometric speaker recognition and verification are used in banking apps and to recognize a user of a device. Speech-to-text is now integrated to dozens of applications, while wake-word detection for voice-enabled devices such as “Hey Siri” or “Alexa” has become ubiquitous. Another technology that may extract information is speech enhancement, which removes background noises and distortions.

A senior member of the Institute of Electrical and Electronics Engineers (IEEE). Bäckström has been working on the issue of privacy in speech technology as part of the International Speech Communication Association (ISCA) special interest group on security and privacy in speech communication (SPSC).

The researcher believes that ensuring privacy does not have to become a burden for companies, but could even improve products and services. Removing private information from speech, for instance, could mean less data transfer and network traffic costs, Bäckström said in an interview with Aalto University.

Bäckström ultimately concludes that the range of privacy threats and the attack surface are “vast” and dynamic, and that ethical issues around speech largely overlap with privacy concerns. The field is still fairly new, and more research is needed, but stronger protections are likely on the way, as he says research into speech privacy is “in a phase of rapid development.”

Article Topics

biometric identification  |  biometrics  |  biometrics research  |  data privacy  |  data protection  |  speech recognition  |  voice analysis  |  voice biometrics