FB pixel

What is emotion recognition, and how can biometrics be used to perform it?

What is emotion recognition, and how can biometrics be used to perform it?
 

At a basic level, emotion recognition (ER) is the process of identifying human sentiment. While there have been many contributions to this discipline from a psychological and philosophical standpoint, the same cannot be said about technology. Attempting to identify human emotion using technology is a relatively new field and one that has been met with mixed results.

Within the realm of these applications, however, biometric approaches have shown promise, particularly face biometrics. The combination of different physiological and behavioral data types, and use of multimodal biometrics have also helped the discipline move forward.

But how can biometric scanners and algorithms measure human emotion in practical terms?

Face biometrics

Most ER tools based on face biometrics work via readings of physiological responses and facial expressions. The latter are categorized into discrete emotions and typically include amusement, awe, enthusiasm, liking, surprise, anger, disgust, fear, and sadness.

As described in a 2018 study by Universiti Putra Malaysia, facial emotion recognition (FER) processes are usually divided into three stages, face detection, feature extraction (nose, mouth, etc.), and emotion classification.

For a FER system to work, in the last stage of this process, a classifier needs to be trained in order to generate labels for individual emotions using the training data. Different facial coding systems have been developed to classify emotions based on the movements of specific facial muscles.

Voice recognition

ER based on voice biometrics is still in its infancy. Still, as voice recognition algorithms become increasingly more efficient, researchers and companies alike are working to apply the technology to measuring sentiment.

Technically, speech ER (SER) can rely on three distinct characteristics of the human voice, namely lexical features (vocabulary), visual features (facial expressions) and acoustic features (sound properties).

According to a 2020 study by Brillio, SER systems relying on lexical features require a transcript of the speech alongside text extraction capabilities to predict emotions from real-time audio. Those depending on visual elements, on the other hand, rely on the analysis of the video of the conversations, making it a form of multimodal biometric ER.

Because of these limitations, many SER rely on acoustic features, which can be analyzed in real-time. The Brillio studio mentioned here, in particular, tried to classify emotions based on two methods.

The first one, called discrete classification, is akin to what is mentioned above for FER, categorizing emotions in discrete labels like anger, happiness, boredom, etc. The second one, “dimensional representation,” aims to represent emotions with dimensions such as Valence (on a negative to positive scale), Activation or Energy (on a low to high ranking) and Dominance (on an active to passive scale).

It is helpful to mention here that with both face and voice biometrics ER tools (as well as standalone devices), wearables are often used to record physiological data, including cardiovascular information, to identify emotions more accurately.

Gait recognition

Methods of reading individuals’ emotions from their gait biometrics are slowly improving, but the technology is still relatively far away from commercial applications.

Still, there have been several attempts by researchers and governments to turn gait ER (GER) into a viable method of assessing a person’s emotional status.

According to a non-peer-reviewed study published by Chinese researchers in September 2022, for instance, it is possible to assess an individual’s emotion based on the position of certain bones in their skeleton as they walk.

The algorithm used in this study, called multi-scale adaptive graph convolution network (MSA-GCN), is intended to extract general gait data and create fine-grained graphs to uncover local information.

A similar technique (generally understood as the most widespread technique to perform GER at the time of writing) was presented in a separate paper published in January 2022 by a joint team of Canadian and Taiwanese researchers.

Legal implications

Biometric surveillance is generally among the most criticized methods of technocratic control by authoritarian regimes worldwide, and is often associated with emotion recognition. ER to many people evokes dystopian images of mind-controlled citizens, immediately punished for thinking or feeling something they should not.

In China, for instance, the government has deployed facial recognition tools to record students’ moods or ones allegedly capable of testing loyalty to the Party.

Beyond the alleged and actual Orwellian schemes associated with ER deployments, these technologies are finding fertile ground among lawyers in the U.S., who have been reportedly using them to assess the emotions of panels hired by attorneys before trials as stand-ins for juries.

Meanwhile, UK politicians are treading carefully regarding emotion recognition technologies. The country’s Information Commissioner’s Office (ICO) issued a warning against the deployment of these technologies in October 2022.

Article Topics

 |   |   |   | 

Latest Biometrics News

 

Ukraine’s biometrics market grows unevenly as digital identity adoption expands

Ukraine‘s biometrics sector continues to develop despite Russia’s ongoing attacks and economic pressures, although adoption remains uneven across government, financial…

 

Cloudflare, Tools for Humanity propose identity framework for AI agent access

Websites can increasingly identify the AI agents accessing their services, but not necessarily the people behind them, what actions they…

 

Biometrics take on new roles as trust and security demands grow

Biometrics are being asked to do more than establish identity. From proving presence in remote transactions to estimating age without…

 

America is building an online ID system by lawsuit

By Raj Ananthanpillai, CEO, Trua Meta has made roughly $5.3 billion of its $18 billion settlement with state and territorial…

 

Listen: C. Maxine Most puts 25 years of biometrics predictions to the test

This year, identity veteran and biometrics market strategy guru C. Maxine Most is celebrating 25 years of her company Acuity…

 

Google proposes using digital credentials for secure account recovery

Google is developing an account recovery method using digital identity credentials to address security weaknesses in traditional recovery methods when…

Comments

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Market Intelligence

Biometric Update Podcast

Featured Company

Top Stories This Week

White Papers

Latest Webinars

Biometrics Industry Events