Researchers say emotion detection based on biometric facial recognition not reliable

Emotion detection based on biometric facial recognition has grown to a $20 billion industry, but facial movements and expressions are not a reliable indication of how someone is feeling, according to research conducted by scientists brought together by the Association for Psychological Science and reported by The Washington Post.

A team of five researchers reviewed more than 1,000 studies, and concluded that the relationship between people’s facial expressions and emotions are not universal, and are nebulous and complicated.
“About 20 to 30 percent of the time, people make the expected expression,” Northeastern University Psychology Professor Lisa Feldman Barret told the Post. “They’re not moving their faces in random ways. They’re expressing emotion in ways that are specific to the situation.”

Humans make assessments about other people’s emotions based on factors including body language and tone of voice, the Post reports, but artificial intelligence systems for detecting emotion mostly rely on facial recognition, following the work of University of California at San Francisco Psychology Professor Paul Ekman, who argued that there are universal facial expressions for six emotions.

The researchers say to deliver the kind of performance they claim, companies must use different analytic strategies. Barret says training programs to consider other types of data, such as body positioning and situational context, could yield better results.

A 2013 review by the U.S. Government Accountability Office of a 2007 Transportation Security Administration program to identify terrorists from facial expressions and behavior found that the TSA did not establish a scientific basis for the process.

Affectiva is identified by the Post as an exception to the usual technique, as it uses naturalistic video rather than still images, and takes input other than facial data into account. The company’s CEO Rana el Kaliouby compares the state of the industry to a toddler, who can understand basic emotions, but has a limited grasp of more complex states. Affectiva trains its systems with more than 8 million faces from 87 countries, and incorporates culturally specific characteristics.

Biometric analysis of facial movements is enough to tell real and fake smiles apart, according to researchers at the University of Bradford. Science Daily reports that a study published in Advanced Engineering Informatics shows that software can detect movements, particularly around a person’s eyes, which make clear whether or not a smile is genuine.

The research was led by University of Bradford Professor of Visual Computing Hassan Ugail.

“A smile is perhaps the most common of facial expressions and is a powerful way of signalling positive emotions,” says Ugail. “Techniques for analysing human facial expressions have advanced dramatically in recent years, but distinguishing between genuine and posed smiles remains a challenge because humans are not good at picking up the relevant cues.”

The researchers tested two data sets, one with real smiles and another with posed smiles.

“We use two main sets of muscles when we smile — the zygomaticus major, which is responsible for the curling upwards of the mouth, and the orbicularis oculi, which causes crinkling around our eyes,” explains Ugail. “In fake smiles it is often only the mouth muscles which move but, as humans, we often don’t spot the lack of movement around the eyes. The computer software can spot this much more reliably.”

Ugail suggests the research could be useful for improving interactions between people and computers, such as biometric identification, as well as to research on human behavior and emotion.

Johannesburg-based facial recognition startup Camatica, meanwhile, has launched “mood analytics” to its suite of AI facial recognition products, according to Business Reports.

“There’s nothing creepy about understanding when employees are experiencing challenges and putting in place solutions to help them,” says Camatica Co-founder Laurence Seberini. “Using Artificial Intelligence (AI) to help a busy boss know when it’s time for a kind word or a warm cup of cocoa is a smart move, not an invasion of privacy.”

Seberini says responsible use of facial recognition by HR professionals can also track employee attendance more reliably than some legacy systems. He advises companies in South Africa to draft AI policies and procedures.

“We specialise in facial recognition with a purpose. Camera-centred AI can make a huge positive difference in the HR environment by rewarding excellent behaviour and bringing certainty to attendance, eliminating potential issues later on,” Seberini says.

Article Topics

artificial intelligence  |  biometrics  |  emotion recognition  |  expression recognition  |  facial recognition