Not so fast: driver monitoring systems put on biometric brakes for drunk drivers

Imagine, if you will, a 19th sequel in the Fast & the Furious franchise. Protagonist Dominic Toretto is angry at the latest sleight to his beloved family of misfits. He jumps into the cockpit of his souped-up Dodge Charger and slams his foot on the gas – but nothing happens. A cheery voice pipes up: “Dom, you’ve been working real hard lately. You’re tired. And do I detect a hint of Corona on your breath? Maybe best to give it til morning.”

Not, admittedly, a recipe for cinematic thrills – but a potential boon for driver safety. Swedish firm Smart Eye, which says its driver monitoring system (DMS) software has been installed into more than 2 million cars globally, has announced that an upgraded version of its Automotive Interior Sensing (AIS) product is “now equipped with real-time alcohol intoxication detection and over-the-air (OTA) capabilities.”

A release claims the system is first-of-its-kind in detecting alcohol-related impairment through DMS – “helping vehicle manufacturers meet growing regulatory demands while offering fleets a powerful tool for operational efficiency.”

Smart Eye’s AIS system consists of purpose-built software and hardware designed for aftermarket installation in commercial vehicles. It combines driver and cabin monitoring to track eye gaze, body key points, and other status data for everyone in the vehicle, using OmniVision RGB-IR image sensors.

Previous versions focused on detecting signs of driver drowsiness and distraction. The new alcohol impairment detection feature is based on “advanced analysis of eye and facial movements” to identify “behavioral patterns associated with intoxication and issues real-time alerts.” It also gauges driver crash probability with risk intelligence tech from Greater Than, which assigns a predictive risk score based on observed behavior.

All of the data processing aligns with GDPR regulations, including for customers outside the EU.

“With Euro NCAP and global regulators placing more emphasis on impairment detection, the timing of this release is no coincidence,” says Martin Krantz, CEO of Smart Eye. “Impairment detection has long been seen as a future goal for in-cabin technology, and we’ve invested in it for several years to have a solution in place. With this update to our AIS system, we’re proving it’s possible to turn safety innovation into a product that’s ready for the road.”

Blueskeye AI wants to keep drivers safe and happy

Dominic Toretto frowns – and the Charger notices. “Dom, I see you’re feeling a bit upset about this. Why don’t you think about something that makes you feel happy.” Toretto raises an eyebrow. “Now,” says the car, “I’m not sure more Coronas are a good idea. How about calling Letty?”

A UK firm is taking a recognized tool that measures how muscle actions cause visual facial changes and applying it to in-car monitoring systems. An article in the EE Times explains that Blueskeye AI is using deep neural networks to supercharge the Facial Action Coding System (FACS) annotator, also known as muscle action coder – a way to measure micro-expressions, which experimental psychologists have been using for decades.

Michel Valstar is chief scientific officer of Blueskeye AI, which was spun out of the University of Nottingham. Valstar says the firm uses temporal dynamics, “so our algorithms can infer the difference between somebody blinking in the sunlight versus an altered blink pattern because of fatigue. We can look at the acceleration of a smile; is its trajectory smooth or stilted?”

“The way people behave is a combination of social signals, emotional signals and expressions of their biophysical state. The complexity lies in what element is conscious and what element is autonomous, and that’s where a lot of our IP sits. Be it your gaze patterns, your blink rates, your smile – there’s so much information in those paths. People don’t really notice it, but our algorithms do.”

The piece notes changes in EU law mandating that all newly registered vehicles be capable of detecting driver drowsiness and distraction by 2026. The Euro NCAP program to which Valstar refers is the European New Car Assessment Programme, which only hands out five-star safety ratings to cars with driver monitoring systems. Valstar believes that “basically, every car in Europe is going to have a driver-facing camera, and once that’s in, we can really help with these safety elements.”

Transformer network ‘excels at processing sequential data’

Dom puts on his serious face and lays it out for the Charger. He says he’s not drunk. Besides, it’s not really the car’s business how many Coronas he’s had. Does the car know who he is? Has it seen Fasts one through eighteen? Toretto tells the car how he once beat up The Rock.

“Look,” the car says. “I don’t want to be forward. But frankly, I know you’re overcompensating because you’re depressed. And you should know, your liver’s in pretty rough shape.”

Intoxication is relatively easy to spot, but capturing emotional nuance requires immense amounts of training data. BlueSkeye AI only says it uses “images and audio from millions of people” to train its models to detect eye gaze direction, head posture estimation, facial point locations, tone of voice and more.

“From there, the models can be refined to recognize fatigue, depression, and apparent emotion.”

Blueskeye AI’s software is based on a transformer network, “a neural network that excels at processing sequential data.” Valstar has modified it to capture probability, so that certainty about different parts of the network can be communicated. “If face tracking is fine but there are uncertainties about the head pose, then inferences won’t be made based on that. By embedding probability on [network] components, we can propagate that throughout the entire network.”

After reading your hormones, your car can tell when you’re sick

More and more companies are exploring the potential in advanced driver monitoring systems. Germany’s trinamiX is developing its Invisible Biometric Sensing Display, using patented Beam Profile Analysis technology for biometric passenger monitoring.

A release says the system is built with a near-infrared camera and an eye-safe laser dot projector, making it capable of tracking vital signs such as heart rate without any physical contact. “The technology operates by capturing the reflections of light emitted by the laser dot projector in the invisible light spectrum, leveraging artificial intelligence algorithms to derive vital signs.” This allows it to identify stressful situations or potential medical emergencies in the car. A 5 MP wide-field-of-view camera enables monitoring of both the driver and the co-driver.

Michael Valstar says the sector is only going to go deeper. “As part of our long-term research and development, we’re looking at how hormones influence your expressive behavior. You’ll be able to start looking at some very small changes over time, which is great for, say, detecting degenerative disease. I expect that in 10 years’ time, you’ll be getting a free assessment of your health, just as a bonus of driving your car.”

Judge dismisses BIPA lawsuit against Hyundai over DMS

This is going too far, Dom thinks. Even if the car only has his safety in mind, it’s not family. Summoning the most intense frown he can muster, to make sure the car can sense his irritation, he gets out of the cockpit, slams the door and begins to walk away. But before he does, he turns, his face flushed with warning.

“Be careful,” Dom says. “Or I’ll drive you to Illinois.”

The car laughs.

“Maybe you didn’t hear. It’s in AutoBody news. A federal court in Illinois has dismissed a class action suit filed against Hyundai, under the Illinois Biometric Information Privacy Act (BIPA).”

“The ruling found that the carmaker did not ‘collect, capture, or otherwise obtain’ biometric data by use of its Forward Attention Warning System (FAWS), which monitors driver eye position using an infra-red camera. U.S. District Judge Matthew Kennelly said the complaint ‘lacked plausible allegations that Hyundai acquired or accessed biometric data through the system’.”

Dom can only shake his head. He is tired – tired of always being fast, being furious. How many times has he done this? Is anyone but his car watching? Maybe the Charger is right. Maybe it’s time to slow down.

He puts his hand on the hood. “Huh,” he says. “I think maybe you have a point.”

The car beeps playfully, and says, “I know.”

Article Topics

automotive biometrics  |  biometrics  |  BlueSkeye AI  |  consumer electronics  |  emotion detection  |  face biometrics  |  face detection  |  monitoring  |  Smart Eye  |  trinamiX