Loose lips, indeed. AI system hears when people only mouth words

Lip reading, a dreamed-of utility feature for uncounted AI use cases, appears near commercial availability. The news buoys the hopes of some and causes concern in others.

An Irish startup, Liopa, is trialing a phone application that reportedly can interpret simple phrases mouthed by people. The underlying algorithms do not require audio to be trained, which should streamline product development, market introduction and customer training.

In a Liopa marketing video, a young man who is unable to speak mouths short sentences looking into a phone camera, and pauses. The Speech Recognition App for the Voice Impaired, or Sravi, offers three guesses about what the person is trying to say.

Confirming with the subject and choosing an option on a phone screen trains Sravi on how that particular person silently forms words.

According to an article in Vice, the app might end up being the first such app that anyone could buy.

The same technology by Liopa has been researched as a tool to spot the liveness of a person. In February 2019, a Chinese research team not affiliated with Liopa, claimed its LipPass used closeups on mouths for biometric identification verification. It reportedly found spoofers 93 percent of the time.

Liopa is running out of time to meet its goal to launch commercially in the second quarter of this year. That, in turn, hints at delays in the company’s plan to support multiple languages by the end of the third quarter.

Company executives lean heavily into the heartstring-tugging med-tech opportunities — including aiding COVID patients on assisted breathing. Others who have spent a year in lockdown might welcome being able to read what was said unintentionally on mute during a video meeting.

But the real opportunity for Liopa and those to follow will be law enforcement, espionage, warfare and surveillance marketing — not necessarily in that order.

Sravi is yet another example of a biometrics tool that could pull anonymity from people in public and privacy from everyone else. In fact, executives spotlight the app’s reported ability to automatically monitor silent CCTV footage looking for pre-set words.

And the UK’s Defence and Security Accelerator is funding phase 2 of a project involving Sravi designed to understand how and why individuals, groups and large populations act as they do. The goal would be to train models to spot multiple aspects of behavior that lead to favorable or unwanted outcomes.

Article Topics

AI  |  biometrics  |  Liopa  |  lip motion  |  mobile app  |  research and development  |  smartphones  |  speech recognition