Turning that expression of frustration into code: A new human-machine interface

A coder has created a new computer interface that uses more-than-subtle facial expressions and used the software to complete an apparently real job interview code test. The greatest innovation, however, might be giving “duckface” a legitimate reason to exist.

Fletcher Heisler, director of developer enablement at application security firm Veracode, has posted a video demonstrating how he built and tested his interface, called CheekyKeys. It incorporates facial recognition and expression tracking.

The coding appears to be light duty, the commands look tiring but the tale itself is very entertaining. Much work continues in this field, some of it worth more than an eyeroll.

An article at Hackaday.com recounts how Heisler conceived of his innovation as a new father wondering if there was a way to hold his newly arrived sleeping baby while coding.

His first stab was to learn Morse code, which he could transmit with blinking (and, eventually, jaw movements) accompanied by some facial gestures and interpreted by biometrics-based software.

Heisler used Google’s machine learning software, called MediaPipe, principally designed for tasks like animating facial image filters, to track almost 500 facial landmarks.

So armed he dreamed up commands and modifiers, including the infamous but omnipresent duckface pose women and girls strike in ironic Instagram fashion poses.

Heisler’s dry delivery, especially as he makes his interview call, is fun, but there is no getting around the fact that watching an adult perform stationary facial kabuki — and ace his test — is a treat.

Article Topics

biometrics  |  expression recognition  |  face biometrics  |  gesture recognition  |  machine learning  |  research and development