Military takes robot interaction to the next damn level with new speech recognition system

Waking up Alexa or Google with a question works well enough in the relatively unchanging home environment. But when trying to control robots in a combat environment, asking Alexa politely to find the source of enemy fire is a whole different challenge. That’s why U.S. Army and academic researchers joined forces to develop a better way to interact and control autonomous systems such as mobile robots.

Researchers from the U.S. Army Combat Capabilities Development Command’s Army Research Laboratory (ARL), in collaboration with the University of Southern California’s Institute for Creative Technologies, developed a new conversational model called the Joint Understanding and Dialogue Interface (JUDI) capability. The approach to speech recognition “enables bi-directional conversational interactions between soldiers and autonomous systems,” according to a statement from the U.S. Army.

Devices such as Alexa rely on network connectivity to cloud services which have large, labeled datasets to learn and perform tasks for users. JUDI’s dialogue processing, on the other hand, is based on a statistical classification method that interprets a soldier’s intent from their spoken language.

According to the ARL, the classifier was trained on a small dataset of human-robot dialogue where human experimenters stood in for the robot’s autonomy during initial phases of the research. The speech recognizer also leveraged a speech model developed as part of the Intelligence Advanced Research Projects Activity’s Babel program, which was designed specifically for “reverberant and noisy acoustic environments.” The JUDI system leverages those speech technologies while taking autonomous devices into consideration, meaning that it uses multiple sources of context, like a soldier’s speech as well as the robot’s perception system (be it visual, audio or radar mapping, for instance) to help in collaborative decision-making.

What this means is that in a real combat situation where a robot is in a new place and soldiers are under pressure in a “noisy” environment, a more natural two-way interaction can occur. And yes, it did occur to people that soldiers might use colorful language in the heat of battle.

“We want soldiers to feel less constrained, and we know soldiers won’t want to memorize a list of commands to interact with an AI agent, and JUDI reduces that burden,” said Dr. Matthew Marge, a research scientist with ICT told Task & Purpose, a publication covering the U.S. military. “The algorithm listens to the verbal instruction and finds the highest word overlap to apply to a direction,” he added.

The goal “is to enable Soldiers to more easily team with autonomous systems so they can more effectively and safely complete missions, especially in scenarios like reconnaissance and search-and-rescue,” Marge said.

According to the ARL, the technology will be folded into a number of other projects around robotics and AI projects. JUDI will be integrated into the U.S. Army Combat Capabilities Development Command’s (CCDC) ARL Autonomy Stack, a suite of software algorithms, libraries and software components that perform specific functions required by intelligent systems such as those for navigation, planning, perception, control and reasoning. The resulting Army owned intellectual property will be shared with industry partners.

As is the case with many technologies (including the internet itself), the idea behind JUDI could find its way into the broader commercial market. Systems that integrate biometric technologies for identification could be combined with JUDI’s speech command system to enable a generation of devices that are “paired” with specific owners, much like service dogs are trained to respond to individual police officers, for example. Different users could have access to different sets of functionalities based on facial biometrics and speech recognition might be another example of how JUDI’s approach to colorful conversation could be applied outside of the military.

Article Topics

AI  |  biometrics  |  biometrics research  |  robots  |  speech recognition  |  U.S. Army