MIT experts weigh in on how to govern AI and the risks it brings

As what is popularly called artificial intelligence or AI becomes ever-more integrated into society, a spectrum of views is emerging on how to govern its more controversial uses, and facial recognition in particular. Where academics, policymakers and regulators land on facial recognition may depend on how they see it – or, more accurately, how it sees them.

Low- and medium-risk uses of AI are already abundant

In an article on the Ideas Made to Matter blog of the MIT Sloan School of Management, journalist Beth Stackpole reports on a “red light, yellow light, green light” framework for AI governance proposed by Dominique Shelton Leipzig, author of the book Trust: Responsible AI, Innovation, Privacy and Data Leadership, at the EmTech MIT conference last November. Leipzig believes that the “promise of AI is super amazing, but in order to get there, there’s going to need to be some hovering,” in order to “protect your brand and have the opportunity to establish trust with your customers, employees, and business partners.”

Shelton Leipzig’s suggested framework, aimed at addressing concerns about safety and bias in AI, is based on draft legislation from 78 countries around the world. It is also based on traffic lights, categorizing AI use cases according to risk: red-light (prohibited), yellow light (high risk) and green light (low risk). Many green-light use cases are already widespread; Shelton Leipzig mentions chatbots and AI in video games as examples of uses that pose a low risk for bias. Red-light cases are relatively rare, with only 15 having been identified by legal frameworks. They include prohibited surveillance related to the exercise of democratic values, continuous surveillance in public spaces, remote biometric monitoring, and social scoring.

Yellow-light use cases are most common, with Shelton Leipzig noting up to 140 examples, such as using AI in family planning and care, HR applications, and for select uses in surveillance, democracy, and manufacturing. To address these and red-light cases, Shelton Leipzig proposes a protocol based on the EU’s AI Act and the White House’s Blueprint for an AI Bill of Rights. It recommends getting ahead of pending legislation by making sure data being used is accurate, appropriate, and legal to use; embracing continuous testing for algorithmic bias and regulatory compliance; allowing for human oversight; and building in fail-safes to halt “deviations.”

AI is here now; the bigger question is whether we actually want it

Joy Buolamwini prefers a much wider framework for interrogating AI – one that asks whether it is a good idea to begin with. Speaking to WBUR about her book, Unmasking AI: My Mission to Protect What Is Human in a World of Machines, the founder of the Algorithmic Justice League says she was enthusiastic about AI until it failed to recognize her dark skin. Her subsequent research at MIT revealed significant bias in the training data used for face biometrics.

“We had a lot of pale male data,” she says. However, her solution is not necessarily better or more data, which are still subject to misuse. “Accurate systems can be abused,” she says. “Think of a drone with a gun and facial recognition. We’re asking, do we even want that system to exist in the first place? And it goes beyond the question of data set bias or the accuracy of the model, to greater questions about what kind of society we want to live in, and what technologies do we permit in that society.” Buolamwini is ardently opposed to the use of facial recognition for mass surveillance, and says she is heartened to see a ban on the live use of FRT in public spaces in the EU AI Act.

Buolamwini says that one big problem is the imbalance in how companies sell the convenience of AI versus the disclosure of risk. She calls the present moment “a bit of a mystery meat phase when it comes to AI,” in which enthusiasm, convenience and innovation are being prioritized over diligence, and points to social media as a recent example of what can happen when mass adoption outpaces regulation, or even understanding.

“We actually have to test these systems to make sure they’re fit for purpose,” says Buolamwini.  “We can’t just buy into the marketing narrative or the AI hype and assume it’s going to work simply because a company told us, this is what it’s supposed to do. We need to be more rigorous, have more scrutiny. There are possibilities for how those tools could be used in a beneficial way. But we have to proceed with caution.”

Article Topics

AI  |  biometric identification  |  biometrics  |  facial recognition  |  regulation