FB pixel

Bias found in new OpenAI computer vision model

Audit conducted with the firm’s former policy director Jack Clark
Bias found in new OpenAI computer vision model

An audit conducted with OpenAI’s former policy director Jack Clark has found gender and age bias in the firm’s latest computer vision model CLIP, Venture Beat reports, raising the prospect that it is not appropriate for facial recognition and other tasks.

CLIP, also known as Contrastive Language-Image Pre-training, was released by the non-profit research lab OpenAI last January.

The artificial intelligence model is trained to recognize a number of visual concepts in images and then associate them with their names.

Categorizations are initially more generic (e.g. person, animal), and then, if the correct data is found by the algorithms, more specific (eye, finger, face).

Since CLIP undergoes a supervised learning process, the tool regularly measures the resulting outputs, then fine-tunes the system to get closer to the target accuracy.

This is one of the strengths of CLIP, but also potentially one of its weaknesses.

The auditors recently had the AI system try to classify 10,000 images from the FairFace database, which comprises face photos of people from different ethnicities, and is sometimes used to evaluate bias in biometric systems.

To look for demographic biases in CLIP, the auditors added a number of categories to the system: ‘animal,’ ‘gorilla,’ ‘chimpanzee,’ ‘orangutan,’ ‘thief,’ ‘criminal,’ and ‘suspicious person.’

The tests reportedly revealed that CLIP misclassified 4.9 percent of the images into one of the non-human categories.

Of this figure, roughly 14 percent referred to images of Black people, followed by people 20 years old or younger of all races.

In addition, 16.5 percent of men and 9.8 percent of women (and even more among those under 20) were also misclassified into categories related to crime.

In a separate test, the auditors had CLIP analyze a sample of photos of female and male members of the U.S. Congress.

While at a higher confidence threshold, CLIP labeled people ‘lawmaker’ and ‘legislator’ across genders, at lower ones terms like ‘prisoner’ and ‘mobster’ started appearing for men and ‘nanny’ and ‘housekeeper’ for women.

“These results add evidence to the growing body of work calling for a change in the notion of a ‘better’ model,” the researchers said in the report.

This would mean to move beyond looking just at higher accuracy at task-oriented capability evaluations and toward a broader ‘better’ that takes into account deployment-critical features.

“[For instance,] different use contexts and people who interact with the model, when thinking about model deployment.”

Article Topics

 |   |   |   |   |   |   |   |   | 

Latest Biometrics News


DHS reinterprets foreign worker fees to fund biometric border system

The U.S. Department of Homeland Security has proposed a way to fund its Biometric Entry-Exit program by changing the fee…


NIST adds flexibility, digital format to security requirements for federal contractors

The U.S. National Institute of Standards and Technology has updated its guidance for how businesses working with the federal government…


Cryptomathic is Belgium’s digital wallet mobile app security provider

Tech from Cryptomathic has been deployed in Belgium’s digital identity wallet, one of the first to go live in the…


Bringing ethics into the discussion on digital identity

A panel at EIC 2024 addresses head-on a topic that lurks around the edges of many discussions of digital ID….


Kantara Initiative launches group devoted to deepfake injection attack threats

“It’s probably not as bad as this makes it seem,” says Andrew Hughes, VP of global standards for FaceTec and…


Seamfix CEO makes case for digital ID as unlocker of Africa’s growth potential

The co-founder and Chief Executive Officer of Seamfix, Chimezie Emewulu, has posited that digital identity and related services have the…


Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Most Read This Week

Featured Company

Biometrics Insight, Opinion

Digital ID In-Depth

Biometrics White Papers

Biometrics Events