For image recognition systems the world still looks like ‘Daddy Knows Best’ — research

A U.S.-European team of researchers say image recognition systems “mimic and even amplify real-world bias” involving men and women. The bias was “systemic and pervasive,” according to a new research paper.

It is hard to argue that the results of the research would not tack closely with the performance of some facial recognition systems.

Using anodyne images of a pair of U.S. House representatives, one male and one female, the researchers found that image-labeling algorithms see genders — particularly females — through a filter that might as well be 70 years old.

The paper found that women’s images get three times more annotations about their appearance compared to men, and that was when women’s presence in images were even recognized. The researchers said that women are recognized at “substantially lower rates” than are men.

This may not be entirely surprising, given the difficulties some face biometric algorithms have shown in identifying women.

It appears that image recognition systems are labeling, tagging and categorizing more images of men than of women.

The researchers conducted most of their analysis on Google’s Cloud Vision (GCV), which they found is more widely used commercially than competitors’ including Amazon.com’s Rekognition and Microsoft’s Azure Computer Vision.

GCV did not recognize women in images 74 percent of the time, but correctly noted them as absent from an image 99 percent of the time. Men in images were not recognized 55 percent of the time by GCV, but were correctly noted as missing 98 percent of the time.

GCV’s labeling showed more bias.

An image of Congresswoman Lucille Roybal-Allard was labeled as a “smiling” “television presenter” with “black hair.” Senator Steve Daines was labeled as being an “official,” “businessperson” and “spokesperson.”

(Two images of U.S. members of Congress with their corresponding labels as assigned by Google Cloud Vision: Courtesy Carsten Schwemmer)

That is not unusual for photos of each gender in the eyes of GCV.

In a control data set of members of congressional portraits on Wikipedia, the top 10 labels (by absolute frequency) attached to men’s’ photos are: official, businessperson, spokesperson, speaker, suit, elder, gentlemen, person, necktie and public speaking.

The same list for women’s image labels is: smile, chin, outerwear, hairstyle, portrait, television presenter, long hair, brown hair, hair coloring and girl (nowhere on the men’s list is “boy”).

Nineteen of the top 25 labels attached to women’s images described appearance (including the word “beauty). Just nine labels on men’s images dealt with physical appearance, and six of those were related to clothing, like “suit.”

It gets worse.

Creating a data set of images posted by members of Congress on Twitter shows that virtually every label attached to women’s photos is specific to appearance or, for lack of a better term, desires.

The top 10 are: fun, social group, smile, girl, outerwear, youth, purple, fashion, red and friendship.

The men’s list is dominated by roles and goals of social attainment: official, suit, vehicle, public speaking, speech, technology, car, meeting, speaker and formal wear.

Article Topics

accuracy  |  algorithms  |  biometrics  |  biometrics research  |  dataset  |  facial recognition  |  image recognition