Researchers question how AI system can determine race from X-rays
A study has revealed that an artificial intelligence (AI) medical diagnostics system can recognize the race of a person just from an X-ray with over 90 percent accuracy. The researchers do not know how, reports the Boston Globe.
Concerns have been raised that the AI-powered diagnostic process could generate inherently biased results with the possibility of misleading human doctors by recommending a particular course of treatment for patients of a particular race.
Researchers from the Massachusetts Institute of Technology (MIT) and Harvard Medical School were part of the research project which included scientists from the United States, Australia, Canada and Taiwan. A paper explaining findings of the study was published last month in The Lancet Digital Health medical journal.
As the Boston Globe reports, the researchers began the process by training algorithms using standard data sets of X-rays and CT scans where each image, taken of different parts of the human body, was labeled with the person’s race. But the diagnostic images processed by the computer software had no obvious indications of the race of the persons from which the images were taken.
The system was able to tell the race of the people in the X-ray images, differentiating black and white patients, after it had examined race-labeled images and another set of unlabeled images.
Marzyeh Ghassemi, a MIT assistant professor of electrical engineering and computer science, and Leo Anthony Celi, an associate professor at Harvard Medical School, who co-authored the research report, say those who carried out the study are unable to explain exactly how the AI system was able to recognize patients’ races from their X-ray images. Ghassemi speculates that the scans may pick up the skin pigment melanin, even though it is not visible in them to human observers.
The Globe however quotes Alan Goodman, a professor of biological anthropology at Hampshire College and another coauthor of the research report, as saying he does not believe the results of the study are a reflection of the inborn differences that exist among people of different races. Goodman thinks the differences detected are likely caused by geography rather than race.
“Instead of using race, if they looked at somebody’s geographic coordinates, would the machine do just as well? My sense is the machine would do just as well. You call this race. I call this geographical variation,” said Goodman, although he also expressed uncertainty over how an AI system can determine geographical differences from just X-rays.
The researchers have advised medics against rushing to embrace such AI-based diagnostic systems that have the potential to automatically produce skewed results based on differences in certain human features, the Globe mentions.
Transparency and explainability in AI are seen as necessary to avoid problems including bias. Efforts to increase explainability in AI have included principles from NIST and a UK government standard for transparency.
Comments