AI can identify race from X-rays, but no one knows why

Doctors cannot tell if someone is black, Asian or white just from an X-ray.But computers can do it, according to new research by a team of scientists from the Massachusetts Institute of Technology (MIT) and Harvard Medical School, among others.

Humans' judgment of whether someone is black, Asian, or white mainly comes from certain physical characteristics: external signs such as skin, hair, eyes, etc., doctors cannot judge someone just by looking at someone's X-ray.Whether the person is black, Asian or white.But recently, according to an MIT study, trained artificial intelligence can effectively identify whether the owners of these unlabeled X-rays are black, yellow, or white with more than 90 percent accuracy, even if theseImages are corrupted, cropped, and noisy medical images that are often beyond the reach of clinical specialists.

Also, the researchers stress that this identification was not due to certain associations of physical characteristics known to be associated with ethnicity (eg, through body mass index [AUC 0.55], disease distribution [0.61] andbreast density [0 61], etc.).

The findings are currently published in The Lancet Digit Health.

“When my graduate students showed me some of the results in this paper, I actually thought it must be a mistake,” said Marzyeh Ghassemi, assistant professor of electrical engineering and computer science at MIT and co-author of the paperSay, "I really thought my students were crazy when they told me."

To tease out and figure out how this magical recognition system works, the researchers ran a series of experiments.To investigate possible mechanisms of ethnicity detection, they looked at variables such as anatomical differences, bone density, image resolution, and more.However, putting these variables aside, the AI ​​still has a high ability to detect ethnicity through chest X-rays.

At a time when AI software is increasingly being used to help doctors make diagnostic decisions, this study raises the troubling prospect that AI-based diagnostic systems could inadvertently produce racially biased results.For example, AI (where X-rays are available) can automatically recommend a specific treatment plan for all black patients, regardless of whether it is best for a specific person.At the same time, the patient's human doctor would not know that the AI's diagnosis was based on racial data.