Summary:
«What’s wrong with correlation?»
«In May 2016, the COMPAS algorithm was flagged as being racially biased [1]. This algorithm was used by the US to guide criminal sentencing by predicting likelihood of reoffending. It estimated a black person was more likely to re-offend than a white person with the same other background factors.
The problem was, the algorithm was mistaking correlation (patterns of crime in the past) with causation (that being black makes you more likely to commit a crime).
This can be a problem in medicine as well. Consider the following:
100 patients are admitted to hospital with pneumonia, of which 15 also have asthma. The doctors know asthma puts them at higher risk of getting more sick, so give them a more aggressive treatment. Because of this, the patients with asthma actually recover more quickly.
If we use this data to train a model, and aren’t careful, the model may conclude that asthma actually improves recovery. As a result, it may recommend treating less aggressively. Of course, we can see this is wrong – but to an AI model it’s not so obvious.»
Article written by Chris Lovejoy
31|10|2020
Source:
Chris Lovejoy