AI Job Interviews - another good example of bias in ML 🤦‍♂️

Two journalists tested some AI tools for assessing job candidates. Even when the candidate read a Wiki article in German instead of answering questions in English, the AI systems gave them good scores 🤷‍♂️

Let's unpack 👇

The Setup 🔬

The journalists created a fake job posting on two AI interview platforms. They specified the traits of the ideal candidate and provided the questions that need to be answered during the interview.

Then they started experimenting... 👇

The Positive Test ✅

One of them did a fake interview giving all the right answers and predictably got very high scores - 8.5 out of 9 👍

Then she tried something different... 👇

The Negative Test ❌

In a second interview, instead of answering the questions in English, she just read the article on psychometrics from the German Wikipedia 😁

One system gave her a score of 6 out of 9, while the other determined she is a 73% match for the job.

Oops... 👇

What happened? 🔍

Interestingly, one of the systems generated a transcript which was obviously meaningless.

This means that the machine learning model behind the tool likely captured nuances of the intonation of the speaker instead of the meaning of the actual words.

👇

This is Karma. Karma is not a machine learning classifier 🐕‍🦺

Karma is a real dog trained to detect drugs. However, he would fail the simplest tests we apply in ML...

Let me take you through this story from the eyes of an ML engineer.

https://t.co/WAXRUlTvSI

Thread 🧵

Story TLDR 🔖

The story is about police dogs trained to sniff drugs. The problem is that the dogs often signal drugs even if there are none. Then innocent people land in jail for days.

The cops even joke about the “probable cause on four legs”.

Let's see why is that 👇

1. Sampling Bias 🤏

Drugs were found in 64% of the cars Karma identified, which was praised by the police as very good. In the end, most people don't carry drugs in their cars, so 64% seems solid.

There was a sampling problem though... 👇

The cars were not sampled at random! The police only did the sniff test if there was a serious suspicion that something is wrong.

The chance there are drugs in the car is much higher in this case!

2. Evaluation Metrics 🔍

The police referred to a 2014 study from Poland measuring the efficacy of sniffer dogs. The problem was that every test actually contained drugs!

This means there was no chance to measure false positives from the dogs! Only recall, not precision 🤦‍♂️

Dataset bias strikes again! 😬

This is a great example of dataset bias! This ML method classifies skin lesions as being malignant or not.

Can you see the problem from the examples? 🤔

Let me shortly explain 👇

The dataset used to train the model contains images of both malignant and benign lesions. The problem is that the malignant images often contain rulers printed on the skin. They are used by doctors to assess the lesion over time.

Guess what the model learned... 👇

The ML model learned to detect the rulers... 🤷‍♂️

"In our dataset, images with rulers were more likely to be malignant; thus the algorithm inadvertently “learned” that rulers are

Machine Learning in the Real World 🧠 🤖

ML for real-world applications is much more than designing fancy networks and fine-tuning parameters.

In fact, you will spend most of your time curating a good dataset.

Let's go through the process together 👇

#RepostFriday

Collect Data 💽

We need to represent the real world as accurately as possible. If some situations are underrepresented we are introducing Sampling Bias.

Sampling Bias is nasty because we'll have high test accuracy, but our model will perform badly when deployed.

👇

Traffic Lights 🚦

Let's build a model to recognize traffic lights for a self-driving car. We need to collect data for different:

▪️ Lighting conditions
▪️ Weather conditions
▪️ Distances and viewpoints
▪️ Strange variants

And if we sample only 🚦 we won't detect 🚥 🤷‍♂️

👇

Data Cleaning 🧹

Now we need to clean all corrupted and irrelevant samples. We need to remove:

▪️ Overexposed or underexposed images
▪️ Images in irrelevant situations
▪️ Faulty images

Leaving them in the dataset will hurt our model's performance!

👇

Preprocess Data ⚙️

Most ML models like their data nicely normalized and properly scaled. Bad normalization can also lead to worse performance (I have a nice story for another time...)

▪️ Crop and resize all images
▪️ Normalize all values (usually 0 mean and 1 std. dev.)

👇