Let's assume you built two different models:
• Model 1: A ResNet model.
• Model 2: A one-shot model (Siamese network.)
They both solve the same problem, so you want to combine their results to pick the right answer.
Do you know what's better than a machine learning model?
Two models.
More than one model working together to solve a problem is called "an emsemble." A simple way to build this is having each model vote for an answer.
But there's a problem with this approach: ↓
Let's assume you built two different models:
• Model 1: A ResNet model.
• Model 2: A one-shot model (Siamese network.)
They both solve the same problem, so you want to combine their results to pick the right answer.
What happens in this case?
• Model 1's answer: Class A
• Model 2's answer: Class B
Which one do you select?
You could have 3 models, each giving you a different answer.
How do you decide which answer to choose?
Important: Some of these ideas might not be feasible depending on your context. They have worked for me before on different situations, but every problem is different.
• Take 6 months' worth of data
• Compute the prior probability of every class
• Run the data through your ensemble
• Track the results of the models
• Use performance and priors to weight these results
Let's try to break these down.
If I tell you that I saw a plane, you would believe me. But how about if I tell you I saw a UFO?
Planes have a higher chance of being the correct answer.
For example, Model 1 might be really good at identifying planes, but Model 2 may constantly make mistakes.
This should tell us how much we should believe the results from each model.
In the case of the ResNet model, the softmax probability. In the case of the one-shot model, the similarity score.
The ensemble then becomes:
• Model 1
• Model 2
• Model 3 ← This one is the new model deciding which answer to pick.
Sometimes, a simple heuristic might be a good enough solution.
It's our job to weigh the pros and cons. Better performance is just one side of the equation.
Do you have any experience dealing with ensemble voting? Any other ideas that come to mind on how to tackle this problem?
More from Santiago
Here is a simple example of a machine learning model.
I put it together a long time ago, and it was very helpful! I sliced it apart a thousand times until things started to make sense.
It's TensorFlow and Keras.
If you are starting out, this may be a good puzzle to solve.
The goal of this model is to learn to multiply one-digit
I put it together a long time ago, and it was very helpful! I sliced it apart a thousand times until things started to make sense.
It's TensorFlow and Keras.
If you are starting out, this may be a good puzzle to solve.
The goal of this model is to learn to multiply one-digit
It is a good example of coding, what is the model?
— Freddy Rojas Cama (@freddyrojascama) February 1, 2021
You gotta think about this one carefully!
Imagine you go to the doctor and get tested for a rare disease (only 1 in 10,000 people get it.)
The test is 99% effective in detecting both sick and healthy people.
Your test comes back positive.
Are you really sick? Explain below 👇
The most complete answer from every reply so far is from Dr. Lena. Thanks for taking the time and going through
You can get the answer using Bayes' theorem, but let's try to come up with it in a different —maybe more intuitive— way.
👇
Here is what we know:
- Out of 10,000 people, 1 is sick
- Out of 100 sick people, 99 test positive
- Out of 100 healthy people, 99 test negative
Assuming 1 million people take the test (including you):
- 100 of them are sick
- 999,900 of them are healthy
👇
Let's now test both groups, starting with the 100 people sick:
▫️ 99 of them will be diagnosed (correctly) as sick (99%)
▫️ 1 of them is going to be diagnosed (incorrectly) as healthy (1%)
👇
Imagine you go to the doctor and get tested for a rare disease (only 1 in 10,000 people get it.)
The test is 99% effective in detecting both sick and healthy people.
Your test comes back positive.
Are you really sick? Explain below 👇
The most complete answer from every reply so far is from Dr. Lena. Thanks for taking the time and going through
Really doesn\u2019t fit well in a tweet. pic.twitter.com/xN0pAyniFS
— Dr. Lena Sugar \U0001f3f3\ufe0f\u200d\U0001f308\U0001f1ea\U0001f1fa\U0001f1ef\U0001f1f5 (@_jvs) February 18, 2021
You can get the answer using Bayes' theorem, but let's try to come up with it in a different —maybe more intuitive— way.
👇
Here is what we know:
- Out of 10,000 people, 1 is sick
- Out of 100 sick people, 99 test positive
- Out of 100 healthy people, 99 test negative
Assuming 1 million people take the test (including you):
- 100 of them are sick
- 999,900 of them are healthy
👇
Let's now test both groups, starting with the 100 people sick:
▫️ 99 of them will be diagnosed (correctly) as sick (99%)
▫️ 1 of them is going to be diagnosed (incorrectly) as healthy (1%)
👇
