A topic that comes up in every interview.

Bias, variance, and their relationship with machine learning algorithms. One of the most basic concepts that you have to know by heart.

Here is a simple summary that you will easily remember.

Every machine learning algorithm deals with 3 types of errors:

1. Bias error
2. Variance error
3. Irreducible error

There's nothing we can do about #3.

Let's focus on the other two.

↓ 1/5
"Bias" refers to the assumptions the model makes to simplify the process of finding answers.

The more assumptions it makes, the more biased the model is.
"Variance" refers to how much the answers given by the model will change if we use different training data.

If the answers stay the same regardless of the data, the model has low variance.
Often, linear models are high-bias, and nonlinear models are low-bias.

Example low-bias algorithms:
• Decision Trees
• SVN
• kNN

Example high-bias algorithms:
• Linear Regression
• Logistic Regression
Often, linear models are low-variance, and nonlinear models are high-variance.

Example low-variance algorithms:
• Linear Regression
• Logistic Regression

Example high-variance algorithms:
• Decision Trees
• SVN
• kNN
Sometimes, you can change how these algorithms work to get a different tradeoff between their bias and variance.

Example:

• By increasing the value of "k" in kNN, we can increase the algorithm's bias.

• By pruning a Decision Tree, we can reduce its variance.
It doesn't matter what you do; the tradeoff is always there:

• Increasing bias decreases variance.
• Increasing variance decreases bias.

To work around this:

• Choose the appropriate algorithm
• Configure it correctly
• Work with the underlying dataset
If you want low-bias and low-variance machine learning content, follow me @svpino.

I come here to write about machine learning, and I promise you'll enjoy it.

More from Santiago

Free machine learning education.

Many top universities are making their Machine Learning and Deep Learning programs publicly available. All of this information is now online and free for everyone!

Here are 6 of these programs. Pick one and get started!



Introduction to Deep Learning
MIT Course 6.S191
Alexander Amini and Ava Soleimany

Introductory course on deep learning methods and practical experience using TensorFlow. Covers applications to computer vision, natural language processing, and more.

https://t.co/Uxx97WPCfR


Deep Learning
NYU DS-GA 1008
Yann LeCun and Alfredo Canziani

This course covers the latest techniques in deep learning and representation learning with applications to computer vision, natural language understanding, and speech recognition.

https://t.co/cKzpDOBVl1


Designing, Visualizing, and Understanding Deep Neural Networks
UC Berkeley CS L182
John Canny

A theoretical course focusing on design principles and best practices to design deep neural networks.

https://t.co/1TFUAIrAKb


Applied Machine Learning
Cornell Tech CS 5787
Volodymyr Kuleshov

A machine learning introductory course that starts from the very basics, covering all of the most important machine learning algorithms and how to apply them in practice.

https://t.co/hD5no8Pdfa

More from All

1. Mini Thread on Conflicts of Interest involving the authors of the Nature Toilet Paper:
https://t.co/VUYbsKGncx
Kristian G. Andersen
Andrew Rambaut
Ian Lipkin
Edward C. Holmes
Robert F. Garry

2. Thanks to @newboxer007 for forwarding the link to the research by an Australian in Taiwan (not on

3. K.Andersen didn't mention "competing interests"
Only Garry listed Zalgen Labs, which we will look at later.
In acknowledgements, Michael Farzan, Wellcome Trust, NIH, ERC & ARC are mentioned.
Author affiliations listed as usual.
Note the 328 Citations!
https://t.co/nmOeohM89Q


4. Kristian Andersen (1)
Andersen worked with USAMRIID & Fort Detrick scientists on research, with Robert Garry, Jens Kuhn & Sina Bavari among


5. Kristian Andersen (2)
Works at Scripps Research Institute, which WAS in serious financial trouble, haemorrhaging 20 million $ a year.
But just when the first virus cases were emerging, they received great news.
They issued a press release dated November 27, 2019:

You May Also Like

https://t.co/6cRR2B3jBE
Viruses and other pathogens are often studied as stand-alone entities, despite that, in nature, they mostly live in multispecies associations called biofilms—both externally and within the host.

https://t.co/FBfXhUrH5d


Microorganisms in biofilms are enclosed by an extracellular matrix that confers protection and improves survival. Previous studies have shown that viruses can secondarily colonize preexisting biofilms, and viral biofilms have also been described.


...we raise the perspective that CoVs can persistently infect bats due to their association with biofilm structures. This phenomenon potentially provides an optimal environment for nonpathogenic & well-adapted viruses to interact with the host, as well as for viral recombination.


Biofilms can also enhance virion viability in extracellular environments, such as on fomites and in aquatic sediments, allowing viral persistence and dissemination.