An NN is like a stack of pancakes, with computation flowing up when we make predictions.
How does it all work?
THREAD: How is it possible to train a well-performing, advanced Computer Vision model ๐ผ๐ป ๐๐ต๐ฒ ๐๐ฃ๐จ? ๐ค
At the heart of this lies the most important technique in modern deep learning - transfer learning.
Let's analyze how it
THREAD: Can you start learning cutting-edge deep learning without specialized hardware? \U0001f916
— Radek Osmulski (@radekosmulski) February 11, 2021
In this thread, we will train an advanced Computer Vision model on a challenging dataset. \U0001f415\U0001f408 Training completes in 25 minutes on my 3yrs old Ryzen 5 CPU.
Let me show you how...
An NN is like a stack of pancakes, with computation flowing up when we make predictions.
How does it all work?
An image is a collection of pixels. Each pixel is just a bunch of numbers describing its color.
Here is what it might look like for a black and white image
Each layer performs computation on the image, transforming it and passing it upwards.
The outputs of a layer are called activations, and the outputs of the last layer have a special meaning... they are the predictions!
We would like for it to have a high value, and for other activations in the last layer to be small...
So far so good! But what about transfer learning?
We know that these layers evolve during training to become feature detectors.
What do we mean by that?
A layer above might have windows that construct shapes from these lines.
We might have a window light up when it sees a square, another when it sees a colorful blob.
Maybe one sliding window will combine lines and detect text... maybe another one will learn to detect faces.
Does all of this sound like a hard task?
But, presumably, once we detect all these lower-level features, we can combine them in a plethora of interesting ways? ๐ค
Or trains, planes, and ships. Or blood cell boundaries. Or aneurysms in x-rays. The possibilities are endless!
We let researchers, large corporations, spend millions of dollars to train very complex models.
And then we get to build on top of their work! ๐
But so much for the theory. How does it all work in practice?
The @fastdotai framework downloaded the model for us and removed the top of it (the part responsible for predicting 1 of 1000 classes).
During training, we kept nearly the entire model frozen, and only trained the uppermost part, making use of all the lower level features that were being detected.
Ingenius! ๐
We will hopefully get a chance to explore all of them ๐
If you enjoyed this thread, let me know please and help me reach others who might also be interested ๐๐
And the visualizations of what the layers can detect?
Next stop - deciphering how it all works in code and finding ways to further improve our model!
Stay tuned for more ๐
More from Tech
The YouTube algorithm that I helped build in 2011 still recommends the flat earth theory by the *hundreds of millions*. This investigation by @RawStory shows some of the real-life consequences of this badly designed AI.
This spring at SxSW, @SusanWojcicki promised "Wikipedia snippets" on debated videos. But they didn't put them on flat earth videos, and instead @YouTube is promoting merchandising such as "NASA lies - Never Trust a Snake". 2/
A few example of flat earth videos that were promoted by YouTube #today:
https://t.co/TumQiX2tlj 3/
https://t.co/uAORIJ5BYX 4/
https://t.co/yOGZ0pLfHG 5/
Flat Earth conference attendees explain how they have been brainwashed by YouTube and Infowarshttps://t.co/gqZwGXPOoc
— Raw Story (@RawStory) November 18, 2018
This spring at SxSW, @SusanWojcicki promised "Wikipedia snippets" on debated videos. But they didn't put them on flat earth videos, and instead @YouTube is promoting merchandising such as "NASA lies - Never Trust a Snake". 2/
A few example of flat earth videos that were promoted by YouTube #today:
https://t.co/TumQiX2tlj 3/
https://t.co/uAORIJ5BYX 4/
https://t.co/yOGZ0pLfHG 5/
"I really want to break into Product Management"
make products.
"If only someone would tell me how I can get a startup to notice me."
Make Products.
"I guess it's impossible and I'll never break into the industry."
MAKE PRODUCTS.
Courtesy of @edbrisson's wonderful thread on breaking into comics โ https://t.co/TgNblNSCBj โ here is why the same applies to Product Management, too.
There is no better way of learning the craft of product, or proving your potential to employers, than just doing it.
You do not need anybody's permission. We don't have diplomas, nor doctorates. We can barely agree on a single standard of what a Product Manager is supposed to do.
But โ there is at least one blindingly obvious industry consensus โ a Product Manager makes Products.
And they don't need to be kept at the exact right temperature, given endless resource, or carefully protected in order to do this.
They find their own way.
make products.
"If only someone would tell me how I can get a startup to notice me."
Make Products.
"I guess it's impossible and I'll never break into the industry."
MAKE PRODUCTS.
Courtesy of @edbrisson's wonderful thread on breaking into comics โ https://t.co/TgNblNSCBj โ here is why the same applies to Product Management, too.
"I really want to break into comics"
— Ed Brisson (@edbrisson) December 4, 2018
make comics.
"If only someone would tell me how I can get an editor to notice me."
Make Comics.
"I guess it's impossible and I'll never break into the industry."
MAKE COMICS.
There is no better way of learning the craft of product, or proving your potential to employers, than just doing it.
You do not need anybody's permission. We don't have diplomas, nor doctorates. We can barely agree on a single standard of what a Product Manager is supposed to do.
But โ there is at least one blindingly obvious industry consensus โ a Product Manager makes Products.
And they don't need to be kept at the exact right temperature, given endless resource, or carefully protected in order to do this.
They find their own way.
What an amazing presentation! Loved how @ravidharamshi77 brilliantly started off with global macros & capital markets, and then gradually migrated to Indian equities, summing up his thesis for a bull market case!
@MadhusudanKela @VQIndia @sameervq
My key learnings: โฌ๏ธโฌ๏ธโฌ๏ธ
First, the BEAR case:
1. Bitcoin has surpassed all the bubbles of the last 45 years in extent that includes Gold, Nikkei, dotcom bubble.
2. Cyclically adjusted PE ratio for S&P 500 almost at 1929 (The Great Depression) peaks, at highest levels except the dotcom crisis in 2000.
3. World market cap to GDP ratio presently at 124% vs last 5 years average of 92% & last 10 years average of 85%.
US market cap to GDP nearing 200%.
4. Bitcoin (as an asset class) has moved to the 3rd place in terms of price gains in preceding 3 years before peak (900%); 1st was Tulip bubble in 17th century (rising 2200%).
@MadhusudanKela @VQIndia @sameervq
My key learnings: โฌ๏ธโฌ๏ธโฌ๏ธ
Bubble or Bull Market? Join us for a short presentation and candid one on one on 27th Jan, 4pm with Shri \u2066@MadhusudanKela\u2069. \u2066@VQIndia\u2069 \u2066@sameervq\u2069 #bubbleorbullmarket pic.twitter.com/LBvlBrz6mS
— Ravi Dharamshi (@ravidharamshi77) January 24, 2021
First, the BEAR case:
1. Bitcoin has surpassed all the bubbles of the last 45 years in extent that includes Gold, Nikkei, dotcom bubble.
2. Cyclically adjusted PE ratio for S&P 500 almost at 1929 (The Great Depression) peaks, at highest levels except the dotcom crisis in 2000.
3. World market cap to GDP ratio presently at 124% vs last 5 years average of 92% & last 10 years average of 85%.
US market cap to GDP nearing 200%.
4. Bitcoin (as an asset class) has moved to the 3rd place in terms of price gains in preceding 3 years before peak (900%); 1st was Tulip bubble in 17th century (rising 2200%).
You May Also Like
Joshua Hawley, Missouri's Junior Senator, is an autocrat in waiting.
His arrogance and ambition prohibit any allegiance to morality or character.
Thus far, his plan to seize the presidency has fallen into place.
An explanation in photographs.
๐งต
Joshua grew up in the next town over from mine, in Lexington, Missouri. A a teenager he wrote a column for the local paper, where he perfected his political condescension.
2/
By the time he reached high-school, however, he attended an elite private high-school 60 miles away in Kansas City.
This is a piece of his history he works to erase as he builds up his counterfeit image as a rural farm boy from a small town who grew up farming.
3/
After graduating from Rockhurst High School, he attended Stanford University where he wrote for the Stanford Review--a libertarian publication founded by Peter Thiel..
4/
(Full Link: https://t.co/zixs1HazLk)
Hawley's writing during his early 20s reveals that he wished for the curriculum at Stanford and other "liberal institutions" to change and to incorporate more conservative moral values.
This led him to create the "Freedom Forum."
5/
His arrogance and ambition prohibit any allegiance to morality or character.
Thus far, his plan to seize the presidency has fallen into place.
An explanation in photographs.
๐งต
Joshua grew up in the next town over from mine, in Lexington, Missouri. A a teenager he wrote a column for the local paper, where he perfected his political condescension.
2/
By the time he reached high-school, however, he attended an elite private high-school 60 miles away in Kansas City.
This is a piece of his history he works to erase as he builds up his counterfeit image as a rural farm boy from a small town who grew up farming.
3/
After graduating from Rockhurst High School, he attended Stanford University where he wrote for the Stanford Review--a libertarian publication founded by Peter Thiel..
4/
(Full Link: https://t.co/zixs1HazLk)
Hawley's writing during his early 20s reveals that he wished for the curriculum at Stanford and other "liberal institutions" to change and to incorporate more conservative moral values.
This led him to create the "Freedom Forum."
5/
