BC MACHINE LEARNING
Saved by @zmbnski

Important paper from Google on large batch optimization. They do impressively careful experiments measuring # iterations needed to achieve target validation error at various batch sizes. The main "surprise" is the lack of surprises. [thread]

https://t.co/7QIx5CFdfJ

 The paper is a good example of lots of elements of good experimental design. They validate their metric by showing lots of variants give consistent results. They tune hyperparamters separately for each condition, check that optimum isn't at the endpoints, and measure sensitivity.
They have separate experiments where the hold fixed # iterations and # epochs, which (as they explain) measure very different things. They avoid confounds, such as batch norm's artificial dependence between batch size and regularization strength.
When the experiments are done carefully enough, the results are remarkably consistent between different datasets and architectures. Qualitatively, MNIST behaves just like ImageNet.
Importantly, they don't find any evidence for a "sharp/flat optima" effect whereby better optimization leads to worse final results. They have a good discussion of experimental artifacts/confounds in past papers where such effects were reported.
The time-to-target-validation is explained purely by optimization considerations. There's a regime where variance dominates, and you get linear speedups w/ batch size. Then there's a regime where curvature dominates and larger batches don't help. As theory would predict.
Incidentally, this paper must have been absurdly expensive, even by Google's standards. Doing careful empirical work on optimizers requires many, many runs of the algorithm. (I think surprising phenomena on ImageNet are often due to the difficulty of running proper experiments.)

More from Machine learning

Prashant \U0001f4da
Prashant 📚
@capeandcode
Ever heard of Autoencoders?

The first time I saw a Neural Network with more output neurons than in the hidden layers, I couldn't figure how it would work?!

#DeepLearning #MachineLearning
Here's a little something about them: 🧵👇


Autoencoders are unsupervised neural networks whose architecture you can picture as two funnels connect from the narrow ends.

These networks are primary focus for compression tasks of data in Machine Learning.

We feed them the data so that they can learn the most important features, a smaller representation while keep the integrity of the data.

Later when someone needs, can just take that small representation and recreate the original, just like a zip file.📥

Being unsupervised, they require no labels.
Our inputs and outputs are same and a simple euclidean distance can be used as a loss function for measuring the reconstruction.

Of course, we wouldn't expect a perfect reconstruction.

We can think of an autoencoder having two components, encoder and decoder, represented by the below equations:

We are just trying to minimize the L here. All the backpropagation rules still hold.
MACHINE LEARNING
Jan Giacomelli
Jan Giacomelli
@jangiacomelli
🤔 Python decorators

What are they? How do you use them?

🧵 Let's find out 👇

1️⃣ Decorator is a function or a class that wraps another function or class modifying its behavior.

So how does that work?

The first thing to know is that everything in Python is an object - functions too


2️⃣ That means they can be passed to another function as an argument or returned from a function

Functions that take other functions as an argument are called higher-order functions


3️⃣ In Python, you can define a function inside other function - such functions are called inner functions


4️⃣ To create a decorator you just need to apply all of that together

log_enter_leave is a decorator.

my_function is the function.

To alter my_function's behavior we reassign it applying log_enter_leave decorator.
MACHINE LEARNING
Advertisement
Fionna O'Leary, \U0001f56f\U0001f1ea\U0001f1fa
Fionna O'Leary, 🕯🇪🇺...
@fascinatorfun
Thanks for this incredibly helpful analysis @dgurdasani1

Two questions. 1/ Does this summarise the AZ published data :
The plan is to extend the time interval for all age groups despite it being largely untested on the over 55yrs, although the full data is not yet published


Do we have the actual numbers of over 55yr olds given a 2nd dose at c12 weeks and the accompanying efficacy data?

Not to mention the efficacy data of the full first dose over that same period?

I’d quite like to know whether I am to be a guinea pig & the ongoing risks to manage

You attached photos of excerpts from a paper. Could you attach the link?

Re Pfizer. As I understand it the most efficacious interval for dosing was investigated at the start of the trial.


Here’s the link to the

I’ve got to say that this way of making and announcing decisions is not inspiring confidence in me and I am very pro vaccination as a matter of principle, not least because my brother caught polio before vaccinations available.
MACHINE LEARNING
Santiago
Santiago
@svpino
10 machine learning YouTube videos.

On libraries, algorithms, and tools.

(If you want to start with machine learning, having a comprehensive set of hands-on tutorials you can always refer to is fundamental.)

🧵👇

1⃣ Notebooks are a fantastic way to code, experiment, and communicate your results.

Take a look at @CoreyMSchafer's fantastic 30-minute tutorial on Jupyter Notebooks.

https://t.co/HqE9yt8TkB


2⃣ The Pandas library is the gold-standard to manipulate structured data.

Check out @joejamesusa's "Pandas Tutorial. Intro to DataFrames."

https://t.co/aOLh0dcGF5


3⃣ Data visualization is key for anyone practicing machine learning.

Check out @blondiebytes's "Learn Matplotlib in 6 minutes" tutorial.

https://t.co/QxjsODI1HB


4⃣ Another trendy data visualization library is Seaborn.

@NewThinkTank put together "Seaborn Tutorial 2020," which I highly recommend.

https://t.co/eAU5NBucbm
MACHINE LEARNING
Pratham Prasoon
Pratham Prasoon
@PrasoonPratham
There are several skills needed for learning machine learning that no one talks about.

Here are some of them.
(from what I've learned over the past two years)
🧵👇

This thread aims to introduce to you some of the skills often ignored for learning machine learning but are actually very important.

These skills will enable you to learn concepts quickly and more efficiently in this field.

(2 / 9)

1⃣ Reading

Probably one of the most underrated skills on this list.

In machine learning, you HAVE to read a lot of articles, papers, documentation, and whatnot.

It is mostly due to the theoretical nature of this field that reading is an important skill to have.

(3 / 9)

2⃣ Strong fundamentals in programming

Machine learning is just a lot of programming mixed with math and data.

Having clear programming fundamentals is crucial for this field.

I highly suggest you learn about these if you are using Python for machine learning.👇

(4 / 9)

- Object oriented programming in Python :Classes, Objects, Methods
- Lists & List functions
- Dunder Methods
- List comprehension
- List slicing
- String formatting
- List, Dictionaries & Tuples
- *args,**kwargs

(5 / 9)
MACHINE LEARNING

You May Also Like

Patrick McKenzie
Patrick McKenzie
@patio11
So the cryptocurrency industry has basically two products, one which is relatively benign and doesn't have product market fit, and one which is malignant and does. The industry has a weird superposition of understanding this fact and (strategically?) not understanding it.


The benign product is sovereign programmable money, which is historically a niche interest of folks with a relatively clustered set of beliefs about the state, the literary merit of Snow Crash, and the utility of gold to the modern economy.

This product has narrow appeal and, accordingly, is worth about as much as everything else on a 486 sitting in someone's basement is worth.

The other product is investment scams, which have approximately the best product market fit of anything produced by humans. In no age, in no country, in no city, at no level of sophistication do people consistently say "Actually I would prefer not to get money for nothing."

This product needs the exchanges like they need oxygen, because the value of it is directly tied to having payment rails to move real currency into the ecosystem and some jurisdictional and regulatory legerdemain to stay one step ahead of the banhammer.
CRYPTO
Aditya Todmal
Aditya Todmal
@AdityaTodmal
The absolute best 15 scanners which experts are using.

Got these scanners from the following accounts:

1. @Pathik_Trader
2. @sanjufunda
3. @sanstocktrader
4. @SouravSenguptaI
5. @Rishikesh_ADX

Share for the benefit of everyone.

Listing all scanners from @Pathik_Trader Sir first.

1. Open

2. 80-20 Reversal Setup


3. Fake

4. Trapping Inside Bar (Positional Setup)
TECH , STOCKLEARNINGS , STOCKS
Advertisement
\U0001d40d\U0001d41e\U0001d421\U0001d41a \U0001f1ee\U0001f1f3
𝐍𝐞𝐡𝐚 🇮🇳...
@Neh_hope_faith
THREAD: What is Golden Crescent and Golden Triangle and how it impacts India?

Golden Crescent region of South Asia - comprises of Afghanistan, Iran, and Pakistan - which is a principal global site for opium production and distribution.


Golden Triangle is the area where the borders of Thailand, Laos and Myanmar meet at the confluence of the Ruak and Mekong rivers. The Golden Triangle is called so because of the triangular shape formed by the locations of New Delhi, Agra and Rajasthan on a map.


India is wedged bw world’s 2 largest areas of illicit opium production, Golden Crescent & Golden Triangle, that has made Indian borders vulnerable to drug trafficking. It flows from 2 fronts, NW & NE, that possesses dual concern - violation of border & threat to national security

- Golden Crescent is world’s leading opium and cannabis producing region since 1983.

- Its proximity to Indo-Pak border has worsened the trafficking of hashish and heroin into Indian soil

- Trafficking happens through the border states of J&K, Punjab, Rajasthan and Gujarat.


- Golden Triangle is leading opium producing region of the South-east Asia, also an ancient narcotics supply route to North America & Europe.

- Myanmar-Northeast nexus is recognized by NCB as the root of heroin & chemical drugs

- Myanmar produces 80% of heroin produced in world
ALL
Nyck de Vries
Nyck de Vries
@nyckdevries
Lately there have been various rumours in the media about myself and Jumbo in relation to Williams. To clarify things, Jumbo is not and will not invest any money in my career. 1/2

Of course we’re collectively looking at career opportunities, however this is certainly not our philosophy and strategy. I hope to announce my future plans soon, be patient 😉 2/2
SPORT
Priyanka (Astrology Guidance)
Priyanka (Astrology Gu...
@AstroAmigo
1

From today, we will memorize the names of 27 Nakshatras in Vedic Jyotish to never forget in life.

I will write 4 names. Repeat them in SAME sequence twice in morning, noon, evening. Each day, revise new names + recall all previously learnt names.

Pls RT if you are in.

2

Today's Nakshatras are:-

1. Ashwini - अश्विनी

2. Bharani - भरणी

3. Krittika - कृत्तिका

4. Rohini - रोहिणी

Ashwini - अश्विनी is the FIRST Nakshatra.

Repeat these names TWICE now, tomorrow morning, noon and evening. Like this tweet if you have revised 8 times as told.

3

Today's Nakshatras are:-

5. Mrigashira - मृगशिरा

6. Ardra - आर्द्रा

7. Punarvasu - पुनर्वसु

8. Pushya - पुष्य

First recall previously learnt Nakshatras twice. Then recite these TWICE now, tomorrow morning, noon & evening in SAME order. Like this tweet only after doing so.

4

Today's Nakshatras are:-

9. Ashlesha - अश्लेषा

10. Magha - मघा

11. Purvaphalguni - पूर्वाफाल्गुनी

12. Uttaraphalguni - उत्तराफाल्गुनी

Purva means that comes before (P se Purva, P se pehele), and Uttara comes later.

Read next tweet too.

5

Purva, Uttara prefixes come in other Nakshatras too. Purva= pehele wala. Remember.

First recall previously learnt 8 Nakshatras twice. Then recite those in Tweet #4 TWICE now, tomorrow morning, noon & evening in SAME order. Like this tweet if you have read Tweets #4 & 5, both.
VEDICNAKSHATRA