BC MACHINE LEARNING

Saved by @zmbnski

Roger Grosse
@RogerGrosse 6 years, 6 months ago 3376 views

Save to PDF Share See On Twitter

Important paper from Google on large batch optimization. They do impressively careful experiments measuring # iterations needed to achieve target validation error at various batch sizes. The main "surprise" is the lack of surprises. [thread]

https://t.co/7QIx5CFdfJ

The paper is a good example of lots of elements of good experimental design. They validate their metric by showing lots of variants give consistent results. They tune hyperparamters separately for each condition, check that optimum isn't at the endpoints, and measure sensitivity.

They have separate experiments where the hold fixed # iterations and # epochs, which (as they explain) measure very different things. They avoid confounds, such as batch norm's artificial dependence between batch size and regularization strength.

When the experiments are done carefully enough, the results are remarkably consistent between different datasets and architectures. Qualitatively, MNIST behaves just like ImageNet.

Importantly, they don't find any evidence for a "sharp/flat optima" effect whereby better optimization leads to worse final results. They have a good discussion of experimental artifacts/confounds in past papers where such effects were reported.

The time-to-target-validation is explained purely by optimization considerations. There's a regime where variance dominates, and you get linear speedups w/ batch size. Then there's a regime where curvature dominates and larger batches don't help. As theory would predict.

Incidentally, this paper must have been absurdly expensive, even by Google's standards. Doing careful empirical work on optimizers requires many, many runs of the algorithm. (I think surprising phenomena on ImageNet are often due to the difficulty of running proper experiments.)

More from Machine learning

Santiago
@svpino

11 key concepts of Machine Learning.

— Supervised Learning Edition —

🧵👇

😜

Before starting, remember that, if you follow me, one of your enemies will be immediately destroyed (and you'll get to read more of these threads, of course.)

And if you don't follow me, well, you just hurt my feelings.

😜

1. Labels

(Also referred to as "y")

The label is the piece of information that we are predicting.

For example:

- the animal that's shown in a picture
- the price of a house
- whether a message is spam or not

👇

2. Features

(Also referred to as "x")

These are the input variables to our problem. We use these features to predict the "label."

For example:

- pixels of a picture
- number of bedrooms of a house
- square footage of a house

👇

3. Samples

(This is also known as "examples.")

A sample is a particular instance of data (features or "x.") It could be "labeled" or "unlabeled."

👇

$Pratham Prasoon \U0001f680$

Pratham Prasoon 🚀...
@PrasoonPratham

Here's how I got started with my first machine learning project and you can too.
Let's take a look.

(This thread will take from zero to being a hero in machine learning, trust me)

(1 / 22)
🧵👇

Getting started with your first machine learning project might actually much easier than it seems, if I can do it, certainly anyone can.

I did not use:
- Any Math
- An expensive computer
- Complex programming concepts

(2 / 22)

Here's what I did use:

- A free GPU on Google Colab
- Python
- TensorFlow
- Numpy
- Pandas
- Kaggle
- Scikit-Learn
- Google
- StackOverFlow

(3 / 22)

This project was actually a Kaggle challenge based on the MNIST dataset which is a collection images of 70,000 hand written digits.

You can find the dataset here👇
🔗//kaggle.com/c/digit-recognizer

(4 / 22)

Before we go over the code of this project, it is highly reccomended that you complete this free course on YouTube👇

Machine Learning foundations course
🔗//youtu.be/_Z9TRANg4c0

Code👇
🔗//colab.research.google.com/github/PrasoonPratham/Kaggle/blob/main/MNIST.ipynb

(5 / 22)

Advertisement

Pratham Prasoon
@PrasoonPratham

Everything you need to know about the math for machine learning as a beginner.

🧵👇

Before diving into the math, I suggest first having solid programming skills.

For example👇

(2 / 17)

In Python, these are the concepts which you must know:

- Object oriented programming in Python : Classes, Objects, Methods
- List slicing
- String formatting
- Dictionaries & Tuples
- Basic terminal commands
- Exception handling

(3 / 17)

If you want to learn python, these courses are freecodecamp could be of help to you.

🔗Basics: youtube .com/watch?v=rfscVS0vtbw
🔗Intermediate :youtube .com/watch?v=HGOBQPFzWKo

(4 / 17)

You need to have really strong fundamentals in programming, because machine learning involves a lot of it.

It is 100% compulsory.

(5 / 17)

Pratham Prasoon
@PrasoonPratham

Here's what you'll hear when you first get into machine learning.

- Neural networks
- Loss
- Weights
- Biases
- Epochs
- Neurons
- Optimizers
...

It can get very confusing really fast!

Here are some of the terms you should know about.
(I wish I had this before)

🧵👇

These terms won't mean anything unless you know what Machine learning is all about.

> Machine learning is the process of making a program which allows a computer to learn from data.

The data could be anything, images, audio or even text.

In machine learning we use something called a neural network, this is essentially an imitation of the human brain.

> Neural Networks are a digital imitation of the neurons you see in the human brain.

In these neural networks, data flows through them and each neuron (the circle) has a numerical value which will change.

> The value of a neuron gets changes to something which is close to what we want each time the data passes through the neural network.

Think of the neurons as dials on a lock, you have to tune every dial to open the lock.

It is almost impossible for a human to tune thousands of dials like these, but a computer certainly can.

$Unknown \U0001f464$

Unknown 👤
@console_404

10 PYTHON 🐍 libraries for machine learning.

Retweets are appreciated.
[ Thread ]

1. NumPy (Numerical Python)

- The most powerful feature of NumPy is the n-dimensional array.

- It contains basic linear algebra functions, Fourier transforms, and tools for integration with other low-level languages.

Ref: https://t.co/XY13ILXwSN

2. SciPy (Scientific Python)

- SciPy is built on NumPy.

- It is one of the most useful libraries for a variety of high-level science and engineering modules like discrete Fourier transform, Linear Algebra, Optimization, and Sparse matrices.

Ref: https://t.co/ALTFqM2VUo

3. Matplotlib

- Matplotlib is a comprehensive library for creating static, animated, and interactive visualizations in Python.

- You can also use Latex commands to add math to your plot.

- Matplotlib makes hard things possible.

Ref: https://t.co/zodOo2WzGx

4. Pandas

- Pandas is for structured data operations and manipulations.

- It is extensively used for data munging and preparation.

- Pandas were added relatively recently to Python and have been instrumental in boosting Python’s usage.

Ref: https://t.co/IFzikVHht4

You May Also Like

Aditya Todmal
@AdityaTodmal

Boss is always Boss @Mitesh_Engr

Mitesh Sir's FUTURES BREAKOUT TRADES 101:

• How to pick stocks at right time?
• What to look for?
• Importance of BO post consolidation
• How to manage SL
• How to get huge profits
• Multiple Examples

In collaboration with @niki_poojary

Importance of Trading only BO or BD stocks by @Mitesh_Engr Sir: 👇

•Two questions to always ask:

~Why stock is going up/down and when?
~ Who is behind big up/down and when?

• Trade only stocks with Momentum
• Instant results
• Small SL or huge profits

Sir's take on how to Trade: 👇

• Observe fight between buyer & seller
• Position will be created within first 3 hours
• Operators will win
• Just find BO or BD
• Look for huge volume support
• Give company to operators
• ATP importance explained

How to take trade in Breakout/Breakdown stocks by @Mitesh_Engr Sir: 👇

• Check Support & Resistance
• Wait for 2-3 months consolidation
• Surge in volumes
• Trade on 3 min charts
• Small SL
• Keep trailing
• Always pyramid in profit

When and Why to Pyramid? By @Mitesh_Engr Sir

• Adding to winning positions
• Signals of continued strength
• Potentially large profits
• An Example of pyramiding in one of his trades

UN Watch
@UNWatch

The UN just voted to condemn Israel 9 times, and the rest of the world 0.

View the resolutions and voting results here:

The resolution titled "The occupied Syrian Golan," which condemns Israel for "repressive measures" against Syrian citizens in the Golan Heights, was adopted by a vote of 151 - 2 - 14.

Israel and the U.S. voted 'No' https://t.co/HoO7oz0dwr

The resolution titled "Israeli practices affecting the human rights of the Palestinian people..." was adopted by a vote of 153 - 6 - 9.

Australia, Canada, Israel, Marshall Islands, Micronesia, and the U.S. voted 'No' https://t.co/1Ntpi7Vqab

The resolution titled "Israeli settlements in the Occupied Palestinian Territory, including East Jerusalem, and the occupied Syrian Golan" was adopted by a vote of 153 – 5 – 10.

Canada, Israel, Marshall Islands, Micronesia, and the U.S. voted 'No'
https://t.co/REumYgyRuF

The resolution titled "Applicability of the Geneva Convention... to the
Occupied Palestinian Territory..." was adopted by a vote of 154 - 5 - 8.

Canada, Israel, Marshall Islands, Micronesia, and the U.S. voted 'No'
https://t.co/xDAeS9K1kW

Advertisement

$\U0001f1ee\U0001f1f3$

🇮🇳
@Aamani_03

॥ॐ॥
अस्य श्री गायत्री ध्यान श्लोक:
(gAyatri dhyAna shlOka)
• This shloka to meditate personified form of वेदमाता गायत्री was given by Bhagwaan Brahma to Sage yAgnavalkya (याज्ञवल्क्य).

• 14th shloka of गायत्री कवचम् which is taken from वशिष्ठ संहिता, goes as follows..

• मुक्ता-विद्रुम-हेम-नील धवलच्छायैर्मुखस्त्रीक्षणै:।
muktA vidruma hEma nIla dhavalachhAyaiH mukhaistrlkShaNaiH.

• युक्तामिन्दुकला-निबद्धमुकुटां तत्वार्थवर्णात्मिकाम्॥
yuktAmindukalA nibaddha makutAm tatvArtha varNAtmikam.

• गायत्रीं वरदाभयाङ्कुश कशां शुभ्रं कपालं गदाम्।
gAyatrIm vardAbhayANkusha kashAm shubhram kapAlam gadAm.

• शंखं चक्रमथारविन्दयुगलं हस्तैर्वहन्ती भजै॥
shankham chakramathArvinda yugalam hastairvahantIm bhajE.

This shloka describes the form of वेदमाता गायत्री.

• It says, "She has five faces which shine with the colours of a Pearl 'मुक्ता', Coral 'विद्रुम', Gold 'हेम्', Sapphire 'नील्', & a Diamond 'धवलम्'.

• These five faces are symbolic of the five primordial elements called पञ्चमहाभूत:' which makes up the entire existence.

• These are the elements of SPACE, FIRE, WIND, EARTH & WATER.

• All these five faces shine with three eyes 'त्रिक्षणै:'.

Hector Navarro
@Hectorisfunny

I'm going to attempt to chronologically list all of the Marvel characters Stan Lee had a hand in creating. Working with his genius collaborators, Stan and his team produced an insane amount of iconic characters in a roughy ten year period. (Thread)

With his younger brother, writer Larry Lieber, and Jack "the King" Kirby:

Groot

Tales to Astonish #13 (November, 1960)

With writer Lieber, and artist Kirby:

Fin Fang Foom

Strange Tales #89 (October, 1961)

With the King, Stan gives birth to the Marvel Universe:

The Fantastic Four

Fantastic Four #1 (November, 1961)

With Jack Kirby:

Mister Fantastic

Fantastic Four #1 (November, 1961)

Meet Shah
@ms89_meet

Strong Monopolies:

IRCTC 100% Market share in Rail Network.

IEX >90% market share in power trading.

Zydus wellness >90% market share in sugar free product.

Eicher motors >85% market share in 250cc bikes category.

MCX >85% market share in commodity trading.

Coal India >80% market share in coal production in India.

ITC >75% market share in cigarettes.

Honda Siel >75% in portable power generators.

Hindustan Zinc >75% market share in primary zinc industry.

Asahi India Glass >70% market share in automotive glass.

NRB Bearings >70% market share in needle roller bearings.

Pidilite >65% market share in adhesives.

CAMS >65% market share in RTA within mutual fund industry.

Time Technoplast >65% market share in polymer based industrial packaging.

Concor >65% market share in domestic container cargo transport.

Exide >60% market share in lead batteries.

Naukri >60% market share in the Indian job market space.

Praj >60% market share in ethanol plant installing.

Indiamart Intermesh >55% market share in the online B2B Classified space.

Borosil Renewables >55% market share in Lab glass.

Vst Tillers >50% market share in
Power tillers.

Delta corp >50% in online poker games.

Vinati Organics >50% market share in IBB.