BC MACHINE LEARNING

Saved by @zmbnski

Roger Grosse
@RogerGrosse 7 years, 11 months ago 3883 views

Save to PDF Share See On Twitter

Important paper from Google on large batch optimization. They do impressively careful experiments measuring # iterations needed to achieve target validation error at various batch sizes. The main "surprise" is the lack of surprises. [thread]

https://t.co/7QIx5CFdfJ

The paper is a good example of lots of elements of good experimental design. They validate their metric by showing lots of variants give consistent results. They tune hyperparamters separately for each condition, check that optimum isn't at the endpoints, and measure sensitivity.

They have separate experiments where the hold fixed # iterations and # epochs, which (as they explain) measure very different things. They avoid confounds, such as batch norm's artificial dependence between batch size and regularization strength.

When the experiments are done carefully enough, the results are remarkably consistent between different datasets and architectures. Qualitatively, MNIST behaves just like ImageNet.

Importantly, they don't find any evidence for a "sharp/flat optima" effect whereby better optimization leads to worse final results. They have a good discussion of experimental artifacts/confounds in past papers where such effects were reported.

The time-to-target-validation is explained purely by optimization considerations. There's a regime where variance dominates, and you get linear speedups w/ batch size. Then there's a regime where curvature dominates and larger batches don't help. As theory would predict.

Incidentally, this paper must have been absurdly expensive, even by Google's standards. Doing careful empirical work on optimizers requires many, many runs of the algorithm. (I think surprising phenomena on ImageNet are often due to the difficulty of running proper experiments.)

More from Machine learning

Ram Bhupatiraju
@RamBhupatiraju

Happy 2⃣0⃣2⃣1⃣ to all.🎇

For any Learning machines out there, here are a list of my fav online investing resources. Feel free to add yours.

Let's dive in.
⬇️⬇️⬇️

Investing Services

✔️ @themotleyfool - @TMFStockAdvisor & @TMFRuleBreakers services

✔️ @7investing

✔️ @investing_city
https://t.co/9aUK1Tclw4

✔️ @MorningstarInc Premium

✔️ @SeekingAlpha Marketplaces (Check your area of interest, Free trials, Quality, track record...)

General Finance/Investing

✔️ @morganhousel
https://t.co/f1joTRaG55

✔️ @dollarsanddata
https://t.co/Mj1owkzRc8

✔️ @awealthofcs
https://t.co/y81KHfh8cn

✔️ @iancassel
https://t.co/KEMTBHa8Qk

✔️ @InvestorAmnesia
https://t.co/zFL3H2dk6s

✔️

Tech focused

✔️ @stratechery
https://t.co/VsNwRStY9C

✔️ @bgurley
https://t.co/NKXGtaB6HQ

✔️ @CBinsights
https://t.co/H77hNp2X5R

✔️ @benedictevans
https://t.co/nyOlasCY1o

✔️

Tech Deep dives

✔️ @StackInvesting
https://t.co/WQ1yBYzT2m

✔️ @hhhypergrowth
https://t.co/kcLKITRLz1

✔️ @Beth_Kindig
https://t.co/CjhLRdP7Rh

✔️ @SeifelCapital
https://t.co/CXXG5PY0xX

✔️ @borrowed_ideas

Pratham Prasoon
@PrasoonPratham

Which libraries do you really need to get started with Machine Learning and why?

🧵👇

First and foremost, make sure that you have nailed the fundamental concepts of Python because machine learning requires a lot of programming!

(2 / 19)

If you know these topics, then you are good to go with machine learning in Python

- Object-oriented programming in Python:Classes,Objects,Methods
- Lists & List functions
- List comprehension
- List slicing
- String formatting
- List,Dictionaries & Tuples

(3 / 19)

Now, let's understand what popular python machine learning libraries do.

We will talk about👇
- TensorFlow (+ Keras)
- PyTorch
- Pandas
- Numpy
- Matplotlib
- SciKit Learn
- Seaborn

(4 / 19)

Now let's cover the libraries that you have to learn (at least the basics) for machine learning

1. Pandas

Pandas is a python library that allows you to store and read data from spreadsheets ( .csv, .xlsv files ) in structures called Dataframes.

(5 / 19)

Advertisement

Santiago
@svpino

10 machine learning YouTube videos.

On libraries, algorithms, and tools.

(If you want to start with machine learning, having a comprehensive set of hands-on tutorials you can always refer to is fundamental.)

🧵👇

1⃣ Notebooks are a fantastic way to code, experiment, and communicate your results.

Take a look at @CoreyMSchafer's fantastic 30-minute tutorial on Jupyter Notebooks.

https://t.co/HqE9yt8TkB

2⃣ The Pandas library is the gold-standard to manipulate structured data.

Check out @joejamesusa's "Pandas Tutorial. Intro to DataFrames."

https://t.co/aOLh0dcGF5

3⃣ Data visualization is key for anyone practicing machine learning.

Check out @blondiebytes's "Learn Matplotlib in 6 minutes" tutorial.

https://t.co/QxjsODI1HB

4⃣ Another trendy data visualization library is Seaborn.

@NewThinkTank put together "Seaborn Tutorial 2020," which I highly recommend.

https://t.co/eAU5NBucbm

Pratham Prasoon
@PrasoonPratham

How to learn Python for machine learning. 🐍

The beginner's guide.
(based on my real life experience of mentoring 300+ students)

🧵👇

In this thread we'll look at all the concepts in Python you need to know for machine learning along with all the free resources that you'll need.

All of this is based on my experience of successfully teaching 300+ students how to code using Python.

(2 / 19)

You can use many languages for machine learning, why Python?

Because of 2 reasons:
- Comparatively easier to learn than other languages
- Has the biggest and most mature community

This makes Python a no-brainer to learn for machine learning as a beginner.

(3 / 19)

These are the absolute basics which you must know about:

- Basic terminal commands
- Basic arithmetic (+,-,/,*)
- Accepting user input
- For & While loops
- Exception handling
- If-Else statements
- Functions, modules & Imports

(4 / 19)

Then comes the more tougher concepts which you must know about:

- Object oriented programming in Python:Classes, Objects, Methods
- PIP (Pypi)
- List slicing
- String formatting
- Dictionaries & Tuples
- Managing environments
- Dunder methods like __init__

(5 / 19)

$Unknown \U0001f464$

Unknown 👤
@console_404

10 PYTHON 🐍 libraries for machine learning.

Retweets are appreciated.
[ Thread ]

1. NumPy (Numerical Python)

- The most powerful feature of NumPy is the n-dimensional array.

- It contains basic linear algebra functions, Fourier transforms, and tools for integration with other low-level languages.

Ref: https://t.co/XY13ILXwSN

2. SciPy (Scientific Python)

- SciPy is built on NumPy.

- It is one of the most useful libraries for a variety of high-level science and engineering modules like discrete Fourier transform, Linear Algebra, Optimization, and Sparse matrices.

Ref: https://t.co/ALTFqM2VUo

3. Matplotlib

- Matplotlib is a comprehensive library for creating static, animated, and interactive visualizations in Python.

- You can also use Latex commands to add math to your plot.

- Matplotlib makes hard things possible.

Ref: https://t.co/zodOo2WzGx

4. Pandas

- Pandas is for structured data operations and manipulations.

- It is extensively used for data munging and preparation.

- Pandas were added relatively recently to Python and have been instrumental in boosting Python’s usage.

Ref: https://t.co/IFzikVHht4

You May Also Like

Mystery Grove Publishi...
@MysteryGrove

By popular demand, here is the official Mystery Grove Film Recommendations list. Watch all of these in order if you want to truly understand the world we live in today.

foone
@Foone

I got a Toshiba T1200, which has a bad screen, and a dead CMOS battery, and I thought the hard drive was broken too...

but I rebooted it and heard a KTHUNK and a really loud spin-up noise and holy shit, the hard drive works!

and it still has files on it. the newest date is January 1994.

It's a 21mb hard drive, with ~37k of bad sectors.

This fucker is never going to spin up again, so I better go find some serial cables and copy shit off now.

The backlight is fucked so I'm shining a flashlight on it.
without it, it looks like this.

Advertisement

DTBhat
@dtbhat

The thread explains my momentum scanner and how to build it in #TradePoint software.
@Definedge
Conditions :
1)EMA 8 above EMA 34 and rising
2)Candle close above previous 5 candle high
3) RSI above 60
4) Volume above previous 5 candles
5) ADX above 30
1/n

Select Double Moving Average 8 and 34
2/n

Under Patterns select Close Above Prev Bars
3/n

Qualify the candle close
4/n

RSI above 60
5/n

Meet Shah
@ms89_meet

Strong Monopolies:

IRCTC 100% Market share in Rail Network.

IEX >90% market share in power trading.

Zydus wellness >90% market share in sugar free product.

Eicher motors >85% market share in 250cc bikes category.

MCX >85% market share in commodity trading.

Coal India >80% market share in coal production in India.

ITC >75% market share in cigarettes.

Honda Siel >75% in portable power generators.

Hindustan Zinc >75% market share in primary zinc industry.

Asahi India Glass >70% market share in automotive glass.

NRB Bearings >70% market share in needle roller bearings.

Pidilite >65% market share in adhesives.

CAMS >65% market share in RTA within mutual fund industry.

Time Technoplast >65% market share in polymer based industrial packaging.

Concor >65% market share in domestic container cargo transport.

Exide >60% market share in lead batteries.

Naukri >60% market share in the Indian job market space.

Praj >60% market share in ethanol plant installing.

Indiamart Intermesh >55% market share in the online B2B Classified space.

Borosil Renewables >55% market share in Lab glass.

Vst Tillers >50% market share in
Power tillers.

Delta corp >50% in online poker games.

Vinati Organics >50% market share in IBB.

Jonathan Ware
@ReassessHistory

So when it first appeared in Normandy, the Sherman Firefly and it's meaty 17 pounder represented an ideal combo of reliable chassis, great gun and devastating firepower right?

Ummm...

Truth is a bit more complex. /1
#WW2 #SWW #History

Deliveries of Firefly to units only commenced in April 1944,
just a few weeks before Overlord & left minimal time for familiarisation.

Armoured Regiments mostly utilised Fireflies on a basis of 1 per troop of 4 tanks. /2

Finally each armoured regiment troop had access to integral, meaty, anti-tank firepower capable of dealing serious damage to any German AFV likely to be encountered. /3

The dual purpose 75mm gun was quite capable of laying down a reasonable HE shell or AP round for dealing with Panzer III or IV.

Firefly's 17 Pounder had.... problems with HE, but excelled at tank killing. So Tigers and Panthers would need to be wary. /4

Firefly's design history dated back to March 1943 when the General Staff published "Policy on Tanks" which effectively stated the majority of tanks should have a good HE weapon and...

[See image below] /5