Very much likely Vipin bhai

Important paper from Google on large batch optimization. They do impressively careful experiments measuring # iterations needed to achieve target validation error at various batch sizes. The main "surprise" is the lack of surprises. [thread]

https://t.co/7QIx5CFdfJ

The paper is a good example of lots of elements of good experimental design. They validate their metric by showing lots of variants give consistent results. They tune hyperparamters separately for each condition, check that optimum isn't at the endpoints, and measure sensitivity.

They have separate experiments where the hold fixed # iterations and # epochs, which (as they explain) measure very different things. They avoid confounds, such as batch norm's artificial dependence between batch size and regularization strength.

When the experiments are done carefully enough, the results are remarkably consistent between different datasets and architectures. Qualitatively, MNIST behaves just like ImageNet.

Importantly, they don't find any evidence for a "sharp/flat optima" effect whereby better optimization leads to worse final results. They have a good discussion of experimental artifacts/confounds in past papers where such effects were reported.

Life is crazy. Today is a good day to stop and reflect

10 simple questions that CHANGED my life

(startup life, personal life)

Here they are:

Q: Can you really control that?

Smart people know what they can and cannot control

If you're living in the past, you're nostalgic
If you're living in the future, you're anxious
If you're living in the present, you're at peace

Q: Did you start today with a victory?

Starting every day with a victory sets you up to win

I don't care what it is:

- Make your bed
- Make a wonderful coffee
- Hug your wife or children

No such thing as a small win. It's a win!

You'll win more, if you start winning early

Q: What are you optimizing for?

#MFSA
1. Money
2. Freedom
3. Stability
4. Ability

The world rewards those who are focused

Perfectly prioritize

Are you being optimistic?

Guaranteed: more luck when you're optimistic

[THREAD: HISTORY OF THE HINDU TEMPLE]
1/102
Four hours south of Aswan in Lower Nubia, right by the Nile, is a region dotted with temple complexes built more than 3,000 years ago. Some like the temples of Amada, Derr, and Beit el-Wali go as far back as 1500 BC.

2/102
A little further south, not far from the border with Sudan, is another complex dedicated to Ramses II, Hathor, and Nefertari in the village of Ipsambul, better known locally as Abu Sunbul and internationally as Abu Simbel. These go back to the 13th century BC.

3/102
Djeser-Djeseru, also known as the mortuary temple of Hatshepsut in Upper Egypt also dates to 1400 BC. Over 500 years older is the nearby Mentuhotep II, also a mortuary temple. Both were dedicated to the cult of Amun Ra, besides their respective pharaohs.

4/102
Far older is the Ġgantija complex of Malta. Going as far back as 3600 BC, this one comprises of two temples entirely made of stone and enclosed in a stone wall. From the figures therein, it's understood to be a site of some kind of a fertility cult worship.

5/102
Dating from the same period are the limestone megaliths of Ħaġar Qim and Mnajdra, both said to have once hosted ritualistic sacrifices as part of the same fertility cult worship as found in the Ġgantija complex. Malta has some of the oldest temple complexes in Europe.