1/ Thoughts on the Myth of the "First Mover"

This thread by @danrose stirred something I've been thinking about for a while - the myth of first mover advantage

To this day, most people assume Amazon Web Services was the first cloud computing service. This isn't quite true

Important paper from Google on large batch optimization. They do impressively careful experiments measuring # iterations needed to achieve target validation error at various batch sizes. The main "surprise" is the lack of surprises. [thread]

https://t.co/7QIx5CFdfJ

The paper is a good example of lots of elements of good experimental design. They validate their metric by showing lots of variants give consistent results. They tune hyperparamters separately for each condition, check that optimum isn't at the endpoints, and measure sensitivity.

They have separate experiments where the hold fixed # iterations and # epochs, which (as they explain) measure very different things. They avoid confounds, such as batch norm's artificial dependence between batch size and regularization strength.

When the experiments are done carefully enough, the results are remarkably consistent between different datasets and architectures. Qualitatively, MNIST behaves just like ImageNet.

Importantly, they don't find any evidence for a "sharp/flat optima" effect whereby better optimization leads to worse final results. They have a good discussion of experimental artifacts/confounds in past papers where such effects were reported.

#Thread
Lessor known facts of Sudarshan Chakra or Neela Chakra of Puri Jagannath Temple.

The metal wheel at the top of Bhagwan Jagannath temple is also known as Neela Chakra (Blue Wheel). The wheel is made of eight metals comprising of.. 1/n

iron, copper, zinc, mercury, lead, brass, silver & gold. It's circumferance is about 36 feet & it is so designed that there is a wheel within the wheel. The circumferance of the inner wheel is about 26 feet. Eight hubs join together the inner and the outer wheels. 2/n

There are decorative designs on the outer wheel. The thickness of the Neela Chakra is 2 inches.The wheel is said to be Bhagwan Vishnu's most powerful weapon, Sudarshana Chakra. Inside Jagannath temple, Lord Vishnu's wheel is also worshipped in the name of 'Sudarshana'. 3/n

But inside the temple Sudarshana is not in the shape of a wheel, but in the shape of a small wood pillar placed to the left of image of bhagwan Jagannath. It is believed that the same Sudarshana is also at the top of the temple and is known by the name of Neela Chakra. 4/n

There is a specific category of people among the temple sevaks to serve the Neela Chakra and they are known as Garuda Sevaka or Chunara Nejoga. The pilgrims hold these sevaks in high esteem because everyday at Sunset. 5/n

A thread of images from a Japanese illustrated history of America from 1861.

Here is George Washington (with bow and arrow) pictured alongside the Goddess of America. 1/

Here is Christopher Columbus (seated at center) reporting his discovery of America to Queen Isabella of Spain.

So far, kinda normal, but wait for it.... 2/

Now it's the American Revolution. Here is George Washington defending his wife "Carol" from a British official named "Asura" (same characters as the Buddhist deity). 3/

And here is Washington's "second-in-command" John Adams battling an enormous snake. 4/

Here is Washington and his wife "Carol" meeting an extremely youthful Benjamin Franklin, who has an impressive squat. 5/