Learn JavaScript Recursion

The Complete Guide:

Thread 🧵

Important paper from Google on large batch optimization. They do impressively careful experiments measuring # iterations needed to achieve target validation error at various batch sizes. The main "surprise" is the lack of surprises. [thread]

https://t.co/7QIx5CFdfJ

The paper is a good example of lots of elements of good experimental design. They validate their metric by showing lots of variants give consistent results. They tune hyperparamters separately for each condition, check that optimum isn't at the endpoints, and measure sensitivity.

They have separate experiments where the hold fixed # iterations and # epochs, which (as they explain) measure very different things. They avoid confounds, such as batch norm's artificial dependence between batch size and regularization strength.

When the experiments are done carefully enough, the results are remarkably consistent between different datasets and architectures. Qualitatively, MNIST behaves just like ImageNet.

Importantly, they don't find any evidence for a "sharp/flat optima" effect whereby better optimization leads to worse final results. They have a good discussion of experimental artifacts/confounds in past papers where such effects were reported.

Sri Devi Karumariamman mandir, Chennai, Tamilnadu

Sri Devi Karumariamman mandir is in Thiruverkadu, a suburb of Chennai. It is said that God Muruga came here and obtained the 'Vel' (spear) from His mother, Goddess Karumari (an avatar of Parvathi) 1/n

to fight the demon Suran and hence this place was called as 'Vel kadu' and became as Verkadu. As per legend Devi Karumariamman once disguised as a soothsayer visited Bhagwan Surya, the Sun God, to predict His future. 2/n

Not realising that the soothsayer is none other than Sakthi Matha, Surya ignored Her. Angered by his indifference, Devi Karumariamman immediately retreated. Bhagwan Surya lost His glory and the earth plunged into darkness. 3/n

Bhagwan Surya then realised His folly and pleaded forgiveness from Devi Karumariamman. In order to pacify mother Karumariamman, Suryadev said that His rays would fall on Her directly twice in a year and touch her feet. (in the month of Panguni (March- April) & during (Sep - Oct)

Mother Karumari Amman is worshipped here in two forms.A swayambu (self manifested) with only head visible above the ground and another one in a graceful sitting posture with all Parasakthi features,with four arms holding skull, sword, trishul, udukkai (damru). 5/n