Predicting Inflation is Hard

"The main argument at the moment about inflation is whether it’s finally here to stay from all of the government spending or simply transitory from a quirky pandemic economy."

https://t.co/0CaOOQUYvU

by @awealthofcs

Important paper from Google on large batch optimization. They do impressively careful experiments measuring # iterations needed to achieve target validation error at various batch sizes. The main "surprise" is the lack of surprises. [thread]

https://t.co/7QIx5CFdfJ

The paper is a good example of lots of elements of good experimental design. They validate their metric by showing lots of variants give consistent results. They tune hyperparamters separately for each condition, check that optimum isn't at the endpoints, and measure sensitivity.

They have separate experiments where the hold fixed # iterations and # epochs, which (as they explain) measure very different things. They avoid confounds, such as batch norm's artificial dependence between batch size and regularization strength.

When the experiments are done carefully enough, the results are remarkably consistent between different datasets and architectures. Qualitatively, MNIST behaves just like ImageNet.

Importantly, they don't find any evidence for a "sharp/flat optima" effect whereby better optimization leads to worse final results. They have a good discussion of experimental artifacts/confounds in past papers where such effects were reported.

1. I was walking in the woods yesterday and I had an epiphany. It wasn’t one I wanted to have. I was thinking about today’s class, where we are reading Kimmerer’s ‘Gathering Moss’. I’ve taught Braiding Sweetgrass in multiple classes every year since 2016. It’s eminently teachable

2. Both books are beautifully written. Students uniformly enjoy them as assigned reading. They do a great job communicating to non-Indigenous folks the contours of basic Indigenous relationships to more-than-human beings. They’re wildly popular with white readers.

3. I’ve found Braiding Sweetgrass to be a great entry point for non-Indigenous folks to Indigenous scholarship. But this is where yesterday’s epiphany comes in. Neither Braiding Sweetgrass or Gathering Moss tends to the deep deep lineages of Indigenous scholarship in Canada or US

4. As the popularity of ‘Braiding Sweetgrass’ has soared, I’ve seen more and more white folks wax on about ‘braiding knowledge’. But they often don’t know any of the lineages of these concepts beyond what is presented in this one book. They confidently run with this.

5. Kimmerer does a commendable job of weaving in her own experiences, her family stories+nods to some very important knowledge keepers who deserve to be celebrated for their work. But Braiding Sweetgrass does not tend to the citational politics Sara Ahmed & others teach us about

THE IMPORTANCE OF DEVI BEEJ / BIJA MANTRA.

Once Suutji in his discourse with King Janmajeya repeated the words of Vyasji regarding the pronounciation of the word ऐ. Vyasji had heard the importance of this word in a meeting of all important saints at Naimisharanya.

There this story was discussed.

In that gathering of such great sages, Rishi Jamdagni asked a question to all the eminent luminaries. He wanted to know which diety is the most revered and gets satisfied easily with our prayers. So Lomash rishi replied that Devi

Bhagwati was one such diety. She will be satisfied and is ready to grant you a boon even if you just say the first letter of her beej mantra. In this context he narrated the story of Satyavrat. The story is as follows.

There lived a Brahmin named Devdutt in Kosal.

He was childless. He arranged for an elaborate yagya for a intelligent son to be born. He invited all the eminent rishi's to conduct this Yagya. Rishi Suhotra, Pael, Brahaspati Yagyavalka were assigned different tasks. Rishi Gobhil was asked to sing the Saamved.

He sang the verses very well, but he had to take respite in order to breathe between two verses. This angered devdutt. He spoke his thoughts to Gobhil rishi.

The rishi got very angry as devdutt did not realise the practicalities of singing. He cursed devdutt that