More Photos of #PinkHatLady #BullHornLady

Important paper from Google on large batch optimization. They do impressively careful experiments measuring # iterations needed to achieve target validation error at various batch sizes. The main "surprise" is the lack of surprises. [thread]

https://t.co/7QIx5CFdfJ

The paper is a good example of lots of elements of good experimental design. They validate their metric by showing lots of variants give consistent results. They tune hyperparamters separately for each condition, check that optimum isn't at the endpoints, and measure sensitivity.

They have separate experiments where the hold fixed # iterations and # epochs, which (as they explain) measure very different things. They avoid confounds, such as batch norm's artificial dependence between batch size and regularization strength.

When the experiments are done carefully enough, the results are remarkably consistent between different datasets and architectures. Qualitatively, MNIST behaves just like ImageNet.

Importantly, they don't find any evidence for a "sharp/flat optima" effect whereby better optimization leads to worse final results. They have a good discussion of experimental artifacts/confounds in past papers where such effects were reported.

💬 I often get Q’s like these:

How did you get your first customers for WIP? How did you grow BetaList’s traffic? Etc.

Makers are looking to reverse-engineer success. I see it everywhere.

I don’t think it works that way and the answers to those questions are mostly useless. 💥

I have built dozens of different products over the last couple of years. The vast majority failed. 😭

Surely if I know the answers to these questions, but still fail over and over again, these answers aren’t that useful. 🤷‍♀️

So what’s a better question to ask? 🤔

99.9% of the questions I receive are about the products that did well. In a way that makes sense, because we quickly forget about those that didn’t succeed.

🧠 This is known as survivorship bias.

Focusing on what survived, while ignoring what made it survive in the first place.

The real question, what you really want to know, is this:

What makes @WIP, @BetaList, and to some extent @AllStartupJobs succeed where my countless other attempts failed?

What separates a failed product 👎 from a successful product 👍?

Honestly, I don’t know. I wish I did.

It’s like Steve Jobs said “I’ll know it when I see it.” 👀

Same is true when we make products. We don’t know upfront what will work. But once we see an inkling of a product that does have potential, it’s not that hard to spot.

Hey, for people interested in this stuff, here's a thread about a change we made to running our facebook account:

So, about 5 years ago, we got on facebook. Why? Well, facebook has a big audience and they like to consume on facebook. So, we started posting comics there, even though we got paid nothing and facebook made money off the free content.

Why? Because back then, once in a while, you would say "also hey, I sell books" or "hey, I'm going to be signing at this event" and your facebook followers would actually see it.

Effectively, the implicit old arrangement was "facebook lets me reach readers efficiently, and I supply facebook with free content they run ads on."

Over time, like a bad business partner, facebook basically made it impossible to reach your audience without *paying them*. Thus, you supply them free content on which they run ads, and you get nothing in exchange.

Stories of Guruvayurappan
Shri Krishna Leela

This is a beautiful story of Guruvayurappan. Once there lived in Guruvayur a poor family. Kitta, his wife & kids were struggling for their daily needs. With only a coconut tree in house, he earned a living by selling coconuts.

Kitta was from a low caste family. He was a great devotee of Guruvayurappan. Everyday before going to work he always prayed infront of a picture of Guruvayurappan in his house. His only wish was to worship at Guruvayur & have annadanam in temple. But he was not allowed there

It was Utsavam time in Guruvayur. From far behind kitta and family worshipped Guruvayurappan. Annadanam was given daily and no one was left empty stomach. Only Kitta didn't receive. That night, it was only Kitta's family who was sleeping without food. After temple nada closed

Shri Krishna, dressed himself as a Brahmin went to temple Kitchen. He packed food and went directly to kitta's house. Reaching Kitta's house, Shri Krishna told he was sent by a cook of Guruvayur temple and gave them the food he packed. Kitta and family was very happy seeing food.

When Kitta finished his food, Brahmin asked it will be better if he get a tender coconut( karikku)to drink. Kitta climbed the tree and gave all coconuts to him. After having 1 tender coconut, the Brahmin asked to bring remaining coconuts to Guruvayur temple the next day.