Important paper from Google on large batch optimization. They do impressively careful experiments measuring # iterations needed to achieve target validation error at various batch sizes. The main "surprise" is the lack of surprises. [thread]

https://t.co/7QIx5CFdfJ

In this thread, I will tell you how to learn python for data science in 1 hour 👇 1/N

It's not possible 2/N

It's not possible 3/N

It's not possible 4/N

It's not possible 5/N

1/ It's probably not the first thing you think of, but when we started .NET (COM+) in the late 90s, C# didn't exist yet. We were working on it at the same time as the CLR and the framework. So, you might wonder, what language was being used to generate IL and write the BCL?

2/ The answer is a language that we called SMC that Peter Kukol wrote the compiler for. Peter is a flat out amazing engineer and wrote the core parts of the compiler in just a few days. This unblocked the framework team, allowed vetting the runtime and interpreter, etc.

3/ SMC was a trimmed down C++ variant and the compiler was written in itself (i.e. SMC). It didn't support things like destructors, multiple inheritance, virtual base classes, etc. But, overall it enabled progress that would have otherwise been stalled.

4/ It did introduce a problem for the C# language and compiler team though. Every day more and more code was written in SMC, a language that we did not intend to ship externally. Designing a new language is hard (well, at least, a general purpose one that becomes popular).

5/ So there was a constant balancing act between laying enough of the foundation of the language and the C# compiler to replace SMC and making sure that the design lived up to the goals and ideals the team was striving for. If we waited too long to adopt C#,

The "polarization is the problem" narrative to me is bad because it implies the status quo was a stable and acceptable set of trade-offs. It's the equivalent of a pollyanna "why can't we all get along" but the we in question is white supremacy and its targets.

like i understand that conflict mediation requires mutual concessions (i did my MA in conflict studies) and while it's important to meet people's deep needs across spectrums of difference, certain things are just not up for negotiation.

Also while peacebuilding work is good. It's not just "seeing each other as human" it's literally about transforming structural issues. (hence Galtung's distinction between positive and negative peace wherin negative peace is like a ceasefire and positive is a transformed society.

And I think that to whatever extent that kind of peacebuilding work is applicable, it's not between fascists and people who think fascists are bad. Fascists are not invited. They are only invited if they seize territories and can't be crushed. Then secret negotiations.

The US is always like "we don't negotiate with terrorists" but literally every president negotiates with who they consider to be terrorists. It's all just down low. But that's just because counter-insurgency failed.

20 machine learning questions that will make you think.

(Cool questions. Not the regular, introductory stuff that you find everywhere.)

🧵👇

1. Why is it important to introduce non-linearities in a neural network?

2. What are the differences between a multi-class classification problem and a multi-label classification problem?

3. Why does the use of Dropout work as a regularizer?

👇

4. Why you shouldn't use a softmax output activation function in a multi-label classification problem when using a one-hot-encoded target?

5. Does the use of Dropout in your model slow down or speed up the training process? Why?

👇

6. In a Linear or Logistic Regression problem, do all Gradient Descent algorithms lead to the same model, provided you let them run long enough?

7. Explain the difference between Batch Gradient Descent, Stochastic Gradient Descent, and Mini-batch Gradient Descent.

👇

8. What are the advantages of Convolution Neural Networks (CNN) over a fully connected network for image classification?

9. What are the advantages of Recurrent Neural Networks (RNN) over a fully connected network when working with text data?

👇