A short thread on what I think is particularly useful intuition for the application of computational statistics.

Deterministic methods, like variational Bayes, utilize _rigid_ approximations. Ultimately these methods try to find the best way to wedge an approximating solid into the desired solid. If the two shapes are close enough then you can get a good fit.
But this rigidity also means that the approximating solid can't always contort to fit into the desired solid. If the two shapes don't match up well enough then we will end up in lots of awkward configurations no matter how hard we push.
In particular it's hard to quantify how good the fit of the approximate solid into the desired solid might be without knowing the shape of the desired solid already, which isn't possible in practice. This is one reason why these methods have such limited empirical diagnostics.
Stochastic, methods, on the other hand, utilize more fluid approximations that are able to take the shape of their container given enough time. In this respect we can think of them like a liquid or gas filling the desired shape.
In some sense the nature of the stochastic approximation determines the viscosity of the fluid, and how quickly it's able to expand into the desired shape (especially when the shape has an awkward geometry).
All stochastic methods will fully expand into the desired shape _eventually_, but the rate of expansion may be too slow or erratic to be particularly useful in practice. That said exactly _how_ the fluid expands helps us understand how well the method is working.
These analogies are particularly useful when trying to compare methods. Fitting deterministic methods is often fast no matter how good of an approximation they provide; there's only so many ways to wedge incongruent shapes together.
On the other hand stochastic methods can quickly expand into simple shapes but they tend to slow down when encountering more complex shapes, for example in the case of statical inference when trying to fit degenerate posterior density functions.
All of this is to say that most analyses that argue "we had to use this deterministic approximation because Markov chain Monte Carlo was too slow" are bullshit. In almost all of these cases the disparity in speed is due to nasty posterior degeneracies and complex uncertainties.
In these cases the speed boost from using a deterministic method arises only by ignoring most of that uncertainty and providing a specious picture of the actual inferences.
Could that approximation actually be equivalent to accurate inferences for some implicit, more reasonably regularized model? Sure, but if you can't explicitly quantify that model how can you tell whether or not the implicit regularization actually is reasonable?
On the other hand improving the initial model with explicit regularization not only clearly communicates the modeling changes but also speeds up the stochastic approximations, too! Yes it's more work but work you should have been doing anyways.
When we say that Markov chain Monte Carlo _explores_ we really mean it. It will do it's damnedest to explore as much as it can, but it can only go so fast when exploring difficult terrain. That's often not a problem of the method so much as the terrain to which it's been sent!
Use that struggle to you advantage. Learn about the degeneracies in your inferences and motivate principled resolutions such as more careful prior models, auxiliary measurements, and the like. See for example https://t.co/vBLqlH7QdA.
And please, please stop assuming that your fits are slow only because Markov chain Monte Carlo is terrible and some magical new algorithm will come and make the scary chains go away. The pathologies have been calling from within your models the entire time!

More from Science

An interesting thing about carp is that they can go into anoxic hibernation and switch to an anaerobic metabolism based on converting glycogen to ethanol.

The waste ethanol is diffused out the gills

https://t.co/V3D1umHf04

Carp can switch over to an anaerobic metabolism and quietly exhale booze until the situation gets better.

They basically evolved the same metabolic pathway as yeast, independently.

In theory, if you spent a few thousand years breeding carp for it, you could use them to make booze.

They'd be enormous, almost entirely glycogen deposits with a fish added as an afterthought.

The really interesting thing about anaerobic carp, is that they can go 4-5 months without oxygen by relying on liver glycogen.

You, a human, have only about 100 grams of glycogen in your liver, about 400 more grams in your skeletal muscles. Call it 500 grams total.

In humans, glycogen is also burned for energy. This is where the marathon runner's bonk comes from: you only have about 2,000 calories worth, and running a marathon burns those 2,000 calories.
So it turns out that an organization I thought was doing good work, the False Memory Syndrome Foundation (associated with Center for Inquiry, James Randi, and Martin Gardner) was actually caping for pedophiles. Uhhhh oops?


Since this, bizarrely, turned out to be one of my longest videos ever (??) here's a quick thread to sum it up for those of you like myself with short attention spans. 1/10

In the '90s the False Memory Syndrome Foundation was founded to call attention to the problem of adults suddenly "remembering" child abuse that never actually happened, often under hypnosis. Skeptics like James Randi & Martin Gardner joined their board. 2/10

A new article reveals that the FMSF was founded by parents who had been credibly and PRIVATELY accused of molestation by their now-adult daughter. They publicized the accusation, destroyed the daughter's reputation, and started the foundation. 3/10

The FMSF assumed any accused pedo who joined was innocent, saying "We are a good-looking bunch of people, graying hair, well dressed, healthy, smiling; just about every person who has attended is someone you would surely find interesting and want to count as a friend" 😬 4/10

You May Also Like

महाभारत की कहानी कौन नहीं जानता।लेकिन क्या आपको पता है कि महाभारत के ज्यादातर पात्र किसी न किसी श्राप में फंसे थे।अगर ये श्राप न होते तो कदाचित महाभारत की कहानी कुछ और होती।हिन्दु पौराणिक ग्रंथों में विभिन्न श्रापों का वर्णन मिलता है व हर श्राप के पीछे कोई कहानी अवश्य होती है।


आइए आज जानते हैं महाभारत कथा में वर्णित कुछ श्रापों के बारे में।

1) राजा पाण्डु को ऋषि किन्दम का श्राप

एकबार महाराज पाण्डु शिकार खेलने वन गए।झाडियों के पीछे कुछ हिल रहा था। मृग है सोचकर राजा ने बाण चलाया जो जाकर ऋषि किन्दम और उनकी पत्नी को लगा।वे दोनो रति-क्रीड़ा में लिप्त थे।

जब राजा ने उन्हें देखा तो बहुत दुखी हुए कि ये मुझसे क्या पाप हो गया।बहुत क्षमा याचना के बाद भी किन्दम ऋषि ने पाण्डु को श्राप दे दिया कि जब भी वो किसी स्त्री को काम भावना से स्पर्श करेंगे उसी क्षण उनकी मृत्यु हो जाएगी।पश्चाताप करने, वे सिंहासन पे अन्धे राजा धृतराष्ट्र को बैठाकर...


..स्वयं अपनी रानियों कुंती व माद्री के साथ वन चले गए।पांडवों का जन्म भी कुंती को ऋषि दुर्वासा द्वारा दिए गए मंत्र से हुआ था जिसमे किसी भी देव का स्मरण कर उस देव से कुंती,पुत्र प्राप्त कर सकती थी।एक बार माद्री पे मोहित हो जब पांडु ने उसे स्पर्श किया,उसी क्षण पांडु की मृत्यु होगयी।


2) उर्वशी का अर्जुन को श्राप

महाभारत युद्ध से पहले जब अर्जुन दिव्यास्त्र प्राप्त करने स्वर्ग गए तो वहां उर्वशी नाम की अप्सरा उन पर मोहित हो गयी। अर्जुन ने जब उन्हें अपनी माता के समान बताया तो यह सुनकर उर्वशी क्रोधित हो गयी और अर्जुन को श्राप दे डाला कि तुम नपुंसक की भांति...