A short thread on what I think is particularly useful intuition for the application of computational statistics.

Deterministic methods, like variational Bayes, utilize _rigid_ approximations. Ultimately these methods try to find the best way to wedge an approximating solid into the desired solid. If the two shapes are close enough then you can get a good fit.

But this rigidity also means that the approximating solid can't always contort to fit into the desired solid. If the two shapes don't match up well enough then we will end up in lots of awkward configurations no matter how hard we push.

In particular it's hard to quantify how good the fit of the approximate solid into the desired solid might be without knowing the shape of the desired solid already, which isn't possible in practice. This is one reason why these methods have such limited empirical diagnostics.

Stochastic, methods, on the other hand, utilize more fluid approximations that are able to take the shape of their container given enough time. In this respect we can think of them like a liquid or gas filling the desired shape.