@nntaleb 1/10
Science is assumed to be “evidence-based” but that term alone doesn’t mean much. What constitutes good evidence? How is evidence being used? Is it supporting or refuting a hypothesis? Was the hypothesis and experimental design predetermined or found ex post facto?

@engexplain @nntaleb 2/10
The reality is you can find “evidence” for almost any narrative. Limit the sample size, cherry-pick studies, etc. Systematic reviews, meta analyses, and randomized controlled trials are all susceptible to selective interpretation/narrative fallacy.
@engexplain @nntaleb 3/10
At the heart of the problem is the over-reliance on simplistic statistical techniques that do little more than quantify 2 things moving together.
@engexplain @nntaleb 4/10
Take Pearson’s correlation, based on covariance. Variation can increase simultaneously across 2 variables for countless reasons, most of which are spurious. Yet this simple notion of “causality” undergirds much of scientific literature.
@engexplain @nntaleb 5/10
Information-theoretic (entropy based) approaches on the other hand can assess *general* measures of dependence. Rather than some specialized (linear) view based on concurrent variation, entropy encompasses the amount of information contained in and between variables.
@engexplain @nntaleb 6/10
If you were genuinely interested in giving the term “evidence” an authentic and reliable meaning then the methods used to underpin an assertion would be rigorous.
@engexplain @nntaleb 7/10
We wouldn’t look to conveniently simplistic methods to denote something as evidential, rather we would look for a measure capable of assessing the expected amount of information held in a random variable; there is nothing more fundamental than information.
@engexplain @nntaleb 8/10
Consider Mutual Information (MI), which quantifies the amount of information obtained about one random variable through observing another random variable. This observing of the relationship between variables is what measurement and evidence is all about.
@engexplain @nntaleb 9/10
MI determines how different joint entropy is from marginal entropies. If there is a genuine dependence between variables we would expect information gathered from all variables at once (joint) to be less than the sum of information from independent variables (marginals).
@engexplain @nntaleb 10/10
If “evidence-based” science was genuinely invested in authentic measurement it would leverage *general* measures of dependence; that demands an approach rooted in information-theory. Without entropy you’re just picking data, choosing a narrative, and calling it “evidence.”

More from Science

Hugh Everett's birthday! Pioneer of the Many-Worlds Interpretation of quantum mechanics. Let us celebrate by thinking about ontological extravagance. I will do so by way of analogy, because I have found that everyone loves analogies and nobody ever willfully misconstrues them.


We look at the night sky and see photons arriving to us, emitted by distant stars. Let's contrast two different theories about how stars emit photons.

One theory says, we know how stars shine, and our equations predict that they emit photons roughly uniformly in all directions. Call this the "Many-Photons Interpretation" (MPI).

But! Others object. That is *so many photons*. Most of which we don't observe, and can't observe, since they're moving away at the speed of light. It's too ontologically extravagant to posit a huge number of unobservable things!

So they suggest a "Photon Collapse Interpretation." According to this theory, the photons emitted toward us actually exist. But photons that would be emitted in directions we will never observe simply collapse into utter non-existence.

You May Also Like

The entire discussion around Facebook’s disclosures of what happened in 2016 is very frustrating. No exec stopped any investigations, but there were a lot of heated discussions about what to publish and when.


In the spring and summer of 2016, as reported by the Times, activity we traced to GRU was reported to the FBI. This was the standard model of interaction companies used for nation-state attacks against likely US targeted.

In the Spring of 2017, after a deep dive into the Fake News phenomena, the security team wanted to publish an update that covered what we had learned. At this point, we didn’t have any advertising content or the big IRA cluster, but we did know about the GRU model.

This report when through dozens of edits as different equities were represented. I did not have any meetings with Sheryl on the paper, but I can’t speak to whether she was in the loop with my higher-ups.

In the end, the difficult question of attribution was settled by us pointing to the DNI report instead of saying Russia or GRU directly. In my pre-briefs with members of Congress, I made it clear that we believed this action was GRU.