The reality is you can find “evidence” for almost any narrative. Limit the sample size, cherry-pick studies, etc. Systematic reviews, meta analyses, and randomized controlled trials are all susceptible to selective interpretation/narrative fallacy.
@nntaleb 1/10
Science is assumed to be “evidence-based” but that term alone doesn’t mean much. What constitutes good evidence? How is evidence being used? Is it supporting or refuting a hypothesis? Was the hypothesis and experimental design predetermined or found ex post facto?
The reality is you can find “evidence” for almost any narrative. Limit the sample size, cherry-pick studies, etc. Systematic reviews, meta analyses, and randomized controlled trials are all susceptible to selective interpretation/narrative fallacy.
At the heart of the problem is the over-reliance on simplistic statistical techniques that do little more than quantify 2 things moving together.
Take Pearson’s correlation, based on covariance. Variation can increase simultaneously across 2 variables for countless reasons, most of which are spurious. Yet this simple notion of “causality” undergirds much of scientific literature.
Information-theoretic (entropy based) approaches on the other hand can assess *general* measures of dependence. Rather than some specialized (linear) view based on concurrent variation, entropy encompasses the amount of information contained in and between variables.
If you were genuinely interested in giving the term “evidence” an authentic and reliable meaning then the methods used to underpin an assertion would be rigorous.
We wouldn’t look to conveniently simplistic methods to denote something as evidential, rather we would look for a measure capable of assessing the expected amount of information held in a random variable; there is nothing more fundamental than information.
Consider Mutual Information (MI), which quantifies the amount of information obtained about one random variable through observing another random variable. This observing of the relationship between variables is what measurement and evidence is all about.
MI determines how different joint entropy is from marginal entropies. If there is a genuine dependence between variables we would expect information gathered from all variables at once (joint) to be less than the sum of information from independent variables (marginals).
If “evidence-based” science was genuinely invested in authentic measurement it would leverage *general* measures of dependence; that demands an approach rooted in information-theory. Without entropy you’re just picking data, choosing a narrative, and calling it “evidence.”
More from Science
Hard agree. And if this is useful, let me share something that often gets omitted (not by @kakape).
Variants always emerge, & are not good or bad, but expected. The challenge is figuring out which variants are bad, and that can't be done with sequence alone.
You can't just look at a sequence and say, "Aha! A mutation in spike. This must be more transmissible or can evade antibody neutralization." Sure, we can use computational models to try and predict the functional consequence of a given mutation, but models are often wrong.
The virus acquires mutations randomly every time it replicates. Many mutations don't change the virus at all. Others may change it in a way that have no consequences for human transmission or disease. But you can't tell just looking at sequence alone.
In order to determine the functional impact of a mutation, you need to actually do experiments. You can look at some effects in cell culture, but to address questions relating to transmission or disease, you have to use animal models.
The reason people were concerned initially about B.1.1.7 is because of epidemiological evidence showing that it rapidly became dominant in one area. More rapidly that could be explained unless it had some kind of advantage that allowed it to outcompete other circulating variants.
Variants always emerge, & are not good or bad, but expected. The challenge is figuring out which variants are bad, and that can't be done with sequence alone.
Feels like the next thing we're going to need is a ranking system for how concerning "variants of concern\u201d actually are.
— Kai Kupferschmidt (@kakape) January 15, 2021
A lot of constellations of mutations are concerning, but people are lumping together variants with vastly different levels of evidence that we need to worry.
You can't just look at a sequence and say, "Aha! A mutation in spike. This must be more transmissible or can evade antibody neutralization." Sure, we can use computational models to try and predict the functional consequence of a given mutation, but models are often wrong.
The virus acquires mutations randomly every time it replicates. Many mutations don't change the virus at all. Others may change it in a way that have no consequences for human transmission or disease. But you can't tell just looking at sequence alone.
In order to determine the functional impact of a mutation, you need to actually do experiments. You can look at some effects in cell culture, but to address questions relating to transmission or disease, you have to use animal models.
The reason people were concerned initially about B.1.1.7 is because of epidemiological evidence showing that it rapidly became dominant in one area. More rapidly that could be explained unless it had some kind of advantage that allowed it to outcompete other circulating variants.
You May Also Like
“We don’t negotiate salaries” is a negotiation tactic.
Always. No, your company is not an exception.
A tactic I don’t appreciate at all because of how unfairly it penalizes low-leverage, junior employees, and those loyal enough not to question it, but that’s negotiation for you after all. Weaponized information asymmetry.
Listen to Aditya
And by the way, you should never be worried that an offer would be withdrawn if you politely negotiate.
I have seen this happen *extremely* rarely, mostly to women, and anyway is a giant red flag. It suggests you probably didn’t want to work there.
You wish there was no negotiating so it would all be more fair? I feel you, but it’s not happening.
Instead, negotiate hard, use your privilege, and then go and share numbers with your underrepresented and underpaid colleagues. […]
Always. No, your company is not an exception.
A tactic I don’t appreciate at all because of how unfairly it penalizes low-leverage, junior employees, and those loyal enough not to question it, but that’s negotiation for you after all. Weaponized information asymmetry.
Listen to Aditya
"we don't negotiate salaries" really means "we'd prefer to negotiate massive signing bonuses and equity grants, but we'll negotiate salary if you REALLY insist" https://t.co/80k7nWAMoK
— Aditya Mukerjee, the Otterrific \U0001f3f3\ufe0f\u200d\U0001f308 (@chimeracoder) December 4, 2018
And by the way, you should never be worried that an offer would be withdrawn if you politely negotiate.
I have seen this happen *extremely* rarely, mostly to women, and anyway is a giant red flag. It suggests you probably didn’t want to work there.
You wish there was no negotiating so it would all be more fair? I feel you, but it’s not happening.
Instead, negotiate hard, use your privilege, and then go and share numbers with your underrepresented and underpaid colleagues. […]