AGI Significance Paradox: As progress accelerates towards AGI, the number of people who realize the significance of each new breakthrough decreases.

Why is it that you can boil a frog in water without it jumping out if you gradually increase the temperature?
The problem for the frog is that it does not have the internal models to realize that there is a change in the water temperature. A cold-blooded creature like the frog has its temperature regulated by the external environment.
To recognize change, an agent must have an internal model of reality that is able to recognize this change. Unfortunately, a majority of the population do not have good models of human general intelligence.
In fact, even the simplistic dual-process model of system 1 and system 2 is not very well known.
Recent big developments was muZero, AlphaFold2, GPT-3 and Dall-E. GPT-3 did receive a lot of attention, but the other 3 likely have not. To understand muZero and AlphaFold2 requires a high level of expertise. Dall-E is actually like GPT-3 but it's more difficult to grasp.
We are going to continue to get these incremental developments for several years. But the audience that recognizes its importance will continue to decline. Then suddenly, boom... we get to AGI and most people will be in shock. Shocked because they thought there was no progress.
The quantum leaps (punctuated) in evolution is a consequence of many incremental developments that accrue. It is only when the final piece in the jigsaw puzzle is found when the revolution is expressed.
But it takes unusual expertise to know that we are accelerating towards AGI. The problem is that it is not obvious how human intelligence actually works. We simply do not know what it means to 'understand'.
Ask most AGI researchers, philosophers or psychologists as to what it means to 'understand'. They will be stumped to give you a good answer.
So if we do not know this answer, then how can we recognize that the water's temperature is gradually increasing?
The number of people who might know continues to diminish. This implies collectively that we know less and less. When AGI happens, it will come as a shock. It is as if, nobody had anticipated it.
@threadreaderapp unroll

More from Carlos E. Perez

More from Science

I want to share my thoughts, as someone who has been so alarmed by the so-called "dissident" scientists like Gupta, Heneghan, Kuldorff, Bhattacharya, & Ioannidis who consider themselves brave Galileos unfairly treated by "establishment scientists." I will try not to swear. 1/n


I want to talk about 3 things:
‼️Their fringe views are inhumane, unethical junk science that promotes harm
‼️They complain that they've been marginalized but this is simply untrue
‼️I am sick of people telling me we have to "listen to both sides." There aren't 2 sides here 2/n

These 'dissident' scientists have consistently downplayed COVID-19, urging policymakers not to take aggressive control measures. They claim it is not a serious threat. Gupta even went on TV saying people under 65 shouldn't worry about it!

RECEIPTS

They have consistently argued that policymakers should just let the virus rip, in an attempt to reach herd immunity by natural infection. Kuldorff *continues* to argue for this even now that we have many highly effective, safe vaccines.


We've never controlled a deadly, contagious pandemic before by just letting the virus spread, as this approach kills & disables too many people. In Manaus, Brazil, 66% of the city was infected & an astonishing *1 in 500* people died of COVID-19
It was great to talk about reproducible workflows for @riotscienceclub @riotscience_wlv. You can watch the recording below, but if you don't want to listen to me talk for 40 minutes, I thought I would summarise my talk in a thread:


My inspiration was making open science accessible. I wanted to outline the mistakes I've made along the way so people would feel empowered to give it a go. Increased accountability is seen as a barrier to adopting open science practices as an ECR

It also comes across as all or nothing. You are either fully open science or your research won't get anywhere. However, that can be quite intimidating, so I wanted to emphasise this incremental approach to adapting your workflow

There are two sides to why you should work towards reproducibility. The first is communal. It's going to help the field if you or someone else can reproduce your whole pipeline.


There is also the selfish element of it's just going to help you do your work. If you can't remember what your work means after a lunch break, you're not going to remember months or years down the line

You May Also Like