AGI Significance Paradox: As progress accelerates towards AGI, the number of people who realize the significance of each new breakthrough decreases.

Why is it that you can boil a frog in water without it jumping out if you gradually increase the temperature?
The problem for the frog is that it does not have the internal models to realize that there is a change in the water temperature. A cold-blooded creature like the frog has its temperature regulated by the external environment.
To recognize change, an agent must have an internal model of reality that is able to recognize this change. Unfortunately, a majority of the population do not have good models of human general intelligence.
In fact, even the simplistic dual-process model of system 1 and system 2 is not very well known.
Recent big developments was muZero, AlphaFold2, GPT-3 and Dall-E. GPT-3 did receive a lot of attention, but the other 3 likely have not. To understand muZero and AlphaFold2 requires a high level of expertise. Dall-E is actually like GPT-3 but it's more difficult to grasp.
We are going to continue to get these incremental developments for several years. But the audience that recognizes its importance will continue to decline. Then suddenly, boom... we get to AGI and most people will be in shock. Shocked because they thought there was no progress.
The quantum leaps (punctuated) in evolution is a consequence of many incremental developments that accrue. It is only when the final piece in the jigsaw puzzle is found when the revolution is expressed.
But it takes unusual expertise to know that we are accelerating towards AGI. The problem is that it is not obvious how human intelligence actually works. We simply do not know what it means to 'understand'.
Ask most AGI researchers, philosophers or psychologists as to what it means to 'understand'. They will be stumped to give you a good answer.
So if we do not know this answer, then how can we recognize that the water's temperature is gradually increasing?
The number of people who might know continues to diminish. This implies collectively that we know less and less. When AGI happens, it will come as a shock. It is as if, nobody had anticipated it.
@threadreaderapp unroll

More from Carlos E. Perez

Programming in abstractions is very different from a system that is capable of its own 'abstracting'. But what does abstracting mean? We only know of its inputs and outputs, but we fail to describe its inner workings.

I like this short video about living in space. This is because it makes you realize the gaps in your knowledge when you turn off something (i.e. gravity) that you have always assumed to be present.


Perhaps we can understand 'abstracting' better if we turn of many assumptions that we unconsciously carry around. Perhaps we need to get rid of the excess baggage that is confusing our thinking about abstraction.

Turning off gravity and living in space is a perfect analogy. We somehow have to turn off a cognitive process to understand the meaning of abstraction.

The first step to divorce ourselves from our habitual cognitive processes is to realize the pervasiveness of 'noun-thinking' .

More from Science

You May Also Like

A brief analysis and comparison of the CSS for Twitter's PWA vs Twitter's legacy desktop website. The difference is dramatic and I'll touch on some reasons why.

Legacy site *downloads* ~630 KB CSS per theme and writing direction.

6,769 rules
9,252 selectors
16.7k declarations
3,370 unique declarations
44 media queries
36 unique colors
50 unique background colors
46 unique font sizes
39 unique z-indices

https://t.co/qyl4Bt1i5x


PWA *incrementally generates* ~30 KB CSS that handles all themes and writing directions.

735 rules
740 selectors
757 declarations
730 unique declarations
0 media queries
11 unique colors
32 unique background colors
15 unique font sizes
7 unique z-indices

https://t.co/w7oNG5KUkJ


The legacy site's CSS is what happens when hundreds of people directly write CSS over many years. Specificity wars, redundancy, a house of cards that can't be fixed. The result is extremely inefficient and error-prone styling that punishes users and developers.

The PWA's CSS is generated on-demand by a JS framework that manages styles and outputs "atomic CSS". The framework can enforce strict constraints and perform optimisations, which is why the CSS is so much smaller and safer. Style conflicts and unbounded CSS growth are avoided.
First thread of the year because I have time during MCO. As requested, a thread on the gods and spirits of Malay folk religion. Some are indigenous, some are of Indian origin, some have Islamic


Before I begin, it might be worth explaining the Malay conception of the spirit world. At its deepest level, Malay religious belief is animist. All living beings and even certain objects are said to have a soul. Natural phenomena are either controlled by or personified as spirits

Although these beings had to be respected, not all of them were powerful enough to be considered gods. Offerings would be made to the spirits that had greater influence on human life. Spells and incantations would invoke their


Two known examples of such elemental spirits that had god-like status are Raja Angin (king of the wind) and Mambang Tali Arus (spirit of river currents). There were undoubtedly many more which have been lost to time

Contact with ancient India brought the influence of Hinduism and Buddhism to SEA. What we now call Hinduism similarly developed in India out of native animism and the more formal Vedic tradition. This can be seen in the multitude of sacred animals and location-specific Hindu gods