CritRats!

I think AI risk is a real existential concern, and I claim that the CritRat counterarguments that I've heard so far (keywords: universality, person, moral knowledge, education, etc.) don't hold up.

Anyone want to hash this out with

For instance, while I heartily agree with lots of what is said in this video, I don't think that the conclusion about how to prevent (the bad kind of) human extinction, with regard to AGI, follows.

https://t.co/nbXUsXvcmW
There are a number of reasons to think that AGI will be more dangerous than most people are, despite both people and AGIs being qualitatively the same sort of thing (explanatory knowledge-creating entities).
And, I maintain, that because of practical/quantitative (not fundamental/qualitative) differences, the development of AGI / TAI is very likely to destroy the world, by default.
(I'm not clear on exactly how much disagreement there is. In the video above, Deutsch says "Building an AGI with perverse emotions that lead it to immoral actions would be a crime."
I wouldn't usually put it in those words, but THAT is what the alignment problem is about:

We don't yet know how to reliably build AGI systems _without_ "perverse emotions." It seems like that might be pretty hard to avoid.)
But maybe I'm misunderstanding these arguments.

I would love to dig into this with someone who thinks that AI is not a serious existential risk for reasons related to the above, and together try and answer the question of how these AI is most likely to go.
My win conditions:

1. I change my mind about AI risk, in some way
2. I understand some new-to-me argument that I need to think about in depth
3. I viscerally "get" what I'm missing from the CritRat frame
4. There's a public refutation of the arguments that turn out to be flawed
Here's @reasonisfun vouching for me.

https://t.co/N3S5mER8Z9
I'm happy to talk to you even if your view is not fully representative of "all Critical Rationalists".
@DavidDeutschOxf @iamFilos @campeters4 @MatjazLeonardis @sashintweets @HermesofReason @adilzeshan @thenumber8008 @mansfield_pablo
@DorfGinger @ks445599 @jchalupa_ @RealtimeAI @JimiSommer @chuggfest
Feel free to share with whoever is most likely to be interested.
Feel free to DM me, if you're interested.

More from Eli Tyre

My catch all thread for this discussion of AI risk in relation to Critical Rationalism, to summarize what's happened so far and how to go forward, from here.

I started by simply stating that I thought that the arguments that I had heard so far don't hold up, and seeing if anyone was interested in going into it in depth with


So far, a few people have engaged pretty extensively with me, for instance, scheduling video calls to talk about some of the stuff, or long private chats.

(Links to some of those that are public at the bottom of the thread.)

But in addition to that, there has been a much more sprawling conversation happening on twitter, involving a much larger number of people.

Having talked to a number of people, I then offered a paraphrase of the basic counter that I was hearing from people of the Crit Rat persuasion.

More from Society

The Nashville Operation - A Battle in the War

A thread exploring the Nashville bombing in the context of the 2020 Digital War (via SolarWinds) against the United States perpetrated by our enemies, likely China, Iran and/or Russia.


SolarWinds Hack

A digital "Pearl Harbor" moment for the United States, whoever was responsible had access to the keys to the kingdom for months during 2020, including sensitive military infrastructure. This is war!

SunGard + SolarWinds

SolarWinds software company is owned by same company that owns SunGard, which essentially provides data center services. A secure place to host internet servers with redundant power and "big pipe" data connections.

https://t.co/U3P3SrrkM1


SunGard Data Center

In Nashville, around the corner from their "big pipe" connection, AT&T. Like any data center, highly secure. Only authorized personnel can enter, and even fewer can access the actual server rooms. Backup generators are available in case of power failure.


If the SunGard hardware was being used to "host" critical command and control software related to SolarWinds, the US powers would be very interested in gaining special access keys that are stored on the hard-drives of specific servers.
Patriotism is an interesting concept in that it’s excepted to mean something positive to all of us and certainly seen as a morally marketable trait that can fit into any definition you want for it.+


Tolstoy, found it both stupid and immoral. It is stupid because every patriot holds his own country to be the best, which obviously negates all other countries.+

It is immoral because it enjoins us to promote our country’s interests at the expense of all other countries, employing any means, including war. It is thus at odds with the most basic rule of morality, which tells us not to do to others what we would not want them to do to us+

My sincere belief is that patriotism of a personal nature, which does not impede on personal and physical liberties of any other, is not only welcome but perhaps somewhat needed.

But isn’t adherence to a more humane code of life much better than nationalistic patriotism?+

Göring said, “people can always be brought to the bidding of the leaders. That is easy. All you have to do is tell them they are being attacked, and denounce the peacemakers for lack of patriotism and exposing the country to danger. It works the same in any country.”+

You May Also Like

🌿𝑻𝒉𝒆 𝒔𝒕𝒐𝒓𝒚 𝒐𝒇 𝒂 𝑺𝒕𝒂𝒓 : 𝑫𝒉𝒓𝒖𝒗𝒂 & 𝑽𝒊𝒔𝒉𝒏𝒖

Once upon a time there was a Raja named Uttānapāda born of Svayambhuva Manu,1st man on earth.He had 2 beautiful wives - Suniti & Suruchi & two sons were born of them Dhruva & Uttama respectively.
#talesofkrishna https://t.co/E85MTPkF9W


Now Suniti was the daughter of a tribal chief while Suruchi was the daughter of a rich king. Hence Suruchi was always favored the most by Raja while Suniti was ignored. But while Suniti was gentle & kind hearted by nature Suruchi was venomous inside.
#KrishnaLeela


The story is of a time when ideally the eldest son of the king becomes the heir to the throne. Hence the sinhasan of the Raja belonged to Dhruva.This is why Suruchi who was the 2nd wife nourished poison in her heart for Dhruva as she knew her son will never get the throne.


One day when Dhruva was just 5 years old he went on to sit on his father's lap. Suruchi, the jealous queen, got enraged and shoved him away from Raja as she never wanted Raja to shower Dhruva with his fatherly affection.


Dhruva protested questioning his step mother "why can't i sit on my own father's lap?" A furious Suruchi berated him saying "only God can allow him that privilege. Go ask him"