CritRats!

I think AI risk is a real existential concern, and I claim that the CritRat counterarguments that I've heard so far (keywords: universality, person, moral knowledge, education, etc.) don't hold up.

Anyone want to hash this out with

For instance, while I heartily agree with lots of what is said in this video, I don't think that the conclusion about how to prevent (the bad kind of) human extinction, with regard to AGI, follows.

https://t.co/nbXUsXvcmW
There are a number of reasons to think that AGI will be more dangerous than most people are, despite both people and AGIs being qualitatively the same sort of thing (explanatory knowledge-creating entities).
And, I maintain, that because of practical/quantitative (not fundamental/qualitative) differences, the development of AGI / TAI is very likely to destroy the world, by default.
(I'm not clear on exactly how much disagreement there is. In the video above, Deutsch says "Building an AGI with perverse emotions that lead it to immoral actions would be a crime."
I wouldn't usually put it in those words, but THAT is what the alignment problem is about:

We don't yet know how to reliably build AGI systems _without_ "perverse emotions." It seems like that might be pretty hard to avoid.)
But maybe I'm misunderstanding these arguments.

I would love to dig into this with someone who thinks that AI is not a serious existential risk for reasons related to the above, and together try and answer the question of how these AI is most likely to go.
My win conditions:

1. I change my mind about AI risk, in some way
2. I understand some new-to-me argument that I need to think about in depth
3. I viscerally "get" what I'm missing from the CritRat frame
4. There's a public refutation of the arguments that turn out to be flawed
Here's @reasonisfun vouching for me.

https://t.co/N3S5mER8Z9
I'm happy to talk to you even if your view is not fully representative of "all Critical Rationalists".
@DavidDeutschOxf @iamFilos @campeters4 @MatjazLeonardis @sashintweets @HermesofReason @adilzeshan @thenumber8008 @mansfield_pablo
@DorfGinger @ks445599 @jchalupa_ @RealtimeAI @JimiSommer @chuggfest
Feel free to share with whoever is most likely to be interested.
Feel free to DM me, if you're interested.

More from Eli Tyre

My catch all thread for this discussion of AI risk in relation to Critical Rationalism, to summarize what's happened so far and how to go forward, from here.

I started by simply stating that I thought that the arguments that I had heard so far don't hold up, and seeing if anyone was interested in going into it in depth with


So far, a few people have engaged pretty extensively with me, for instance, scheduling video calls to talk about some of the stuff, or long private chats.

(Links to some of those that are public at the bottom of the thread.)

But in addition to that, there has been a much more sprawling conversation happening on twitter, involving a much larger number of people.

Having talked to a number of people, I then offered a paraphrase of the basic counter that I was hearing from people of the Crit Rat persuasion.

More from Society

So, as the #MegaMillions jackpot reaches a record $1.6B and #Powerball reaches $620M, here's my advice about how to spend the money in a way that will truly set you, your children and their kids up for life.

Ready?

Create a private foundation and give it all away. 1/

Let's stipulate first that lottery winners often have a hard time. Being publicly identified makes you a target for "friends" and "family" who want your money, as well as for non-family grifters and con men. 2/

The stress can be damaging, even deadly, and Uncle Sam takes his huge cut. Plus, having a big pool of disposable income can be irresistible to people not accustomed to managing wealth.
https://t.co/fiHsuJyZwz 3/

Meanwhile, the private foundation is as close as we come to Downton Abbey and the landed aristocracy in this country. It's a largely untaxed pot of money that grows significantly over time, and those who control them tend to entrench their own privileges and those of their kin. 4

Here's how it works for a big lotto winner:

1. Win the prize.
2. Announce that you are donating it to the YOUR NAME HERE Family Foundation.
3. Receive massive plaudits in the press. You will be a folk hero for this decision.
4. Appoint only trusted friends/family to board. 5/
Brief thread to debunk the repeated claims we hear about transmission not happening 'within school walls', infection in school children being 'a reflection of infection from the community', and 'primary school children less likely to get infected and contribute to transmission'.

I've heard a lot of scientists claim these three - including most recently the chief advisor to the CDC, where the claim that most transmission doesn't happen within the walls of schools. There is strong evidence to rebut this claim. Let's look at


Let's look at the trends of infection in different age groups in England first- as reported by the ONS. Being a random survey of infection in the community, this doesn't suffer from the biases of symptom-based testing, particularly important in children who are often asymptomatic

A few things to note:
1. The infection rates among primary & secondary school children closely follow school openings, closures & levels of attendance. E.g. We see a dip in infections following Oct half-term, followed by a rise after school reopening.


We see steep drops in both primary & secondary school groups after end of term (18th December), but these drops plateau out in primary school children, where attendance has been >20% after re-opening in January (by contrast with 2ndary schools where this is ~5%).

You May Also Like