Effective Altruism Forum
EA Forum

Hide table of contents
by yanni kyriacos
Aug 15 20231 min read3 answers 5

24

CommunityAI safetyAI governanceSlowing down AI
Frontpage

I feel like I'm taking crazy pills. 

It appears that many EAs believe we shouldn't pause AI capabilities until it can be proven to have < ~ 0.1% chance of X-risk. 

Put less confusingly, it appears many EAs believe we should allow capabilities development to continue despite the current X-risks.

This feels obviously a terrible thing to me.

What are the best reasons EA shouldn't be pushing for an indefinite pause on AI capabilities development??

24

0
0

Reactions

0
0
New Answer
New Comment

3 Answers sorted by
Top

Zach Stein-Perlman

Aug 16, 2023

20
6
0

Messy practical reasons.

I agree with Larks that most of us would press a magic button to slow down AI progress on dangerous paths.

But we can't, which raises two problems:

  1. Tractability.
  2. Effectiveness of moderate success. If you get a non-global slowdown, or a slowdown that ends too early, or a slowdown regime that's evadable, or if you differentially slow cautious labs, or even if you just differentially slow leading labs, the effect is likely net-negative. (Increasing multipolarity among labs + differentially boosting less-cautious actors + compute overhang enabling rapidly scaling up training compute. See Slowing AI: Foundations.)

(I'd be excited to talk about proposals more specific than 'push for a pause,' or outcomes more specific than 'pause until proven <0.1% doom.' Who is doing the pausing; what are the rules? Or maybe you don't have specific proposals/outcomes in mind, in which case I support you searching for new great ideas, but it's not like others haven't tried and failed already.)

yanni kyriacos8mo3
0
0

Thanks for the comment Zach. 

1. Can you elaborate on your comment "Tractability"?

2. I'm less worried about multipolarity because the leading labs are so far ahead AND I have short timelines (~ 10 years). My guess is if you had short timelines, you might agree?

3. If we had moderate short term success, my intuition is that we've actually found an effective strategy that could then be scaled. I worry that your thinking is basically pointing to 'it needs to be an immediately perfect strategy or don't bother!'

Reply
2
Zach Stein-Perlman
8mo
1. Pushing a magic button would be easy; affecting the real world is hard. Even if slowing is good, we should notice whether there exist tractable interventions (or: notice interventions' opportunity cost). 2. Nope, my sense is that DeepMind, OpenAI, and Anthropic do and will have a small lead over Meta, Inflection, and others, such that I would be concerned (re increasing multipolarity among labs) about slowing DeepMind, OpenAI, and Anthropic now. (And I have 50% credence on human-level AI [noting this is underspecified] within 9 years.) 3. Yeah, maybe, depending. I'm relatively excited about "short term success" that seems likely to support the long-term policy regimes I'm excited about, like global monitoring of compute and oversight of training runs with model evals for dangerous capabilities and misalignment, maybe plus a compute cap. I fear that most pause-flavored examples of "short term success" won't really support great long-term plans. (Again, I'd be excited to talk about specific proposals/outcomes/interventions.)

Ben_West

Aug 16, 2023

12
0
0
1

This sequence is still in progress but is the best collection of resources that I know of regarding slowing AI (including an indefinite pause).

Greg_Colbourn8mo2
0
0

Worth cross-posting to the EA Forum. @Zach Stein-Perlman 

Reply
yanni kyriacos8mo2
0
0

Thank you Ben I will take a look :)

Reply

Ariel Simnegar

Aug 15, 2023

0
0
8

I think the best reason is that it's not within the Overton window :)

Peter Berggren8mo9
4
2

Last I checked, the whole point of the Overton window is that you can only shift it by advocating for ideas outside of it.

Reply
4
yanni kyriacos
8mo
I'm confused - where is the evidence it is outside the overton window?
5
Peter Berggren
8mo
Don't really think there is any; in fact, there's plenty of evidence to the contrary, from the polls I've seen.
5
David_Moss
8mo
We found 51% support, 25% opposition in our polling here.
Comments5
Sorted by
New & upvoted
Click to highlight new comments since: Today at 3:19 AM
Larks8mo25
10
0

There's a big gap between "believe we should press it if presented with a magic global pause button" and "think pro-pause advocacy is the most efficient use of their time on the current margin". I suspect the majority of Xrisk-concerned EAs would press such a button if given the chance.

Reply
yanni kyriacos8mo-3
0
3
  1. i think you'd be surprised how many wouldn't press such a button
  2. what publicly available thinking have you seen on the potential impact of advocacy in this context (within EA)? i'm interested how you've formed this opinion
Reply
Larks8mo4
2
0

You can run a poll to try to find out the answer to 1) if you want!

Reply
Pato8mo12
2
0

it appears many EAs believe we should allow capabilities development to continue despite the current X-risks.

Where do you get this from?

Also, this:

have < ~ 0.1% chance of X-risk. 

means p(doom) <~ 0.001

Reply
yanni kyriacos8mo4
1
0

"Where do you get this from?"

  • the lack of content and discussion regarding a pause on the forum / podcasts / twitter
  • the dismal success rate of grant requests to drive such a strategy
  • vibe
Reply
More from yanni kyriacos
View more
Curated and popular this week
Relevant opportunities
View more