In this post, I propose the idea that the communication around current AI alignment risk could be improved. I then suggest a possible improvement which is easy for people to adopt individually, and could also be adopted as a whole. This is not meant as an authoritative recommendation, but more of a "putting it out there" discussion topic.

Outline

• My experience of learning about AI alignment, and why it didn’t persuade me
• The current approach used to evangelize AI alignment awareness, and why it is ineffective
• Why we arrived at this ineffective approach
• An alternative approach that’s more effective
• Proposed action: use the effective approach to spread awareness of AI risk, not the current ineffective approach

Epistemic statuses:

Certain for my personal experience

Very high for the average person's experience

High for the ineffectiveness of current communication method

Medium-high for the effectiveness of proposed communication method

My experience of learning about AI alignment

As a regular person who is new to EA, I would like to share my experience of learning about AI alignment risk. The first thing I came across was the “paperclip maximizer” idea. Then, I read about superintelligence explosion “foom”. These are the top results when searching for “AI existential risk”, so I think it’s a representative sample.

After reading about these two topics, I rejected AI risk. Not in the sense of “I read the pros and cons and decided that the risk was overrated”, but in the sense of “this is stupid, and I’m not even going to bother thinking about it”.

I did this because these scenarios are too far from my model of the world and did not seem credible. Since I had no reason to believe that this line of enquiry would lead to useful learning, I abandoned it.

The current persuasion approach is ineffective

There are two types of people we want to persuade. Firstly, people with power and influence, and secondly, the public. The first category is important because they have leverage to create outcomes, and the second category is important because it is the main determinant of how favorable the “playing field” is.

I should be much easier to persuade than the average member of the public, because I was interested in learning more about AI risk! But even I bounced off. It will surely be an insurmountable climb for regular people who are disinterested in the topic. This is a disaster of messaging if the goal is to persuade the public.

Is it a good approach for persuading the first category of people, those with power and influence? We might expect it to be more effective, because they have stronger analytical capabilities and would be more receptive to logical analysis. Unfortunately, this misunderstands power and influence. This might work if we’re targeting rich people, because analytical power is correlated with wealth generation. But people with power and influence generally do NOT respond well to analytic arguments. Instead, they respond to arguments focusing on power and influence, which is completely unaddressed by the paperclip foom approach. This line of persuasion may even backfire, by reducing its perceived credibility.

The current persuasion approach is correct!

What’s vexing is that persuading people by talking about paperclip foom is clearly correct. As Carl Sagan said, “extraordinary claims require extraordinary evidence”. Taking the outside view, AI alignment risk is an extraordinary claim, because it’s so alien to our general understanding of the world. So if that’s the case, we need to provide extraordinary evidence.

Unfortunately, extraordinary evidence consists of claiming catastrophic outcomes and then showing how they could come about. This is ineffective because the reader is required to agree to a huge inferential distance. In most cases, the reader will not even read the evidence or evaluate the evidence before dismissing it.

This is an “Always Be Closing” approach. Make a big ask, if you encounter resistance, you can then retreat slightly and make a sale. A small ask such as “AI alignment is potentially dangerous” is feeble, instead, let’s alarm people by announcing a huge danger, getting them to pay attention. Attention, Interest, Desire, Action!

It’s a pity that this approach crashes and burns at the Interest stage.

An alternative, effective approach

Another sales maxim is “don’t sell past the close”. Just do enough to get the sale - in this case, the “sale” is to get people thinking that AI alignment is a serious problem that we should care about. What would this approach look like?

Recently Nintil posted his thoughts about AGI. His approach basically claims that the “Always Be Closing” maximalist approach is unnecessary. This is encapsulated by a quote within the blogpost:

“Specifically, with the claim that bringing up MNT [i.e. nanotech] is unnecessary, both in the "burdensome detail" sense and "needlessly science-fictional and likely to trigger absurdity heuristics" sense. (leplen at LessWrong, 2013)”

Nintil makes a case that AGI is a concern even without the “needlessly science-fictional” stuff. I found this interesting and worth serious consideration, because it didn’t require a large inferential leap and so it was credible. “Here are some assumptions that you probably already have. I’ll show how this leads to bad outcomes” is a much more effective approach, and it caused me to re-evaluate the importance of AI alignment.

We should use the effective approach, not the ineffective approach

AI alignment persuasion does not require paperclip foom discussions, and these are likely counterproductive. Since an effective explanation exists, I think it would be helpful to focus on this line when evangelizing AI risk.

You may be tempted to discount my opinion because I am an average person, but I think my viewpoint is useful because the extremely high intelligence of the median effective altruist leads to some blindspots. A recent example in the news is the failure to elect Carrick Flynn, an effective altruist running for a congressional seat in Oregon. It wasted $10 million and backfired as “a big optical blunder, one that threatened to make not just Bankman-Fried but all of EA look like a craven cover for crypto interests.”

Being highly intelligent makes it harder to visualize how a regular person perceives things. I claim that my perspective here is valuable for EA, since we’re considering the question of how to persuade regular people. Please give this serious consideration!

Finally, this would also help other EA cause areas. All effective altruists contribute to the EA commons by working hard on various causes. Maximalist AI alignment discussion is burning the commons because it makes EAs look “weird”, which reduces the influence of EA as a whole. Therefore, not just AI alignment EAs, but all EAs have a stake in moving to a more effective communications method for AI risk.