Hide table of contents

This is a summary of the following paper by Alexia Georgiadis (Existential Risk Observatory): https://existentialriskobservatory.org/papers_and_reports/The_Effectiveness_of_AI_Existential_Risk_Communication_to_the_American_and_Dutch_Public.pdf

Thanks to @Lara Mani, @Karl von Wendt, and Alexia Georgiadis for their help in reviewing and writing this post. Any views expressed in this post are not necessarily theirs.

The rapid development of artificial intelligence (AI) has evoked both positive and negative sentiments due to its immense potential and the inherent risks associated with its evolution. There are growing concerns that if AI surpasses human intelligence and is not aligned with human values, it may pose significant harm and even lead to the end of humanity. However, the general publics' knowledge of these risks is limited. As advocates for minimising existential threats, the Existential Risk Observatory believes it is imperative to educate the public on the potential risks of AI. Our introductory post outlines some of the reasons why we hold this view (this post is also relevant). To increase public awareness of AI's existential risk, effective communication strategies are necessary. This research aims to assess the effectiveness of communication interventions currently being used to increase awareness about AI existential risk, namely news publications and videos. To this end, we conducted surveys to evaluate the impact of these interventions on raising awareness among participants.

Methodology

This research aims to assess the effectiveness of different media interventions, specifically news articles and videos, in promoting awareness of the potential dangers of AI and its possible impact on human extinction. It analyses the impact of AI existential risk communication strategies on the awareness of the American and Dutch populations, and investigates how social indicators such as age, gender, education level, country of residence, and field of work affect the effectiveness of AI existential risk communication. 

The study employs a pre-post design, which involves administering the same intervention and assessment to all participants and measuring their responses at two points in time. The research utilises a survey method for collecting data, which was administered to participants through an online Google Forms application. The survey consists of three sections: pre-test questions, the intervention, and post-test questions.

The effectiveness of AI existential risk communication is measured by comparing the results of quantitative questions from the pre-test and post-test sections, and the answers to the open-ended questions provide further understanding of any changes in the participant's perspective. The research measures the effectiveness of the media interventions by using two main indicators: "Human Extinction Events" and "Human Extinction Percentage."

The "Human Extinction Events" indicator asks participants to rank the events that they believe could cause human extinction in the next century, and the research considers it as effective if participants rank AI higher post intervention or mention it after the treatment when they did not mention it before. If the placement of AI remained the same before and after the treatment, or if participants did not mention AI before or after the treatment, the research considered that there was no effect in raising awareness.

The "Human Extinction Percentage" indicator asks for the participants' opinion on the likelihood, in percentage, of human extinction caused by AI in the next century. If there was an increase in the percentage of likelihood given by participants, this research considered that there was an effect in raising awareness. If there is no change or a decrease in the percentage, this research considered that there was no effect in raising awareness.

The study recruited American and Dutch residents who were 18 years or older. The data was collected through the use of Prolific, a platform that finds survey participants based on the researcher's predetermined criteria. Each of the 10 surveys conducted involved 50 participants, for a total of 500 participants. 

Some of the findings

Our study found that media interventions can effectively raise awareness of AI existential risk. Depending on the type of media item used, the percentage of participants whose awareness increased per the Human Extinction Events indicator ranged from 26% to 64%, with an average of 40%. While we did not investigate the duration of the effect, previous research has shown that about half of the measured effects tend to persist over time.

Moreover, we found an average increase of 6.8% in reported Human Extinction Percentage among the participants. This means that participants' perceptions of the likelihood of human extinction due to AI increased from 13.5% to 20.3% on average for all 500 participants. The percentage increase in awareness was higher for video-based interventions than for articles, with an average increase of 4.8%, and this difference was statistically significant (p < 0.01).

We also found that women were more influenced by media interventions compared to men, with an average increase of 4.3% (p < 0.01), and participants with a Bachelor's degree were more susceptible to the media interventions than those with other education levels, with an average increase of 3.0% (p < 0.05). In addition, our research revealed that people tend to view institutions as important in addressing existential threats and may favour government regulation or prohibition of AI development.

Interestingly, educating people about the potential risks of AI through media interventions may not necessarily result in them supporting military involvement in AI development. Furthermore, our study discovered a widespread endorsement of government participation in AI regulation among individuals with heightened awareness of the risks.

Finally, our research found that mainstream media channels are generally perceived as more trustworthy than YouTube channels, and scholarly articles are the preferred research resource for Americans and Dutch individuals seeking to learn more about AI's existential threat. Overall, our findings demonstrate that media interventions can play a crucial role in raising awareness of the potential risks associated with AI, and suggest that future efforts should continue to explore effective communication strategies.

Conclusion

The study aimed to investigate the effectiveness of communication strategies in raising awareness of AI existential risks among the general public in the US and the Netherlands. Results showed that mass media items can successfully raise awareness, with female participants and those with a bachelor's degree being more receptive. The study also found that despite increased awareness, there was no rise in the proportion of people who endorse military involvement in AI development. Future research should investigate the impact of narratives on media interventions and explore other aspects such as content length and different formats. Research on other countries could also be valuable to identify general trends in AI existential risk communication.

Appendix

Findings from Human Extinction Events and Human Extinction Percentage per Survey

Figure 1: Percentage of participants who exhibited higher awareness after the intervention across surveys. 

Figure 2: Pre-post summary of the distribution of percentage values for the Human Extinction Percentage indicator across surveys

Comments11
Sorted by Click to highlight new comments since: Today at 8:39 PM

Otto (& Lara, Karl, & Alexia) -- thanks very much for sharing this fascinating research. Great to see empirical work on X risk communication strategies.

It's encouraging to know that even fairly brief interventions (watching videos or reading text) can significantly increase people's awareness of AI X risks. 

It looks like the CNN clip featuring Stephen Hawking was especially effective -- maybe given Hawking's scientific status and reputation as a genius, and the fact that he's not seen as a politically polarizing or controversial figure (unlike Elon Musk or PewDiePie). 

I look forward to reading the paper in depth, and I hope it gets more attention here, and when it's published. (Have you submitted it to a journal yet?)

It looks like the CNN clip featuring Stephen Hawking was especially effective -- maybe given Hawking's scientific status and reputation as a genius, and the fact that he's not seen as a politically polarizing or controversial figure (unlike Elon Musk or PewDiePie).

I don't think reputation can explain the effectiveness:

  1. Both the CNN video and the CNBC article mention Elon Musk.
  2. The CNBC article had unremarkable effects compared to the other media items, even though it was mostly about Stephen Hawking.
  3. Most of the CNN video (1:20 to the end, 5:51) is an interview with James Barrat, a documentary filmmaker and author.

James Barrat seems to be an excellent orator. I disagree with some of his points in the CNN video, but his presentation is very smooth. Maybe this explains the video's success.

BTW I'd be careful about drawing inferences from data via Prolific without attempts to reweigh the sample for representativeness. The raw Prolific crew is quite liberal and quite educated. 74% of them voted for Joe Biden over Donald Trump.

Thanks for publishing this — really useful to have this data.

Thanks for sharing!

If there was an increase in the percentage of likelihood given by participants, this research considered that there was an effect in raising awareness. If there is no change or a decrease in the percentage, this research considered that there was no effect in raising awareness.

Would it make more sense to measure the "mean variation in likelihood" = "mean post-intervention likelihood" - "mean pre-intervention likelihood"? I think so, because it accounts for:

  • Potential negative effects of the intervention. I would say classifying these as "no effect in raising awareness" is incomplete.
  • The magnitude of the decrease or increase in awareness. With the method you describe, a very large decrease/increase would be classified as "raising awareness" just like a very small decrease/increase.

Am I missing something?

Hi Vasco, thank you for taking the time to read our paper!

Although we did not specify this in the methodology section, we addressed the "mean variation in likelihood" between countries and surveys throughout the research such as in section 4.2.2. I hope this clarifies your question. This aspect should have been better specified in the methodology section.

If you have any more questions, do not hesitate to ask.

Thanks for this great work - I am keen to see more related research!

Thanks Peter for the compliment! If there is something in particular you're interested in, please let us know and perhaps we can take it into account in future research projects!

Interesting, thanks ! 

I think something like "here's a list of resources to share on AI Existential risk that have a good track record of raising awareness for public X" would be very interesting.

Thanks for this! I think it would be helpful to plot the median changes in extinction probabilities against the number of words in the article/video. I'm noticing a correlation as I click through the links and would be curious how strong it is (so this effect can be disentangled from the style of the source).

Hi Joshc, thanks and sorry for the slow reply, it's a good idea! Unfortunately we don't really have time right now, but we might do something like this in the future. Also, if you're interested in receiving the raw data, let us know. Thanks again for the suggestion!