This post summarizes the new paper Current and Near-Term AI as a Potential Existential Risk Factor, authored by Benjamin Bucknal and Shiri Dori-Hacohen. The paper diverges from the traditional focus on the potential risks of artificial general intelligence (AGI), instead examining how existing AI systems may pose an existential threat. The authors underscore the urgency of mitigating the harms from these AI systems by detailing how they might contribute to catastrophic outcomes.

Key points:

  1. Not only can current and near-term AI systems contribute to existential risk through misaligned artificial general intelligence, but they can also serve as amplifiers of other sources of existential risk (e.g., AI may help design dangerous pathogens.)
  2. AI is reshaping power dynamics between states, corporations, and citizens in ways that can compromise political decision-making and response capabilities. This includes:
    • An AI arms race between states could lead to nuclear conflict and divert resources away from mitigating existential risks.
    • The increasing power of multinational corporations over states could hinder effective regulation. AI technology is already outpacing ethical and legal frameworks.
    • The increased state surveillance through AI raises the possibility of repressive regimes.
  3. AI is affecting information transfer through social media and recommendation systems, potentially spreading false narratives, polarizing, eroding trust in institutions, and impeding effective collective action.
  4. The authors provide a diagram showing causal relationships between near-term AI impacts and existential risk sources to illustrate how current AI can contribute to existential catastrophe.

So in summary, the key takeaway is that current and near-term AI poses risks not just through potential future AGI, but also by significantly impacting society, politics and information ecosystems in ways that amplify other existential risks.

Acknowledgment: The idea for this summary and its commission came from Michael Chen, whom I would like to thank for his support.

12

0
0

Reactions

0
0
Comments1


Sorted by Click to highlight new comments since:
[anonymous]6
1
0

Thanks for the summary :)

Curated and popular this week
 ·  · 1m read
 · 
I recently read a blog post that concluded with: > When I'm on my deathbed, I won't look back at my life and wish I had worked harder. I'll look back and wish I spent more time with the people I loved. Setting aside that some people don't have the economic breathing room to make this kind of tradeoff, what jumps out at me is the implication that you're not working on something important that you'll endorse in retrospect. I don't think the author is envisioning directly valuable work (reducing risk from international conflict, pandemics, or AI-supported totalitarianism; improving humanity's treatment of animals; fighting global poverty) or the undervalued less direct approach of earning money and donating it to enable others to work on pressing problems. Definitely spend time with your friends, family, and those you love. Don't work to the exclusion of everything else that matters in your life. But if your tens of thousands of hours at work aren't something you expect to look back on with pride, consider whether there's something else you could be doing professionally that you could feel good about.
 ·  · 14m read
 · 
Introduction In this post, I present what I believe to be an important yet underexplored argument that fundamentally challenges the promise of cultivated meat. In essence, there are compelling reasons to conclude that cultivated meat will not replace conventional meat, but will instead primarily compete with other alternative proteins that offer superior environmental and ethical benefits. Moreover, research into and promotion of cultivated meat may potentially result in a net negative impact. Beyond critique, I try to offer constructive recommendations for the EA movement. While I've kept this post concise, I'm more than willing to elaborate on any specific point upon request. Finally, I contacted a few GFI team members to ensure I wasn't making any major errors in this post, and I've tried to incorporate some of their nuances in response to their feedback. From industry to academia: my cultivated meat journey I'm currently in my fourth year (and hopefully final one!) of my PhD. My thesis examines the environmental and economic challenges associated with alternative proteins. I have three working papers on cultivated meat at various stages of development, though none have been published yet. Prior to beginning my doctoral studies, I spent two years at Gourmey, a cultivated meat startup. I frequently appear in French media discussing cultivated meat, often "defending" it in a media environment that tends to be hostile and where misinformation is widespread. For a considerable time, I was highly optimistic about cultivated meat, which was a significant factor in my decision to pursue doctoral research on this subject. However, in the last two years, my perspective regarding cultivated meat has evolved and become considerably more ambivalent. Motivations and epistemic status Although the hype has somewhat subsided and organizations like Open Philanthropy have expressed skepticism about cultivated meat, many people in the movement continue to place considerable hop
 ·  · 7m read
 · 
Introduction I have been writing posts critical of mainstream EA narratives about AI capabilities and timelines for many years now. Compared to the situation when I wrote my posts in 2018 or 2020, LLMs now dominate the discussion, and timelines have also shrunk enormously. The ‘mainstream view’ within EA now appears to be that human-level AI will be arriving by 2030, even as early as 2027. This view has been articulated by 80,000 Hours, on the forum (though see this excellent piece excellent piece arguing against short timelines), and in the highly engaging science fiction scenario of AI 2027. While my article piece is directed generally against all such short-horizon views, I will focus on responding to relevant portions of the article ‘Preparing for the Intelligence Explosion’ by Will MacAskill and Fin Moorhouse.  Rates of Growth The authors summarise their argument as follows: > Currently, total global research effort grows slowly, increasing at less than 5% per year. But total AI cognitive labour is growing more than 500x faster than total human cognitive labour, and this seems likely to remain true up to and beyond the point where the cognitive capabilities of AI surpasses all humans. So, once total AI cognitive labour starts to rival total human cognitive labour, the growth rate of overall cognitive labour will increase massively. That will drive faster technological progress. MacAskill and Moorhouse argue that increases in training compute, inference compute and algorithmic efficiency have been increasing at a rate of 25 times per year, compared to the number of human researchers which increases 0.04 times per year, hence the 500x faster rate of growth. This is an inapt comparison, because in the calculation the capabilities of ‘AI researchers’ are based on their access to compute and other performance improvements, while no such adjustment is made for human researchers, who also have access to more compute and other productivity enhancements each year.