This is mostly a problem with an example you use. I'm not sure whether it points to an underlying issue of your premise:
You link to the exponential growth of transistor density. But that growth is really restricted to just that: transistor density. Growing your number of transistors doesn't necessarily grow your capability to compute things you care about, both from a theoretical perspective (potential fundamental limits in the theory of computation) as well as a practical perspective (our general inability to write code that makes use of much circuitry at the same time + the need for dark silicon + Wirth's law). Other numbers, like FLOP/s, don't necessarily mean what you'd think either.
Moore's law does not posit exponential growth in amount of "compute". It is not clear that the exponential growth of transistor density translates to exponential growth of any quantity you'd actually care about. I think it is rather speculative to assume it does and even more so to assume it will continue to.
(This article was originally posted on my website here. I'd love to get an Effective Altruist critique of it. In general, I think long-termist EAs should give meta x-risk more attention. I'm sorry the images are so small.)
“Since Auschwitz we know what man is capable of. And since Hiroshima we know what is at stake." —Viktor Frankl, Man's Search for Meaning
"The greatest shortcoming of the human race is our inability to understand the exponential function." —Al Bartlett, Professor at University of Colorado
The goal of this article is to help readers understand the concept of Meta Existential Risk. I'm surprised that (essentially) nothing has been written on it yet.
Before understanding Meta Existential Risk, it's helpful to understand Existential Risk (x-risks). Existential Risks are possible future events that would "wipe out" all sentient life on Earth. A massive asteroid impact, for example. Existential Risks can be contrasted with non-existentialGlobal Catastrophic Risks, which don't wipe out all life but produce lots of instability and harm. A (smaller) asteroid impact, for example :).
Meta Existential Risk gets at at the question: whydo existential risks occur? The answer: exponential curves combined with bad game theory. Let's look at our 4 primary existential risks. As you can see, each one is an exponential curve.
Magnitude for Destruction (AI, BioTech, Nukes): In this case, the x-risk is deadly simply because it represents a massive increase in power. There is a lot of power (destructive or not) in 60,000 nukes, super cheap genomic sequencing, or robust AI. And, as the Orthogonality Thesis states: power is orthogonal to alignment with human values. More power doesn't necessarily mean more responsibility.
Negative Externalities (Climate Change): In this case, the x-risk is deadly not because of "the tech itself", but because of the tech's negative externalities. The Industrial Revolution (inadvertently) pumped a lot of CO2 into the atmosphere, which has led to climate change. The problem is not factories or planes or whatever, but rather the ignored negative externality.
(Daniel Schmachtenberger defines the 1st issue as: "Rivalrous (win-lose) games multiplied by exponential technology self terminate." And the 2nd issue as: "The need to learn how to build closed loop systems that don’t create depletion and accumulation, don’t require continued growth, and are in harmony with the complex systems they depend on.")
These risks might not be such a big deal if we were able to look at the negative impact of the exponential curve and say "oh, let's just fix it". The problem is that we can't because we're stuck in two kinds of bad game theory problems (often called "coordination problems").
Our "Power" Problems are Arms Races: For powerful new tech, all of the players are incentivized to make it more powerful without thinking enough about safety. If one player thinks about safety but goes slower, then the other player "wins" the arms race. e.g. Russia and the U.S. both build more nukes, Google and Baidu create powerful AI as fast as possible (not as safe as possible), and CRISPR companies do the same.
Negative Externalities are a Tragedy of the Commons: It is in each country's best interest to ignore/exploit the global commons because everyone else is doing the same. If they try to carbon tax, underfish, etc. and no one else does, then they "lose".
This is the root of meta x-risk: exponential curves with negative impacts that humans need meta-coordination to stop.
These problems don't just hold for these four specific exponential curves. Those are just the four that we're dealing with today. There will be more exponential curves in the future that will move more quickly and be more powerful. This leads to our final definition.
Meta Existential Risk is the risk to humanity created by (current and future) exponential curves, their misalignment with human values and/or our Earth system, and our inability to stop them given coordination problems.
Categorically Solving Meta X-Risk
I won't dive deeply into my hypotheses for how to "categorically solve" meta x-risk here. However, I'll show you what this looks like in a directional sense. For any current/future exponential:
To solve the negative externality problem, we need to turn exponentials into S-curves that create closed loop systems with our biological substrate.
To solve the power problem, we need to ensure our exponentials align with human values.
Hope this article was helpful in understanding meta x-risk. Comment below with your thoughts/feedback!
Notes:
Notice how similar the two solves for meta x-risk are. Both are through-time vectors that align the exponential with its substrate. In case 1, it's aligning the exponential with the physical world (Nature 1.0). i.e. Not going too fast and replenishing the complex biological system. In case 2, it's aligning the exponential with human values (Nature 2.0). (Nature 3.0 is AI/tech btw.) I'm not totally sure, but this feels very connected to Forrest Landry's idea of continuity.
I wanted to note this more explicitly in the visuals, but there's another key way to conceptualize of meta x-risk: as a natural conclusion of entropy—it's easier to destroy than to create. (1000 years to make a city, a single a-bomb to destroy it in minutes.)
In the article above I make it seem like all Negative Externalities are a Tragedy of the Commons, and all Power Problems are Arms Races. I don't think this is necessarily true. Or at least, I haven't formalized why this might be true. Thoughts?
In these possibly intense times, I like to remind myself of thing like "we're incredibly insignificant". I mean damn, the universe can only harbor life for like 1 quintillionth of it's existence. (See Melodysheep's Timelapse of the Future.)
Resilience, Antifragility, and the Precautionary Principle: We should probably just assume lots of crazy stuff is going to happen, then make ourselves able to handle it.
Ok, so we want to evolve a new Theory of Choice that keeps our exponential reality from self-terminating. What else do we need? I’d also argue that we need “Abstract Empathy”.
This is a crosspost from the new Animal Welfare Alignment Newsletter by Anima International. You can subscribe on Substack if you are interested in following these efforts. Audio reading also available on Substack.
The goals of this post are to:
1. Raise a question I see as crucially important to the goal of aligning AI to animal welfare...
I used AI to fix transcription errors, rerrarange the ideas, and suggest tweaks to the title and some sentences.
Three of the most exciting projects to come out of EA in recent years are, in a vague sense, CEA spinouts:
* Kairos is directly a spinout of CEA and now handles most support for university AI safety groups. Basically everyone I've found who knows them is really excited about what they do
* NEST is an opinionated ideas-fi...
Hello! I'm Justin Portela. I got hired by GWWC to make YouTube videos after AI in Context did such a kickass job.
My channel is using that same cinematic, high-production value beauty to talk about everything in the EA universe that isn't AI.
...
A more relevant curve for nuclear weapons might be "TNT equivalents" or "cost per ton TNT equivalent".