Epistemic status: Shower thought quickly written (once dry). Partly just a reframing of existing ideas. See also other work on differential progress. My examples are oversimplified, and I don’t necessarily actually endorse the intervention ideas mentioned.

Summary

Let’s say, for example, that we want certain major advances in AI or synthetic biology to occur 50 rather than 5 years from now. I think there are three major reasons why we might want that - three different “active ingredients” that could account for the potential benefits of slower technology development:

• Ordering: it might be good if a given technology were developed after some other specific development(s)
• Gradualness: it might be good if a given technology were developed gradually rather than suddenly/“discontinuously”
• Distance from now: it might be good if a given technology were developed later rather than sooner, for reasons other than ordering and gradualness

I think it would often be useful to explicitly distinguish between these reasons and consider how much we care about each in a given case, because they suggest different interventions and different factors to consider. I give some examples below.

Ordering

We might think it would be good if a given technology (let’s give it the imaginative name “technology X”) were developed after some other specific development(s), because those other developments reduce the risks from technology X. For example, we might want various developments in AI capability to occur after rather than before various developments in AI safety, alignment, policy, or governance.

For this, absolute distance from now doesn’t matter in itself, but rather serves as a useful proxy - since, generally speaking, the longer it’ll be till technology X is developed, the more likely it is that the risk-reducing development(s) occur first. For example, we might not mind whether technology X will be developed 5 or 50 years from now if we believed that, either way, the risk-reducing developments are equally likely to occur first.

The more we care about ordering, the more we might be interested in:

• Improving our forecasts of when the risk-reducing developments will occur, or our forecasts on binary questions about which of various pairs of developments will occur first
• Accelerating progress towards risk-reducing developments
• Increasing the chance that progress toward the risk-reducing developments speeds up if development of technology X speeds up, or that the development of technology X slows down if progress toward the risk-reducing developments seems to be lagging behind
  • E.g., increasing the chance that AI development slows down if it becomes apparent that AI safety/alignment progress is proceeding much more slowly than anticipated

Gradualness

We might think it would be good if technology X were developed gradually rather than suddenly/“discontinuously”. That is, roughly speaking, we might want it to be the case that the development proceeds in many small steps rather than a smaller number of big jumps (separately from how long from now those steps/jumps occur). See also AI Takeoff and Strategic considerations about different speeds of AI takeoff.

I think there are basically four key reasons why more gradual development could be good:

• “Fire alarms” and “warning shots”: More gradual development provides more chances for various actors to notice that the risky technology is coming, is coming soon, and could be very risky. That would probably increase how many resources end up getting allocated to various ways of reducing the risks (e.g., developing risk-reducing technologies or improving governance mechanisms).

• More time with more clarity: Roughly speaking, if the technology jumps suddenly to a very risky point, almost all risk-reducing work has to be done with quite little clarity on what the technology will ultimately look like, how it will be used, who’ll develop it, who’ll govern it, etc. The more gradual the development is, the more time we’ll have before the especially risky period but with a decent picture of how things will ultimately end up.^[1]

• Ordering: More gradual development also just makes it more likely that the technology reaches a dangerously mature stage after specific other risk-reducing developments occur, which could be good for reasons discussed in the previous section.

• Distance from now: More gradual development also just makes it more likely that the technology reaches a dangerously mature stage longer from now, which could be good for reasons discussed in the previous section.

For gradualness, as with ordering, absolute distance from now doesn’t matter in itself, but rather could serve as a somewhat useful proxy - that is, a technology being developed further in the future could serve as some evidence that it will be developed more gradually. (Though the opposite can also be true - e.g. if AI software improvements happen further in the future, that could increase the chance that there’s a large hardware overhang at that point, which could increase takeoff speeds.)

The more we care about gradualness, the more we might be interested in:^[2]

• Improving our forecasts of how gradually the development will occur
• Reducing our need for fire alarms, warning shots, or more time with more clarity. For example, we could:
  • Convince more people that development may not be gradual and that they should therefore ramp up risk-reducing work now
  • Improve our insight now into how the development and deployment will play out
• Perhaps trying to increase that chance that, when some jumps in progress occur or appear to be on the horizon, development then slows to a more gradual pace. For example, we could:
  • Highlight the importance of gradualness to people involved in doing, funding, regulating, etc. the development, and suggest they slow down after certain key breakthroughs occur
  • Make it more likely that the jumps will actually be noticed by relevant actors (e.g., increasing the transparency of the relevant development processes)

Distance from now

Finally, we might think it would be better if the technology were developed a longer time from now, for reasons unrelated to how gradually it’s developed or whether it’s developed before/after specific risk-reducing developments occur. I think the main reason for this is if various risk-reducing efforts are already occurring or are expected to occur in future by default (not just in response to initial steps of a gradual development progress), such that extending timelines would “buy more time” and mean that more such work occurs by the “deadline”.^[3]

There are two ways this is distinct from “ordering”:

• As I defined “ordering”, it was just about the order of technological developments, which most people would interpret as not including things like the emergence of norms and governance mechanisms that can mitigate risks from the technology. In contrast, “distance from now” also “buys time” for things like norms and governance mechanisms.
• “Ordering” is about the order in which a specific set of developments we already have in mind occur, whereas “distance from now” buys time for a broad range of possible risk-reducing efforts. So more “distance from now” offers a less targeted but also less brittle protection than a better ordering of specifically identified events does.

The more we care about distance from now, the more we might be interested in:

• Improving our forecasts of when the technological development(s) will occur
• Improving our estimates of how much work is currently occurring and how much will occur in various future years (perhaps given that there are no major advancements in the risky technology), to gain a clearer sense of how much we’d really gain by “buying more time”
• Slowing overall progress towards the risky technological development(s)
• Ramping up risk-reducing efforts now or in future, without relying on things like fire alarms or warning shots to cause this

My thanks to Neil Dullaghan, Ben Snodin, and James Wagstaff for helpful comments on an earlier draft.