Like many people, I have quite a lot of stuff that I've written that exists only in googledoc form. In writing a comment response to Wei Dai, I wanted to point to a concept that I realised I hadn't published. So I thought I should practice what I preach, and put the idea out as a post. But warning — the level of polish is likely a lot lower than other things I might write.
Lock-in escape velocity
One dynamic that makes long-term path-dependence more likely to occur in the near future is lock-in escape velocity: that short periods of entrenchment can turn into longer periods of entrenchment.
Suppose that a one-world government is formed, and the leaders of that government are able to entrench their power for a comparatively short period of time, so they very probably stay in power for 10 years. But, in that time, they are able to make it very likely they can stay in power for a further 20 years. Then, in that 20 years, they can develop the means to make it very likely to maintain power for a further 40 years… and so on. Even though initially, the political leaders were only able to entrench their power for a short period, they could turn that short-term entrenchment into indefinite lock-in; they achieved lock-in escape velocity.
Similarly, shallow entrenchment can turn into deeper entrenchment. For example, some group could initially merely ensure that they are in power, and only later start to lock in specific laws or values. Or the whole world could initially commit only to some minimal set of norms; but those minimal norms could inexorably lead to more thoroughgoing lock-in over time. Various forms of bad lock-in could happen gradually, without anyone initially having some grand long-term plan.
The "point of no return" for society going down one particular path, then, might come well before there exist full-fledged mechanisms for indefinite persistence. Given how likely AGI is to come in the next few years or decades, it seems likely to me that it's currently possible to achieve lock-in escape velocity.
Bootstrapping to viatopia
But there's a positive version of the same dynamic. We could bootstrap our way toward viatopia — that is, a society that is on-track to achieve some near-best outcome, whatever that may be.
The key insight is this: we can implement some change to society that gives us enough breathing room to reflect. During that time to reflect we can, if we need to, decide take a little more time to reflect further, and so on.
For example, suppose that, as the intelligence explosion kicks off, whichever country leads in AI decides to implement a one-month pause on frontier AI development. A one-month pause is a far cry from the decades-long bans that some people are asking for. But a one-month pause could turn into something much longer.
By the end of that month, the relevant decision-makers might reasonably decide that they need more time to figure this out; so they decide to extend the pause by another month. At the end of that month, it might become clear that it would be valuable to extend the pause further by a few months. And so on.
A one-month pause seems much more feasible to achieve, off the bat, than a decades-long pause. The leading country could plausibly implement it unilaterally while still maintaining their lead. But it could create enough breathing room to ultimately result in something much longer.
A further benefit of this approach (vs committing to some longer pause right now) is that the decision of how long to pause for is made with all the relevant context available at the time, and with the help of much more advanced AI advice than we have access to now.
The example I've given was about pausing AI development. But the same bootstrapping dynamic could hold more generally, too.
For example, suppose that once aligned human-level AI has been achieved, whichever country leads in AI convenes a global constitutional convention. It invites representatives from all countries of the world. They have one month to write a constitution to govern a post-AGI world (e.g. how are space resources allocated; what rights do AIs have, etc). In the course of that month, they could choose to implement a temporary constitution that lasts only a year until they renegotiate. And so on. In such a situation, the deliberation process would take as long, but only as long, as it needs to. Perhaps, in order to make sufficiently responsible decisions, we really do need thousands of years (because human decision-makers want to go through the full reflective process themselves, and aren't willing to defer to AI advisors). Or perhaps we could be done over the course of just months.
The idea of a one-month global constitutional convention is radical, but it's not completely outside the bounds of possibility in the way that a centuries-long deliberative pause is. And, if well-designed, it could be enough to get us all the way to a reflection period that's as long as it needs to be.
The idea of bootstrapping needn't be just about time, but also about depth or type of reflection. For example, by getting the most-important decision-makers to reflect just a little bit, they might realise, in the course of that little bit of reflection, that really they should reflect much more; once having crossed a minimal threshold, they see the need to reflect as much as possible.
On any of these ideas, the key thought is this. It’s currently at best unclear whether society has the collective wisdom to pause and reflect sufficiently hard on decisions with far-reaching consequences. But perhaps we can do things to get society to pause for a little bit, or reflect a little bit harder. And perhaps that society would be wise enough to know to pause or reflect at least a little bit longer. And from there we could slowly work ourselves up to an outcome where society is genuinely thinking through the most important decisions we’ll ever face for as long and hard as we should.
Nice I like/agree with this.
I am imagining how more accelerationist/techno-libertarian people will perceive this though, and think we might have a rhetorical problem (insofar as those interest groups will be powerful). What to us is maintaining useful optionality by pausing briefly, with the option to extend, to them may seem like a slippery slope towards an indefinite/very long pause. That is, by foregrounding the fact that a short pause may lead to a far longer pause, we might make it harder to build a coalition around a short pause.
Note: I don't think this necessarily means we should obscure this bootstrappingidea, as being honest and high-integrity is important. But it does seem like a dynamic worth tracking.