This is a special post for quick takes by alesziegler. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Is the recent partial lifting of US chip export controls on China (see e.g. here: https://thezvi.substack.com/p/selling-h200s-to-china-is-unwise) good or bad for humanity? I’ve seen many takes from people whose judgment I respect arguing that it is very bad, but their arguments, imho, just don’t make sense. What am I missing?
For transparency, I am neither Chinese nor American, nor am I a paid agent of them. I am not at all confident in this take, but imho someone should make it.
I see two possible scenarios: A) you are not sure how close humanity is to developing superintelligence in the Yudkowskian sense. This is what I believe, and what many smart opponents of the Trump administration’s move to ease chip controls believe. Or B) you are pretty sure that humanity is not going to develop superintelligence any time soon, let’s say in the next century. I admit that the case against the lifting of chip controls is stronger under B), though I am ultimately inclined to reject it in both scenarios.
Why is easing of chip controls, imho, a good idea if the timeline to superintelligence might be short?
If superintelligence is around the corner, here is what should be done: an immediate international pause of AI development until we figure out how to proceed.
Competitive pressures and resulting prisoner’s dilemmas have been identified as the factor that might push us toward NOT pausing even when it would be widely recognized that the likely outcome of continuing is dire.
There are various relevant forms of competition, but plausibly the most important is that between the US and China. In order to reduce competitive dynamics and thus prepare the ground for a cooperative pause, it is important to build trust between the parties and beware of steps that are hostile, especially in domains touching AI.
Controls make sense only if you are very confident that superintelligence developed in the US, or perhaps in liberal democracy more generally, is going to turn out well for humanity, and you are not so confident about superintelligence developed in China.
I do think that the US political system is, in an important sense, whose exposition would require more than an oversized quick take, better than the Chinese political system, but despite this, I am not at all confident that superintelligence developed in a liberal democracy would be good for humanity; frankly, I find such beliefs wholly unfounded.
Why is easing of chip controls a good idea if the timeline to superintelligence is long?
Disclaimer: I know following argument is perhaps too abstract and I am even less confident in it than in previous section, nevertheless.
It is a feature of the current international order that there is a strong taboo on military action of one country against another outside narrow and contested exceptions. I think this taboo is very good and very important. And yes, I know that this taboo is sometimes broken; that does not invalidate previous two sentences.
Another feature of the current international order is that some countries are what is sometimes called pariah states; I am vaguely aware that the word “pariah” has unpleasant connotations in the Indian cultural context, but I am going to use the term anyway, since I don’t have a suitable replacement, sorry. Anyway, good examples are, e.g., Russia or Afghanistan.
Being a pariah state means essentially that some hostile actions which are considered beyond the pale against a non-pariah state are considered ok against them. Notably, the taboo against direct military action is so strong that it mostly applies even to attacks against pariah states, but certain types of heavy restrictions on trade and cross-border investment are what essentially define pariah status. Another sometimes used action against a pariah state is external support, including by providing arms, to an internal opposition, but that is facultative. Many pariah states don’t have internal opposition capable of or interested in armed resistance.
In trade, pariah status tended to be a package deal. If a country is a pariah, it usually had heavy restrictions placed on its trade and investment; if it is a normal member of the international community, it could expect that it would not have certain kinds of restrictions placed on it.
Of course, the previous paragraph has to be immediately qualified in that there are many, many types of restrictions on international trade considered completely acceptable even among normal members of the international community, such as tariffs, bans on investment in critical infrastructure, or bans on selling some military or military-adjacent technologies. But there are other kinds of restraints that were usually deployed only against pariahs.
And pariah status is not a category enshrined in international law; really, making and unmaking a pariah is based on a decision of the government of the United States, since the US is the current hegemon of the international order. E.g., Cuba is, imho, a pariah state mainly because of random historical contingency plus quirks of the US electoral system. BUT breaking a taboo against militarily attacking another country is the sort of thing that tends to get you on the pariah list.
This all means that avoiding a pariah designation is a strong incentive for countries NOT to engage in military aggression.
Now, I do think that the chip controls imposed on China by the Biden administration are breaking this logic. These controls are restrictions on selling a critical component of consumer technology which is related to military stuff only in the sense that, like, steel or oil is related to military stuff. And restrictions on selling those is something which has not usually been done against non-pariah states; the Biden administration, however, had no plan to impose a whole package of pariah status on China. They made an awkward exception in an important area, which devalues the status of being non-pariah member of international community.
Thus, the imposition of chip controls reduces the incentive for China, and indirectly for everyone else, to avoid pariah status. Which means that it reduces their incentive to avoid military aggression. I think that that is very bad and it outweighs benefits of those controls even in no-near-term-superintelligence-threat world.
Is the recent partial lifting of US chip export controls on China (see e.g. here: https://thezvi.substack.com/p/selling-h200s-to-china-is-unwise) good or bad for humanity? I’ve seen many takes from people whose judgment I respect arguing that it is very bad, but their arguments, imho, just don’t make sense. What am I missing?
For transparency, I am neither Chinese nor American, nor am I a paid agent of them. I am not at all confident in this take, but imho someone should make it.
I see two possible scenarios: A) you are not sure how close humanity is to developing superintelligence in the Yudkowskian sense. This is what I believe, and what many smart opponents of the Trump administration’s move to ease chip controls believe. Or B) you are pretty sure that humanity is not going to develop superintelligence any time soon, let’s say in the next century. I admit that the case against the lifting of chip controls is stronger under B), though I am ultimately inclined to reject it in both scenarios.
Why is easing of chip controls, imho, a good idea if the timeline to superintelligence might be short?
If superintelligence is around the corner, here is what should be done: an immediate international pause of AI development until we figure out how to proceed.
Competitive pressures and resulting prisoner’s dilemmas have been identified as the factor that might push us toward NOT pausing even when it would be widely recognized that the likely outcome of continuing is dire.
There are various relevant forms of competition, but plausibly the most important is that between the US and China. In order to reduce competitive dynamics and thus prepare the ground for a cooperative pause, it is important to build trust between the parties and beware of steps that are hostile, especially in domains touching AI.
Controls make sense only if you are very confident that superintelligence developed in the US, or perhaps in liberal democracy more generally, is going to turn out well for humanity, and you are not so confident about superintelligence developed in China.
I do think that the US political system is, in an important sense, whose exposition would require more than an oversized quick take, better than the Chinese political system, but despite this, I am not at all confident that superintelligence developed in a liberal democracy would be good for humanity; frankly, I find such beliefs wholly unfounded.
Why is easing of chip controls a good idea if the timeline to superintelligence is long?
Disclaimer: I know following argument is perhaps too abstract and I am even less confident in it than in previous section, nevertheless.
It is a feature of the current international order that there is a strong taboo on military action of one country against another outside narrow and contested exceptions. I think this taboo is very good and very important. And yes, I know that this taboo is sometimes broken; that does not invalidate previous two sentences.
Another feature of the current international order is that some countries are what is sometimes called pariah states; I am vaguely aware that the word “pariah” has unpleasant connotations in the Indian cultural context, but I am going to use the term anyway, since I don’t have a suitable replacement, sorry. Anyway, good examples are, e.g., Russia or Afghanistan.
Being a pariah state means essentially that some hostile actions which are considered beyond the pale against a non-pariah state are considered ok against them. Notably, the taboo against direct military action is so strong that it mostly applies even to attacks against pariah states, but certain types of heavy restrictions on trade and cross-border investment are what essentially define pariah status. Another sometimes used action against a pariah state is external support, including by providing arms, to an internal opposition, but that is facultative. Many pariah states don’t have internal opposition capable of or interested in armed resistance.
In trade, pariah status tended to be a package deal. If a country is a pariah, it usually had heavy restrictions placed on its trade and investment; if it is a normal member of the international community, it could expect that it would not have certain kinds of restrictions placed on it.
Of course, the previous paragraph has to be immediately qualified in that there are many, many types of restrictions on international trade considered completely acceptable even among normal members of the international community, such as tariffs, bans on investment in critical infrastructure, or bans on selling some military or military-adjacent technologies. But there are other kinds of restraints that were usually deployed only against pariahs.
And pariah status is not a category enshrined in international law; really, making and unmaking a pariah is based on a decision of the government of the United States, since the US is the current hegemon of the international order. E.g., Cuba is, imho, a pariah state mainly because of random historical contingency plus quirks of the US electoral system. BUT breaking a taboo against militarily attacking another country is the sort of thing that tends to get you on the pariah list.
This all means that avoiding a pariah designation is a strong incentive for countries NOT to engage in military aggression.
Now, I do think that the chip controls imposed on China by the Biden administration are breaking this logic. These controls are restrictions on selling a critical component of consumer technology which is related to military stuff only in the sense that, like, steel or oil is related to military stuff. And restrictions on selling those is something which has not usually been done against non-pariah states; the Biden administration, however, had no plan to impose a whole package of pariah status on China. They made an awkward exception in an important area, which devalues the status of being non-pariah member of international community.
Thus, the imposition of chip controls reduces the incentive for China, and indirectly for everyone else, to avoid pariah status. Which means that it reduces their incentive to avoid military aggression. I think that that is very bad and it outweighs benefits of those controls even in no-near-term-superintelligence-threat world.