On point 3: I think essentially everyone talking about pivotal acts is envisioning a world like the present one, where they are concerned about x-risk from AGI but most other organizations aren't. What good does it do someone in that position to say "well, this is the UN's job" if they have no way to convince the UN to care?
I think people who expect e.g. the UN or "humanity" to be persuadable to do something are not generally in favor of unilateral pivotal acts as a concept for exactly the reasons you highlight, but IMO this is the biggest crux.
"Humanity should do X" is not a plan.
EDIT: You address this in point 2, but I think people aren't engaging in a fallacy, they're just much more pessimistic about the prospects of gathering evidence and convincing global institutions to take action.
I hasten to add: I agree with you about the downsides of advocating for aggressive unilateral actions.
Also, if people are actively advocating for arbitrary AI labs (that they have little influence over) taking unilateral actions like this, that seems like potentially the worst of both worlds: harmful to even talk about, without being especially actionable.
On point 3: I think essentially everyone talking about pivotal acts is envisioning a world like the present one, where they are concerned about x-risk from AGI but most other organizations aren't. What good does it do someone in that position to say "well, this is the UN's job" if they have no way to convince the UN to care?
I think people who expect e.g. the UN or "humanity" to be persuadable to do something are not generally in favor of unilateral pivotal acts as a concept for exactly the reasons you highlight, but IMO this is the biggest crux.
"Humanity should do X" is not a plan.
EDIT: You address this in point 2, but I think people aren't engaging in a fallacy, they're just much more pessimistic about the prospects of gathering evidence and convincing global institutions to take action.
I hasten to add: I agree with you about the downsides of advocating for aggressive unilateral actions.
Also, if people are actively advocating for arbitrary AI labs (that they have little influence over) taking unilateral actions like this, that seems like potentially the worst of both worlds: harmful to even talk about, without being especially actionable.