I'm linking to an lc post on LessWrong about why I'm generally skeptical of greater good arguments that presuppose that we shouldn't fix AI problems prosaically because we should let warning shots occur to get an AI pause.
There are some differences from the regular version, in particular there's a lot of people on LW who do genuinely believe that prosaic alignment is ineffective or harmful, but I still basically agree with lc's argument on the topic here.
Choice quotes below:
An execrpt on criticisms of EA:
Back in the before-time of the 2010s, when EA was still known mostly for funding health efforts in the developing world, there were many baskets of criticism floating around from the left. The most prominent was a cluster centered around EAs being white men, and therefore suspicious. Another category, in my experience the second-most common, was that in attempting to rectify the problems in the world, EA was masking, whitewashing, and misdirecting efforts away from solving the "real problem", usually capitalism but often other contradictory forces.
These arguments were often not specific to EA, but mainstream arguments against charitable giving in general. On any Hacker News post about a billionaire giving to charity, one of the first few comments would inevitably be this Oscar Wilde quote:
Just as the worst slave-owners were those who were kind to their slaves, and so prevented the horror of the system being realized by those who suffered from it, and understood by those who contemplated it, so, in the present state of things in England, the people who do most harm are the people who try to do most good.
These arguments have a long history. Anticapitalist critiques of charity go back to Engels and Marx, who called charitable givers a bourgeois faction intent on "redressing social grievances in order to secure the continued existence of bourgeois society." In their most explicit form these critiques had the following shape:
- There is a "root cause" of most societal ills, such as capitalism.
- Billionaires giving to deworming efforts addresses the symptoms of this problem, such as poverty and disease. But it's not treating the root cause.
- Additionally, a side effect of such giving is that it gives people reason to tolerate the current system better, or even begin to like their oppressors, which gets in the way of fomenting change.
- So actually, deworming people in Africa, or giving legal aid to poor people in the U.S., is net-negative. You should just spend all of your time trying to change capitalism (or democracy, etc.)
An excerpt of a repeat around prosaic AI safety:
Recently, many people on LessWrong have made structurally similar arguments about prosaic AI safety work. Here are two such shortforms with >100 karma presenting this argument, though it has been raised in many other places. It generally goes like this:
- Aligning superintelligence is very difficult.
- Prosaic AI safety measures, such as chain-of-thought monitoring for broken RL environments, address the symptoms of misalignment but don't solve the underlying problem, or at least don't scale to superintelligence.
- Additionally, getting better at aligning current AIs means that there are fewer containment breaches, and so there is less support for the best possible solution, an AI pause.
- So actually, you should not do chain-of-thought monitoring. Ideally you should spend all of your time trying to Pause AI, but if you don't want to do that you should do nothing.
And some points about why the argument above is likely wrong:
- We are rapidly approaching AIs that are sufficient for bootstrapping recursive self-improvement. Current AIs inside companies like Anthropic already have a lot of say in designing their successors, so "mundane alignment of present-day AIs" is affecting the safety of future models.
- Raising the ceiling for the most capable model we can use safely is good. It delays AI takeover, and therefore allows us to develop new prosaic safety techniques, or refine the ones we have. Techniques need not "scale to superintelligence" in order to be helpful in this way.
- AI progress is not currently bottlenecked on alignment. This seems like something that alignment people say copefully but is wrong. The two things that matter for AI progress are frontier labs' data efficiency and compute efficiency, two highly quantifiable metrics that can be improved even with models that don't care a whole lot about people or generalize well to other tasks.
- Further, the entire paradigm of asking whether a particular technique "scales to superintelligence" seems wrong and confused to me, somewhat akin to asking if hashing passwords "scales to APTs". The question to ask is not whether a technique is sufficient to contain superintelligence, but whether it could help guard against correlated failures in other components. Derisively call this a "grab bag" approach if you want, but that phrase can be equally used to describe many useful practices in fields like information security.
