I define “alignment” as “the AI is trying to do things that the AI designer had intended for the AI to be trying to do”, see here for discussion.

If you define “capabilities” as “anything that would make an AI more useful / desirable to a person or company”, then alignment research would be by definition a subset of capabilities research.

But it’s a very small subset!

Examples of things that constitute capabilities progress but not alignment progress include: faster and better and more and cheaper chips (and other related hardware like interconnects), the development of CUDA, PyTorch, etc., the invention of BatchNorm and Xavier initialization and adam optimizers and Transformers, etc. etc.

Here’s a concrete example. Suppose future AGI winds up working somewhat like human brain within-lifetime learning (which I claim is in the broad category of model-based reinforcement learning (RL).) A key ingredient in model-based RL is the reward function, which in the human brain case loosely corresponds to “innate drives”, like pain being bad (other things equal), eating when hungry being good, and hundreds more things like that. If future AI works like that, then future AI programmers can put whatever innate drives they want into their future AIs. So there’s a technical problem of “what innate drives / reward function (if any) would lead to AIs that are honest, cooperative, kind, etc.?” And this problem is not only currently unsolved, but almost nobody is working on it. Is solving this problem necessary to create economically-useful powerful AGIs? Unfortunately, it is not!! Just look at human high-functioning sociopaths. If we made AGIs like that, we could get extremely competent agents—agents that can make and execute complicated plans, figure things out, do science, invent tools to solve their problems, etc.—with none of the machinery that gives humans an intrinsic tendency to compassion and morality. Such AGIs would nevertheless be very profitable to use … for exactly as long as they can be successfully prevented from breaking free and pursuing their own interests, in which case we’re in big trouble. (By analogy, human slaves are likewise not “aligned” with their masters but still economically useful.) I have much more discussion and elaboration of all this stuff here.

I mostly think of alignment as about avoiding deception or catastrophic misgeneralization outside of testing settings.

In general I also believe that AI companies have a massive incentive to align their systems with user intent. You can't profit if you are dead.

Dan Hendrycks' lecture on "Safety-Capabilities Balance" might be helpful here.

When people distinguish between alignment and capabilities, I think they’re often interested in the question of what research is good vs. bad for humanity. Alignment vs. capabilities seems insufficient to answer that more important question.  Here’s my attempt at a better distinction:

There are many different risks from AI. Research can reduce some risks while exacerbating others. "Safety" and "capabilities" are therefore incorrectly reductive. Research should be assessed by its distinct impacts on many different risks and benefits. If a research direction is better for humanity than most other research directions, then perhaps we should award it the high-status title of "safety research." 

Scalable oversight is a great example. It provides more accurate feedback to AI systems, reducing the risk that AIs will pursue objectives that conflict with human goals because their feedback has been inaccurate. But it also makes AI systems more commercially viable, shortening timelines and perhaps hastening the onset of other risks, such as misuse, arms races, or deceptive alignment. The cost-benefit calculation is quite complicated. 

"Alignment" can be a red herring in these discussions, as misalignment is far from the only way that AI can lead to catastrophe or extinction. 

I think you missed out:

(3) many techniques that are supposed to improve capabilities also improve alignment. 

Take OpenAI's Superalignment approach. It involves "building a roughly human-level automated alignment researcher" then "using vast amounts of compute to scale efforts, and iteratively align superintelligence".

AI capabilities is central to the alignment approach because us humans are way too limited to achieve alignment ourselves.

I agree with Jaime's answer about how alignment should avoid deception. (Catastrophic misgeneralization seems like it could fall under your alignment as capabilities argument.)

I sometimes think of alignment as something like "aligned with universal human values" more than "aligned with the specific goal of the human who programmed this model". One might argue there aren't a ton of universal human values. Which is correct! I'm thinking very basic stuff like, "I value there being enough breathable oxygen to support human life".