A natural way of tagging AI-related content on the EA Forum might be something like:

Discussion of the sort of AI that existential risk EAs are worried about.
Discussion of other sorts of AI.

And within 1:

1a. Technical work aimed at increasing the probability that well-intentioned developers can reliably produce good outcomes from category-1 AI systems.
1b. Attempts to forecast AI progress as it bears on category-1 AI systems.
1c. Attempts to answer macrostrategy questions about such AI systems: How should they be used? What kind of group(s) should develop them? How do we ensure that developers are informed, responsible, and/or well-intentioned? Etc.

Plausibly 1b and 1c should just be one tag (which can then link to multiple different daughter wiki articles explaining different subtopics), since there's lots of overlap and keeping the number of tags small makes it easier to find articles you're looking for and remember what the tags are.

(There may also be no need to make 1 its own category, if everything falls under at least one of 1a/1b/1c anyway. But maybe some things will be meta enough to benefit from a supertag -- e.g., discussions of the orgs working on AI x-risk.)

If those are good categories, then the next question is what to name them. Some established options for category 1 (roughly in increasing order of how much I like them for this purpose):

Superintelligent AI - Defined by Bostrom as AI "that is much smarter than the best human brains
in practically every field, including scientific creativity, general wisdom and
social skills". This seems overly specific: AI might destroy the world with superhuman science and engineering skills even if it lacks superhuman social skills, for example.
Transformative AI - Defined by Open Phil as "AI that precipitates a transition comparable to (or more significant than) the agricultural or industrial revolution". I think this is too vague, and wouldn't help people discussing Bostrom/Christiano/etc. doomsday scenarios find each other on the forum. E.g., Logan Zoellner wonders whether existing AI is already "transformative" in this sense; whatever the answer, it seems like a question that's tangential to the kinds of considerations x-risk folks mostly care about.
Advanced AI - A vague term that can variously mean any of the terms on this list. Its main advantage is that its lack of a clear definition would let the EA Forum stipulate some definition just for the sake of the wiki and tagging systems.
Artificial general intelligence - AI that can do the same sort of general reasoning about messy physical environments that allowed humans to land on the Moon and build particle accelerators (even though those are very different tasks, neither capability was directly selected for in our ancestral environment, and chimpanzees can't do either). This seems like a good option relative to my own way of thinking about AI x-risk. The main disadvantage of the term (for this use case) is that some thinkers worried about AI x-risk are more skeptical that "general intelligence" is a natural or otherwise useful category. A slightly more theory-neutral term might be better for a tag or overview article.
Prepotent AI - A new term defined by Andrew Critch and David Krueger to mean AI whose deployment "would transform the state of humanity’s habitat—currently the Earth—in a manner that is at least as impactful as humanity and unstoppable to humanity". This seems to nicely subsume both the slower-takeoff Christiano doomsday scenarios and the faster-takeoff Bostromite doomsday scenarios, without weighing in on whether "AGI" is a good category.

Category 1 could then be called "Non-prepotent AI" or "Narrow AI" or similar.

I haven't used the term "prepotent AI" much, so it might have issues that I'm not tracking. But if so, giving it a test run on the EA Forum might be a good way to reveal such issues.

I think the best term for 1a is AI alignment, with the wrinkle that most researchers are focused on "intent alignment" (getting the AI to try to produce good outcomes), and Paul Christiano thinks it would be more natural to define the field's goal as intent alignment, while some other researchers want to include topics like boxing in 'AI alignment' (I've called this larger category "outcome alignment" to disambiguate).

For 1b+1c, I like AI strategy and forecasting. The term "AI governance" is popular, but seems too narrow (and potentially alienating or confusing) to me. You could also maybe call it 'Prepotent AI strategy and forecasting' or 'AGI strategy and forecasting' to clarify that we aren't talking about the strategic implications of using existing AI tech to augment anti-malaria efforts or what-have-you.

		EA Wiki assistant	v1.1.0May 4th 2021	(+63)
		Pablo	v1.0.0Dec 22nd 2020	(+80)

AI safety

Related entries