Comments
More from the author
Curated and popular this week
This post is co-authored with Ben Garfinkel. It is cross-posted from the CEA blog. A PDF version can be found here.
Summary: Some strategic decisions available to the effective altruism m...
Disclaimer: Although I work on the Groups Team at CEA, I’m writing this in a personal capacity, and this post does not constitute an endorsement by CEA.
Agency - the realisation that you really can just do things.
TL;DR
Biosecurity needs people (of any background) who are agentic and have a high execution velocity and track record....
44
Summary
Making AI safe and good requires many people to be ready to address crucial problems. However, our current “pipelines” for finding, training, and accelerating talented individuals who could work on these problems are “leaky”, meaning there are many ways they can be improved. This post shows my full overview of the AI safety talent pipeline: where people come from, where they go to learn more, and why this is impactful. I also describe some “leaks” in the pipeline: stages that...
Recent opportunities to take action
I agree that Pinker's chapter on existential risk is an extremely bad interlude in an otherwise excellent book. But I think it would be best if there were a temperate tight critique that dealt with the most egregious mistakes therein in a cautious way that could be presented to the otherwise EA-sympathetic Pinker. I would suggest just meeting his arguments with lots of examples of experts who disagree with him on the crucial points about nuclear, AI and biorisk. Informing him that he has drastically misrepresented Stuart Russell would be a start.
This draft doesn't really fit the bill and takes more of a sprawling scattergun approach with too much extraneous argument and information.
Having not read the book, I'm curious why you think this chapter is exceptional rather than perhaps revealing a problem with the other chapters you didn't notice, perhaps because you're not so close to the topic that you can notice the way Pinker makes mistakes?
I have a pretty good knowledge of some of the other issues talked about in the book, including climate change, resource depletion, population, trends in living standards over time, and I thought he was on point or nearly on point with all of them. He just goes completely off the rails in the ex risk bit
Sidenote: The general phenomenon you mentioned is sometimes referred to as the Gell-Mann amnesia effect.
I agree that this isn't the ideal format for engaging with someone like Pinker, who seems capable of recognizing his mistakes without needing them spelled out to quite this degree.*
Based on my experience as a journalist, conversations I've had with journalists, and materials I've read from journalists and other writers (essays, interviews, etc.), there's often a lot of pressure, when publishing for a broad audience, to make a strong argument with few qualifications. It's possible that Pinker gradually removed nuance from this chapter during the editing process. It's also possible that he decided to write something forcefully one-sided to counteract what he sees as a heavily skewed public conversation.
(Finally, I'll note that it's probably really difficult to write a nuanced 500-page book about twenty different topics on a publisher's deadline when you are also a famous professor with a number of other commitments.)
Even if none of my suppositions are true, I still think he'd be open to a more balanced discussion of X-risk if it started out on the right foot. Maybe Rob Wiblin could have him on the 80,000 Hours podcast? Enlightenment Now wasn't perfect, even outside this chapter, but I think Pinker is (mostly) on our side.
----
*That said, while it isn't charitable or warm enough to work as public communication, I do think it's an excellent essay about anti-X-risk fallacies in general. I especially appreciate Torres' efforts to track down original sources for quotes that were taken out of context. People who aren't as well-intentioned as Steven Pinker will make the same arguments for decades to come, and it seems good to have a collection of strong, well-sourced counterarguments.
IMO there is a room for both: temperate tight critique that would deal with the most egregious mistakes for general audience (and possibly Pinker), and more detailed critique for those interested in the topic in more depth. Possibly someone with good public communication skill can use the later as a source to write the former.