Comments
More from the author
Curated and popular this week
125
Epistemic status: Speculation from two decently informed advocates armed with anecdata.
Note on process: After having some version of this conversation several times and saying, “we should probably write about this publicly,” we took the less heroic route: we recorded one of our conversations, fed the transcript into an LLM, and then substantially revised the structure, substance, and framing ourselves. We will not be sharing the transcript, as it is in...
100
TLDR: Take the population ethics quiz here: https://mdickens.me/pop-ethics/
Population ethics is an oft-overlooked subfield within ethics. Many people hold views that they don't realize contradict each other, or that have strange implications that they wouldn't endorse if they thought about it more.
Not just that—population ethics is a BIG DEAL. A lot of ethical decisions hinge on how you think about changes in future populations....
120
Headline finding: I audited 17 AI Safety Talent programmes. Zero of 17 have published any comparison group, rejected-applicant follow-up, matched control or randomisation. Not one. Every programme that mentions a counterfactual does it by asking participants to self-report.
Background
At least $70 million...
Recent opportunities to take action
Thanks for sharing this.
What’s your view on “why doesn’t OpenAI slow down work on capabilities to allow more time for alignment research?”
Is it
OpenAI is worried that Deepmind might build AGI before OpenAI, and OpenAI is worried that Deepmind is more likely to build AGI unsafely or misuse it
OpenAI is worried that another group we aren’t aware of yet might build AGI before OpenAI and Deepmind, and OpenAI is worried that this group is more likely to build AGI unsafely or misuse it
OpenAI is already only scaling up capabilities where it allows them to do better alignment research
OpenAI is in practice prioritising revenue over existential risk mitigation, despite claiming not to
something else / a mix
I think that OpenAI is not worried about actors like DeepMind misusing AGI, but (a) is worried about actors that might not currently be on most people's radar misusing AGI, (b) thinks that scaling up capabilities enables better alignment research (but sees other benefits to scaling up capabilities too) and (c) is earning revenue for reasons other than direct existential risk reduction where it does not see a conflict in doing so.