Compiled by Markus Anderljung and Alexis Carlier
Junior researchers are often wondering what they should work on. To potentially help, we asked people at the Centre for the Governance of AI for research ideas related to longtermist AI governance. The compiled ideas are developed to varying degrees, including not just questions, but also some concrete research approaches, arguments, and thoughts on why the questions matter. They differ in scope: while some could be explored over a few months, others could be a productive use of a PhD or several years of research.
We do not make strong claims about these questions, e.g. that they are the absolute top priority at current margins. Each idea only represents the views of the person who wrote it. The ideas aren’t necessarily original. Where we think someone is already working on or has done thinking about the topic before, we've tried to point to them in the text and reach out to them before publishing this post.
If you are interested in pusuing any of the ideas, feel free to reach out to contact@governance.ai. We may be able to help you find mentorship, advice, or collaborators. You can also reach out if you’re intending to work on the project independently, so that we can help avoid duplication of effort.
You can find the ideas here. Our colleagues at the FHI AI Safety team put together a corresponding post with AI safety research project suggestions here.
Other Sources
Other sources of AI governance research projects include:
- AI Governance: A Research Agenda, Allan Dafoe
- Research questions that could have a big social impact, organised by discipline, 80,000 Hours
- The section on AI in Legal Priorities Research: A Research Agenda, Legal Priorities Project
- Some parts of A research agenda for the Global Priorities Institute, Global Priorities Institute
- AI Impact’s list of Promising Research Projects
- Phil Trammell and Anton Korinek's Economic Growth under Transformative AI
- Luke Muehlhauser's 2014 How to study superintelligence strategy
- You can also look for mentions of possible extensions in papers you find compelling
A list of the ideas in the document:
- The Impact of US Nuclear Strategists in the early Cold War
- Transformative AI and the Challenge of Inequality
- Human-Machine Failing
- Will there be a California Effect for AI?
- Nuclear Safety in China
- History of existential risk concerns around nanotechnology
- Broader impact statements: Learning lessons from their introduction and evolution
- Structuring access to AI capabilities: lessons from synthetic biology
- Bubbles, Winters, and AI
- Lessons from Self-Governance Mechanisms in AI
- How does government intervention and corporate self-governance relate?
- Summary and analysis of “common memes” about AI, in different communities
- A Review of Strategic-Trade Theory
- Mind reading technology
- Compute Governance ideas
- Compute Funds
- Compute Providers as a Node of AI Governance
- China’s access to cutting edge chips
- Compute Provider Actor Analysis
Thanks for compiling this! I like the way you've approached this (e.g., the google form you've linked to), and most of these ideas do seem interesting and important to me.
I'd be interested to hear later on how many people filled in the form and how much good stuff came from that.
Also, I've now added this to my central directory for open research questions (in the section for non-technical AI focused stuff), to hopefully make it a little more likely people looking for research projects in future can find it.
Thanks Michael! Yeah, I hope it ends up being helpful.
Thanks, I enjoyed reading the ideas and get a rough overview about what you think is interesting and useful.
One part I stumbled upon and would be interested in thoughts:
I was wondering if it makes sense to frame this as „social responsibility“, especially when it is about radical technological change. Social responsibility is already used for companies to do additional socially beneficial things, and doesn‘t communicate how radical the impact might be and how much it will shape our future and how much care/security mindset might be required if say Google would be first do develop transformative AI systems.
Thanks! I agree that using a term like "socially beneficial" might be better. On the other hand, it might be helpful to couch self-governance proposals in terms of corporate social responsibility, as it is a term already in wide use.
Yeah. What I thought is that one might want to somehow use a term that also emphasizes the potentially transformative impact AI companies will have, as in „We think your AI research might fit into the reference class of the Manhattan Project“. And „socially beneficial“ doesn‘t really capture this either for me. Maybe something in the direction of „risk-aware“, „risk-sensitive“, „farsighted“, „robustly beneficial“, „socially cautious“…
Edit: Just stumbled upon the word word „stewardship“ in the most recent EconTalk episode from a lecturer wanting to kindle a sense of stewardship over nuclear weapons in military personnel.