I sometimes get a vibe that many people trying to ambitiously do good in the world (including EAs) are misguided about what doing successful policy/governance work looks like. An exaggerated caricature would be activities like: dreaming up novel UN structures, spending time in abstract game theory and ‘strategy spirals[1]’, and sweeping analysis of historical case studies.
Instead, people that want to make the world safer with policy/governance should become experts on very specific and boring topics. One of the most successful people I’ve met in biosecurity got their start by getting really good at analyzing obscure government budgets.
Here are some crowdsourced example areas I would love to see more people become experts in:
- Legal liability - obviously relevant to biosecurity and AI safety, and I’m especially interested in how liability law would handle spreading infohazards (e.g. if a bio lab publishes a virus sequence that is then used for bioterrorism, or if an LLM is used maliciously in a similar way).
- Privacy / data protection laws - could be an important lever for regulating dangerous technologies.
- Executive powers for regulation - what can and can't the executive actually do to get AI labs to adhere to voluntary security standards, or get DNA synthesis appropriately monitored?
- Large, regularly reauthorized bills (e.g., NDAA, PAHPA, IAA) and ways in which they could be bolstered for biosecurity and AI safety (both in terms of content and process).
- How companies validate customers, e.g., for export control or FSAP reasons (know-your-customer), and the statutes and technologies around this.
- How are legal restrictions on possessing or creating certain materials justified/implemented e.g. Chemical Weapons Convention, narcotics, Toxic Substances Control Act?
- The efficacy of tamper-proof and tamper-evident technology (e.g. in voting machines, anti-counterfeiting printers)
- Biochemical supply chains - which countries make which reagents, and how are they affected by export controls and other trade policies?
- Consumer protection laws and their application to emerging tech risks (e.g. how do product recalls work? Could they apply to benchtop DNA synthesizers or LLMs?)
- Patent law - can companies patent dangerous technology in order to prevent others from developing or misusing it?
- How do regulations on 3d-printed firearms work?
- The specifics of congressional appropriations, federal funding, and procurement: what sorts of things does the government purchase, how does this relate to biotech or AI (software)? Related to this, becoming an expert on the Strategic National Stockpile and understanding the mechanisms of how a vendor managed inventory could work.
A few caveats. First, I spent like 30 minutes writing this list (and crowdsourced heavily from others). Some of these topics are going to be dead ends. Still, I’d be more excited about somebody pursuing one of these concrete, specific dead ends and getting real feedback from the world (and then pivoting[2]), rather than trying to do broad strategy work and risk ending up in a never-ending strategy spiral. Moreover, the most impactful topics are probably not on this list and will be discovered by somebody who got deep into the weeds of something obscure.
For those of you that are trying to do good with an EA mindset, this also means getting out of the EA bubble and spending lots of time with established experts[3] in these relevant fields. Every so often, I’ll get the chance to collect biosecurity ideas and send them to interested people in DC. In order to be helpful, these ideas need to be super specific, e.g. this specific agency needs to task this other subagency to raise this obscure requirement to X. Giving broad input like ‘let’s have better disease monitoring’ is not helpful. Experts capable of producing these specific ideas are much more impactful, and impact-oriented people should aspire to work with and eventually become those experts.[4]
I appreciated feedback and ideas on the crowdsourced list from Tessa Alexanian, Chris Bakerlee, Anjali Gopal, Holden Karnofsky, Trevor Levin, James Wagstaff, and a number of others.
- ^
'Strategy Spiral' is the term I use to describe spending many hours doing ‘strategy’ with very little feedback from the real world, very little sense of what decision-makers would actually find helpful or action-relevant, and no real methodology to actually make progress or get clarity. The strategy simply goes in circles. Strategy is important so doing strategy can make you feel important, but I think people often underestimate the importance of getting your hands dirty directly, plus in the long run it will help you do better strategy.
- ^
And then if you write up a document explaining why this was a dead end, you benefit everybody else trying to have an impact (or perhaps inspire somebody to perhaps see a different angle on the problem).
- ^
One of the people reading this said ‘I feel like one thing I didn't understand until pretty recently is how much of (the most powerful version of) this kind of expertise basically requires being in a government office where you have to deal with an annoying bureaucratic process. This militates in favor of early-career EAs working in government instead of research roles’
- ^
Concretely, this looks like either getting an entry level job in government, or being at a think tank but working closely with somebody in government who actually wants your analysis, or drilling deep on a specific policy topic where there is a clear hypothesis for it being ‘undervalued’ by the policy marketplace. Doing independent research is not a good way of doing this.
4 and 7 are not really questions that one can meaningfully develop expertise on. Even politicians, whose jobs depend on understanding public opinion, are worse at this than just running a poll, and depend heavily on polling to assess public opinion when they have the money to run adequate polls. They do bring a useful amount of additional judgment to that process and can give you a sense of when a poll result is likely to not hold up in an adversarial environment, but I don't think you can develop an equivalent skill without actually spending a lot of time talking to the public. I also don't think that would allow you to do much prediction of where public opinion is headed. Hillary Clinton would probably have been elected President in 2008 if she had been able to predict how Dem primary voters' opinions on her Iraq vote would change, and she never lacked access to world-class experts at the time she was making her decision.
You could spend 30k to run a poll and get a better sense of current public sentiment and specific ways opinions can be pushed given information currently available. A world-expert level pollster could perhaps help you write better questions, and you could review the history of pubic opinion on topics you find to be analogous. I think with all that you'd outperform most unelected policymakers in understanding current public opinion, but only because their condescension toward the average person makes them especially bad at it (see, e.g. their obvious bungling of covid). I'd be extremely skeptical that you'd do any better at predicting what shifts public and elite opinion better than an average swing-district member of Congress who took 10 min to review your poll.