You'd have to get a ton of people with minimal overlap in professional interests/skills to work together in such an agency. And depending on the 'pet projects' of the people at the highest levels, you might get a disproportionate focus on one particular risk (similar to what you see in EA right now - might be difficult to retain biologists working on antimicrobial resistance, or gene drive safety). Then the politics of funding within such an agency would be another minefield entirely - "oh, why does the bioterrorism division get x% more funding than t...
I am not sure you are giving governments enough credit. Wrt things like gene drive safety, certain agencies are already working on these things. I know some researchers who just got a grant to work on how to contain and manage gene drives. US military research also includes plenty of stuff on bioterrorism - both agricultural and through pathogens. Grantmaking efforts are relatively rapid ways to get this stuff done, I think?
X-risk is so broad and cuts across so many different fields that dedicating an entire agency to it seems difficult, especially if you consider effectiveness.
I think, in this type of analysis, for an infectious disease, it's really important to look at potential for spread as well.
Malaria is region-constricted (only places with the right mosquitoes), whereas HIV is not. Therefore, there's a natural cap at the amount of malaria we can have if malaria control ceased to exist, whereas HIV's 'natural cap' is potentially "all susceptible humans".
If you include "all future infections" into the analysis, how much suffering due to HIV can be avoided due to current efforts to control HIV? I mean, ...
"Even if an outside expert spots a significant risk (think AI risk), if there's not a clear department responsible (in the UK: Business, Enterprise, and Innovation? Digital, Culture, Media, and Sport?) then nothing will happen." - I disagree with this point. You yourself pointed out the environmental movement. You can get x-risks onto the agenda without creating an entire branch of the government for it. Also, having x-risks identified by experts in the field who can talk about it amongst each other without immediately assuming “this is an x-risk... (read more)