929Joined Feb 2020


Biosecurity at Open Phil


Excited to see this kind of analysis!

Worried that this is premature:

there is no reason for the great powers to ever deploy or develop planet-killing kinetic bombardment capabilities

This seems true to a first approximation, but if the risk we are preventing is tiny, then a tiny chance of dual-use  becomes a big deal.  The behavior of states suggests that we can't put less than a 1 in 10,000 chance on something like this.  Some random examples:

  • During WW2, there were powerful elements  within the Japanese government that advocated total annihilation rather than surrender (Wikipedia).
  • Deterrence can benefit from credible signals of suicidal craziness (e.g. the 'Samson Option' named after biblical character who destroyed a temple, killing himself and taking everybody with him).
  • The Soviet bioweapons program invested heavily in contagious weapons (e.g. smallpox) and modifying them to overcome medical countermeasures.  This work seemed to be driven by weird bureaucratic incentives that were pretty divorced from rational strategic goals/objectives of the Soviet Union.

See Daniel Greene's comment about creating better norms around publishing dangerous information (he beat me to it!).

I won't comment on their endorsements or strategy, but I will say that even if Carrick is a longshot it doesn't necessarily follow that it's a bad use of marginal dollars.

Thanks for flagging, I missed this and agree this should be in blog category per the policy. Will chat with mods to figure out how to fix.

Update: after discussing and looking at some background documentation with Oli, we think the claim about ‘potentially thousands of lives’ is sufficiently supported.

Dropping a quick comment to say I've upvoted this and might respond with more later. I do concede the claim about thousands of lives was not throughly scrutinized and I'm getting more info on that now (and will remove if it doesn't check out). I otherwise stand by what I've written and also think Oli has worthwhile points.

Thanks! And yes, this seems right to me.

Huge +1 to this. If anybody is reading this and wants to get funded to start down this career track, please apply to Open Phil's biosecurity scholarship:

The program supports independent projects for people to learn about a field as well as degree programs.

Thanks Evan, and welcome to the forum!  I agree this is an important question for prioritization, and does imply that AI is substantially more important than bio (a statement I believe despite working on biosecurity, at least if we are only considering longtermism).  As Linch mentioned, we have policies/norms against publicly brainstorming information hazards.  If somebody is concerned about a biology risk that might constitute an information hazard, they can contact me privately to discuss options for responsible disclosure.

One possible advantage to using the platform would be that donations to charities are tax deductible, whereas donations to campaigns are not. If set up well, this mechanism could enable somebody to 'donate' to a campaign with tax deductibility.

Load More