EA funders have funded various organisations working on biosecurity and pandemic preparedness, including:

  • John Hopkins Center for Health Security
  • Georgetown Center for Global Health Science and Security
  • Center for International Security and Cooperation Biosecurity Initiative 
  • Nuclear Threat Initiative
  • Blue Ribbon Study Panel on Biodefense

It seems to be widely accepted that many mainstream institutions got important things about COVID wrong, such as masks, travel bans, and lockdowns. Have there been any reviews of how these and other EA-funded things performed on COVID-related matters, with the benefit of hindsight?

New Answer
Ask Related Question
New Comment

4 Answers sorted by

Anecdote: On March 10, 2020 I watched a February 28, 2020 lecture by Dr. Amesh Adalja with Johns Hopkins Center for Health Security that currently has 51,000 views.

In the lecture, he predicted:

"The reaction will be worse than the virus."

He said the virus was not containable and that we shouldn't be trying to quarantine people to contain it.

A few minutes later, he said:

[...] Be alert, wash your hands a lot where you're in areas where there's a lot of common touch surfaces you might be touching, avoid sick people as much as you can, but there's not much else you can do at this point.

He also said wearing masks would be going overboard in response to a question if people should buy masks. He said he travelled internationally "yesterday" (which would have been February 9th if the video was uploaded the day of the lecture) and didn't wear a mask. He said he saw people wearing masks with their noses out or with masks around their neck (implying it wouldn't be effective to tell people to wear masks?) and also that it's uncomfortable to wear an N95 for too long, so he wouldn't recommend the general public to wear a mask unless sick (in which case "maybe" they should wear a surgical mask).

I think his prediction and advice should probably be judged negatively and reflect poorly on him / Center for Health Security, but I'm not sure how harshly he/ CHS should be judged.

Edit: Also, at 42:10 he said "I do think that it's not containable in any country, it just appears to be so now." I think this was also wrong, since clearly some countries have managed to avoid major outbreaks.

He said he travelled internationally "yesterday" (which would have been February 9th if the video was uploaded the day of the lecture) and didn't wear a mask.

This seems totally okay to me, FWIW. In most places (e.g., London or the US), it would have seemed a bit overly cautious to wear masks before the end of February, no? 

I think his prediction and advice should probably be judged negatively and reflect poorly on him / Center for Health Security, but I'm not sure how harshly he/ CHS should be judged.

I generally agree with that, but it's worth noting that it was extremely common for Western epidemiologists to repeat the mantra "you cannot do what Asian countries are doing; there's no way to contain the virus." 

It's actually worse than that. As I discovered when researching COVID giving opportunities for the FRAPPE donor group last year, Johns Hopkins experts explicitly recommended against wearing DIY masks in early March (a position reversed by the end of the month) and were not discouraging people from pressing ahead with travel plans as late as March 6. Sanjay had a phone call with them about a year ago in which he confronted them about these reversals, and they offered a sort of half-hearted defense.

I don't have any inside information about why CHS made the choices it did, but my naive view is that I agree with your comment that mistakes like these should reflect poorly on CHS. CHS's core competency may be more in the area of pandemic preparedness than dealing with the pandemic once it's already here, but their experts were quoted in the media a TON last spring and had significant ability (= responsibility) to shape the public conversation about COVID, particularly in the US. And yet lots and lots of people far less credentialed than CHS epidemiologists had correctly figured out by the first week of March that it was smart to wear a mask and to avoid being around others more than... (read more)

"And yet lots and lots of people far less credentialed than CHS epidemiologists had correctly figured out by the first week of March that it was smart to wear a mask" Not sure how much this is an answer - as I said in a different response, the question isn't whether CHS was right (much less right about one specific thing,) but whether they did better overall than the other policy-influencing organizations.

Fwiw, my vague memory is that some other people at CHS, including Tom Inglesby (the director) did better than Adalja. I think Inglesby's Twitter was generally pretty sensible though I don't have time to go back and check. I'd guess that, like most experts, he was too pessimistic about travel restrictions, though. Maybe masks, too?

I do think CHS should get some credit for arguing for taking pandemic response very seriously early on. For example, I think Tom had some tweets arguing for pulling out all the stops on manufacturing more PPE in January 2020. 

Note - I'm a bit biased since I was working on biorisk at Open Phil the first time Open Phil funded CHS.

I think the key question for CHS, or even Amesh specifically, is whether they outperformed other organizations, and by how much - i.e. were investments in CHS positive contributions to response overall. And they / he could have done almost arbitrarily bad in February, and still overall have vastly outperformed the alternatives.

And I think that it's clear that CHS's work was impactful in increasing preparedness in the US, even if the level of preparedness was far short of what was needed.

(Note that I have personal biases about this, and know many of the people at CHS.)

To be fair, the Johns Hopkins Center isn't just Adalja. I'm not aware of the list of things they do, but for instance, they kept an updated database in the early stage of the virus outbreak that was extremely helpful for forecasting!

If you're referring to what I think you are, it was a different group at Hopkins

Oh, you're probably right then!
In case anybody's curious: https://coronavirus.jhu.edu/map.html

Adalja also confidently predicted the infection fatality rate for the rest of 2020 to be around 0.6% (on the Sam Harris podcast)  despite thinking the virus can't be contained (if true, this would have led to more ICU beds and oxygen shortages in lots of places). In reality, the IFR was more like 0.9% or higher for countries like the US and UK. Probably it was lower for countries with younger demographics, but I don't even think Adelja was basing his estimates on that. 

(TBC, this isn't as big a mistake compared to other statements or compared to Ioannidis who completely disgraced himself throughout 2020 and ongoing, but I find it worth pointing out because I remember distinctly that, at the time when Adalja said this, there was a lot of fairly strong evidence for higher IFRs, including published estimates. I thought 0.6% seemed [edit]hard to defend, though I don't remember how much he flagged that there's a substantial chance it's significantly higher. Importantly, it would have been higher than it actually turned out to be, if Adalja had been right about "the virus can't be contained.")  

Was the prediction for infection fatality rate (IFR) or case fatality rate (CFR)? And high-income or all countries? Globally, the CFR is 2% (3.7M/173M), but the IFR is <0.66%, because <1/3 of cases were detected.
IFR (but back in February/March 2020, a lot of people called everything "CFR"). I think he was talking about high-income countries (that's what my 0.9% estimate for 2020 referred to – note that it's lower for 2020+2021 combined because of better treatment and vaccines). I'd have to look it up again, but I doubt that Adalja was talking about a global IFR that includes countries with much younger demographics than the US. It could be that he left it ambiguous. Here's the Sam Harris podcast in question [https://www.youtube.com/watch?v=E9vIUtXa9ug&ab_channel=SamHarris]; I haven't re-listened to it yet.
Seems unlikely that <1/3 of all cases were detected at this point, since the recent outbreaks had far higher detection rates than the initial ones.
Ah. If global IFR is worse than rich-countries' IFR, that seems to imply that developing countries had lower survival rates, despite their more favourable demographics, which would be sad.

It's still unclear, and the developing world detection and survival rates are a bit uncertain. I think you could probably get a decent approximation by looking at test positivity rates and testing volume compared to death rates over time in different countries, but I'm not going to put together the model to do it. 

We're doing something related with IFR estimates by age at 1DaySooner, but using seroprevalence data, i.e. only where there is really good data for the estimate. I don't have results of that yet.

This isn't impossible because there seems to be a correlation where people with lower socioeconomic status have worse Covid outcomes, but I still doubt that the IFR was worse overall in developing countries. The demographics (esp. the proportion of people age 70-80, and older) make a huge difference. But I never looked into this in detail, and my impression was also that for a long time at least, there wasn't any reliable data. From excess deaths in some locations, such as Guayaquil (Ecuador), one could rule out the possibility that the IFR in developing countries was incredibly low (it would have been at least 0.3% given plausible assumptions about the outbreak there, and possibly a lot higher).

A few opinions:

I think John Hopkins' advice has generally been well respected but that is just rumour on the grapevine, I cannot say exactly why I think that.

Nuclear Threat Initiative's Global Health Security Index said "international preparedness for epidemics and pandemics remains very weak" which seems correct. But the index also put the UK and the USA as the most prepared which seems incorrect (or at least gives a reason not to trust very shallow global indices). Eg yes the UK had a plan, tick box – but turns out it was not very good. Yes the UK did emergency exercises, tick box – but turns out it did not update plans based on the exercises. Etc

Not on your list but in this interview from Feb 2020, 80,000 Hours and the Future of Humanity Institute get a lot correct (eg need for social distancing) but they do both seem to disagree with the case for most travel bans, which seems incorrect in hindsight. See the intro where this is discussed.

Also not a bio org but EA Funded Our World in Data has done a good job on COVID data gathering and presentation.

One feature of the things that me and William have picked up on is that early on (say in Feb 2020 or earlier) advice coming from very respectable organisations was relatively poor.

I don’t think this should be seen as evidence that these organisations did badly (maybe a bit that they were over-confident) but that this was a very difficult situation to do things well in.

I don’t think this should be seen as evidence that these organisations did badly (maybe a bit that they were over-confident) but that this was a very difficult situation to do things well in.

I somewhat agree, but I think this point becomes much weaker if it was the case that at the same time when these organizations were giving poor advice some amateurs in the EA and rationality communities had already arrived at better conclusions, would have given better advice, etc.

I didn't follow the relevant conversations closely enough to have much of an inside view on how strongly the latter is true, but my impression is that many people in the EA/rationality communities (including ones who did follow the conversations more closely) think it's true. Even I am aware of some data points that seem to suggest such a conclusion (e.g., some conversations I remember).

FWIW, at least given what I know I think I find this less compelling as a 'vindication of EA/rationalist epistemics' or whatever than some other people seem to - I think the lessons we should learn depend on a number of additional things which I'm currently uncertain about:

  • Are we comparing like with like rather than cherrypicking? (I.e
... (read more)

Hi, Yes good point, maybe I am being too generous.

FWIW I don’t remember anyone in the EA / rationalist community calling for the strategy that post-hoc seems to have worked best of a long lock-down to get to zero cases followed by border closures etc to keep cases at zero. (I remember  a lot of people for example sharing this note which gets much right but stuff wrong: eg short lock-dock and comparatively easy to keep R below 1 with social distancing)

Can you say more about the strategy you have in mind and give examples of the countries that implemented it? It's hard to judge whether EAs or rationalists advocated it from your characterization, which seems to build into it the relevant success conditions. Lots of countries had long lockdowns and strict border closures, but failed miserably at bringing cases down to zero or anything remotely approaching zero. Furthermore, whether this approach was best all-things-considered should obviously also take into account the enormous costs associated with these measures (in all of the countries that implemented long lockdowns and strict border closures).

weeatquince's is sharing a widely held view, i.e. that eradication is superior to containment in health and economic outcomes, see e.g. this analysis. The idea itself is plausible, since a successful lockdown allows complete reopening of the internal economy afterwards.    

Sample size is however small, especially when it comes to non-island countries. I only know of two non-island countries that seriously went for eradication coupled with border closures, namely Vietnam and Israel. Israel gave up at one point when cases started to rise (which is why it is listed among the containment countries in the analysis above) , but Vietnam succeeded (although it had to restrict travel heavily domestically as well). Personally, I believe it is a suboptimal strategy for non- authoritarian, non-island countries.  

I think their original point stands though, that EA/rationalists did not seem to entertain the idea of eradication enough,  but probably neither did biorisk organizations last year.

Thanks for the clarification. I feel that this discussion is not framed correctly. Yes, successful eradication is superior to containment in health and economic outcomes. This is a pretty weak claim that lots of people can agree with who otherwise differ considerably in their policy proposals. But the original claim was that EAs and rationalists hadn't advocated for long lockdowns and border closures, and that this was relevant for retroactively assessing their performance. The plausibility of the latter claim must be evaluated by considering all the countries that implemented long lockdowns and border closures, and not just the tiny minority that were successful in attaining (near-)eradication by adopting those measures. I took a quick look at the study you shared. Their analysis compared covid deaths, GDP growth and lockdown stringency in two groups of OECD countries during the first twelve months of the pandemic, and offered this as their original contribution to the study's main thesis that countries which favored elimination had better health and economic outcomes than countries which favored mitigation (the rest of the study is a brief and unsystematic summary of some of the relevant literature). It turns out that the group which supposedly favored elimination consists of just five nations, four of which are islands and the fifth of which (South Korea) shares borders with a single county which has been completely isolated from the rest of the world for decades. Let's pause for a moment and consider how quickly this kind of evidence would have been dismissed if it had been presented in support of a politically inconvenient conclusion. Yet here it is offered, in the world's most prestigious medical journal, to establish that "elimination, not mitigation, creates best outcomes for health, the economy, and civil liberties". For what it's worth, I personally have no strong views on how the pandemic should have been handled. (My only strong meta-view is that dec
I don’t think I can help much with answering these questions. I was thinking of counties like Australia and New Zealand and Taiwan. But whether or not the strategies adopted in these places was actually optimal or best with the available information or applicable to most countries that are not islands or had a high chance of failure – I cannot say! All I can say is that there is at least one plausible strategy that seems to have worked well in at least some countries and I personally don’t really remember it being discussed within the EA space a year ago. Feel free to draw what conclusions or analysis you will from that. Just to add, I expect (but I might be wrong) that these countries have had lower welfare and economic costs than most other places.

I discuss the GHS index at greater length in my answer.

Thank you :-)

they do both seem to disagree with the case for most travel bans, which seems incorrect in hindsight

Can you say more about this, or point me to good resources off the top of your head? I didn't follow the conversation on travel bans, and currently don't have a good sense of what people think about them.

In particular, is the rough basis for "incorrect in hindsight" the impression that travel bans were a potentially essential part of the successful containment strategy of some countries such as New Zealand? Or are there other major considerations as well?

My understanding is that travel bans were widely believed to have greater costs than benefits before COVID. There are various quotes along those lines described in the (rather cynical) Lessons From the Crisis post on the topic of border closures.

In February 2020, I believed border closures weren't worth it. I thought they disincentivised countries from being transparent about emerging outbreaks (because said countries would face economic punishment via closed borders) and could only slow down the spread of a disease, not stop it. While I'm still not entirely sure about the relative benefits of open reporting vs. slowed spread, I was definitely underestimating the benefits of the latter. Evidence from Vietnam and New Zealand shows that early and strong international border controls can indeed slow the spread to the point where local outbreaks don't spiral beyond easy containment.

To link you to some resources, a September 2020 Cochrane meta-analysis, Travel‐related control measures to contain the COVID‐19 pandemic: a rapid review, concluded that:

There was insufficient evidence to draw firm conclusions about the effectiveness of travel‐related quarantine on its own. Some of the incl

... (read more)
Thank you, very helpful!

Another slightly tangential but very similar question that came up in conversation I had recently is:

"How well have EA-funded orgs built on the momentum created by the COVID-motivated global interest in GCRs (global catastrophic risks) to drive policy change or other changes to help prevent GCRs and x-risks"

I could have imagined a world where the entire longtermist community pivoted towards this goal and at least for a year or two and focused all available time skill and money on driving GCR related policy change – but this doesn’t seem to have happened much. I could imagine the community looking back at this year and regretting the collective lack of action.

The organisation where I work, the APPG for Future Generations pivoted significantly, kickstarted a new Parliament Committee on risks and I wrote a paper on lessons learned from COVID which had significantly government interest and seems to have driven policy change (writeup forthcoming).

But beyond that there has definitely been some exciting stuff happening. I know:

  • CSER are starting a lessons learned from COVID project, although this is only just getting started.
  • FHI staff have submitted a some evidence to parliamentary inquiries (example).
  • The CLTR (funded by the EAIF) has launched a report on risk (I'm unsure if this was a change in direction or always the plan).
  • No more pandemics (not funded) was started.

This stuff is all great and I am sure there is more happening – but my general sense is that it is much less than and much slower than I would have expected.

I also loosely get the impression (from my own experience and that of 2-3 other orgs that I have talked to) that various EA funders have been disinterested in pivoting to support lessons learned from COVID focused policy work, some of which could scale up quite significantly, and that maybe funding is the main bottleneck for some of this (I think funding for more policy work is a bottleneck for all of the orgs listed above except FHI).

[Disclaimer – I will be bias given that I pivoted my work to focus on COVID lessons learned and policy influencing and looked for funding for this.] 

The Global Health Security Index looks like a misfire. This isn't directly about performance during the pandemic, but Nuclear Threat Initiative, funded by Open Phil for this purpose (h/t HowieL for pointing this out) and collaborating with the Johns Hopkins Center for Health Security, made the 2019 Global Health Security Index which seems invalidated by COVID-19 outcomes and may have encouraged actors to take the wrong moves. This ThinkGlobalHealth article describes how its ratings did not predict good performance against the virus. The article relies on official death counts rather than excess mortality, but I made that correction and reached similar results.

Looking through the index, there are some indicators which don't make sense, like praising countries for avoiding travel restrictions (which is perverse), praising them for having more ethical regulations against surveillance and clinical trials (which may be ethically justified but is more likely to make it harder to fight a pandemic), and praising them for gender equality (a noble sentiment but not directly relevant to pandemics).

Even cutting some of those dubious measures out, I found the index was not predictive of excess mortality. In general it appears that effective pandemic response is not about preparation and this may have been systematically overlooked by EA efforts and funding recipients in the realm of biorisk.

Some people have also criticized the index for rating China moderately highly on prevention of pathogen release, considering that COVID-19 came from China, but considering that COVID-19 is just one data point of virus emergence or lab leak and that China is a very large country I don't think this is right.

"effective pandemic response is not about preparation"

FYI - my impression is that pandemic preparedness is often defined broadly enough to include things like research into defensive technology (e.g. mRNA vaccines). It does seem like those investments were important for the response.

Hm, certainly the vaccine rollout was in hindsight the second most important thing after success or failure at initial lockdown and containment. It does seem to have been neglected by preparation efforts and EA funding before the pandemic, but that's understandable considering how much of a surprise this mRNA stuff was.
I think research into novel vaccine platforms like mRNA is a top priority. It's neglected in the sense that way more resources should be going into it but also my impression[1] is that the USG does make up a decent proportion of funding for early stage research into that kind of thing. So that's a sense in which the U.S.'s preparedness was prob good relative to other countries though not in an absolute sense. Here's an article I skimmed about the importance of govt (mostly NIH) funding for the development of mRNA vaccines. https://www.scientificamerican.com/article/for-billion-dollar-covid-vaccines-basic-government-funded-science-laid-the-groundwork/ [https://www.scientificamerican.com/article/for-billion-dollar-covid-vaccines-basic-government-funded-science-laid-the-groundwork/] Fwiw, I think it's prob not the case that the mRNA stuff was that much of a surprise. This 2018 CHS report had self-amplifying mRNA vaccines as one of ~15 technologies to address GCBRs. https://jhsphcenterforhealthsecurity.s3.amazonaws.com/181009-gcbr-tech-report.pdf [1] Though I'm rusty since I haven't worked directly on biorisk for five years and was never an expert.
I second the impression that it's not that much of a surprise. For example, CEPI was founded with a goal of accelerating vaccine development against the WHO R&D Blueprint priority diseases [https://www.who.int/activities/prioritizing-diseases-for-research-and-development-in-emergency-contexts] and according to their R&D webpage [https://cepi.net/research_dev/technology/]: I think it was a surprise that non-self-amplifying mRNA vaccines work as well as they do (mRNA is more immunogenic than predicted, I guess, at least for COVID?). 18 months ago, I don't think I would have bet on mRNA platform vaccines as the future over DNA or adenovirus vaccines.

Hello, Thank you for the interesting thoughts. The comments on the GHS index are useful and insightful.

Your analysis of COVID preparation on Twitter is really really interesting. Well done for doing that. I have not yet looked at your analysis spreadsheet but will try to do that soon.

To touch on a point you said about preparation, I think we can take a bit more of a nuanced approach to think about when preparation works rather than just saying "effective pandemic response is not about preparation". Some thoughts from me on this (not just focused on pandemi... (read more)

I don't think most people would consider prevention a type of preparation. EA-funded biorisk efforts presumably did not consider it that way. And more to the point, I do not want to lump prevention together with preparation because I am making an argument about preparation that is separate from prevention. So it's not about just semantics, but precision on which efforts did well or poorly. Conventional wisdom is worth little when it is the product of armchair speculation rather than experience. If people live through half a dozen pandemics and still have that conventional wisdom then we can have a different conversation. Wouldn't preparation seem to be a part of the story of COVID-19 outcomes given a similarly superficial level of inquiry? Forget semantics. Did EA funding efforts and recipients design systems that made good decisions about COVID-19? Did anyone who talked about "pandemic preparation" pre-2020 use the term to encompass the design of systems like that? Well you can't just define preparation as "good plans", that's a no-true-Scotsman argument. If you have some way of ensuring that your preparation will be good preparation then it's a different story. That isn't necessarily due to physical preparation, it could easily be intangible changes in the culture and political system, granting that there is in fact a causal connection as opposed to East Asia and Australasia just being better at this stuff. iirc there was a study which found that American cities that lived through the Spanish Flu (1919) suffered less death early in the COVID19 outbreak. Cannot find the study now but if it's really true then that would be hard to explain through preparation. I'm not sure exactly what anti-fragile means but that doesn't sound right, decision systems in the US/UK for instance didn't fall apart, they were just apathetic and unresponsive to good ideas just like they are for mundane problems that aren't big crises. In other words they calmly kept operating the wa
I think it actually is common to include prevention under the umbrella of pandemic preparedness. for example, here's the Council on Foreign Relation's independent committee on Improving Pandemic Preparedness [https://www.cfr.org/report/pandemic-preparedness-lessons-COVID-19/recommendations/] : "Based on the painful lessons of the current pandemic, the Task Force makes recommendations for improving U.S. and global capacities to deliver each of the three fundamentals of pandemic preparedness: prevention, detection, and response. " Another example: https://www.path.org/articles/building-epidemic-preparedness-worldwide/ [https://www.path.org/articles/building-epidemic-preparedness-worldwide/] So it might be helpful to specify what you're referring to by preparation.

It seems fair to call avoiding travel restrictions a dubious measure in hindsight, but circa 2019 it strikes me as a reasonable metric to put under "compliance with international norms". There was an expert consensus that travel norms weren't a good pandemic response tool (see my other comment) and not implementing them is indeed part of complying with the WHO IHRs.

I am not totally sure that compliance with international norms a good measure of national health security! However, the according to the Think Global Health article you linked on Twitter, even t... (read more)

1 Related Questions

7Answer by Mats Olsen1y
A group of us developed coviddash.org [https://coviddash.org/] which was referenced in the NYTimes [https://www.nytimes.com/2020/08/05/health/volunteer-coronavirus-vaccine-trial.html] and had about ~30K visitors, directing between 10K-20K to vaccine trial screening websites or to the sites themselves. We had sites thank us for patients so we know we our counterfactual impact on trial enrollment was >0, but we are not sure of the total number we actually directed. If we accelerated just one vaccine for just one day though we probably helped save a few hundred lives. We were not funded by any EA org but some of us are EA-aligned. I plan to write a more detailed recap/analysis on this and to discuss accelerating vaccine R&D during pandemics but haven't gotten to it yet because I already moved onto the next project...

He was also apparently instrumental in pushing through a big grant for Our World In Data despite the sclerotic procurement process.

This NYTimes Magazine article [https://www.nytimes.com/2021/06/16/magazine/cdc-covid-response.html?smid=url-share] might be interesting. Its framing is basically "why did the CDC fail, and how can it do better next time?". It mentions some other groups that allegedly did better than the CDC. Though I don't know to what extent these groups were or were not EA-funded. E.g., it says:

Patrick McKenzie isn’t a “group”, and he probably doesn’t need your money, but he did get ahead of coronavirus impacts in Japan successfully: https://www.kalzumeus.com/2020/04/21/japan-coronavirus/

2JP Addison1y
And VaccinateCA was very impressive.
4Answer by Rory Greig1y
I would add the New England Complex Systems Institute, particularly Yaneer Bar Yam: https://necsi.edu/corona-virus-pandemic [https://necsi.edu/corona-virus-pandemic] In this article [https://necsi.edu/systemic-risk-of-pandemic-via-novel-pathogens-coronavirus-a-note] from January 2020, which has aged very well, they were advocating for restrictions on international movement and warning of the effect of superspreader events on estimates of R0. Yaneer Bar Yam also started this multidisciplinary effort to tackle covid: https://www.endcoronavirus.org/ [https://www.endcoronavirus.org/]
7Neel Nanda1y
To me the shining example of this is Jacob Falkovich's Seeing the Smoke [https://putanumonit.com/2020/02/27/seeing-the-smoke/], which somehow helped convince the UK [https://www.lesswrong.com/posts/2JxyKXgyqzovNDvnE/monastery-and-throne] to lockdown in March 2020.
9 comments, sorted by Click to highlight new comments since: Today at 7:43 PM

Great question, and I look forward to following this discussion!

A tangential (but important in my opinion) comment… You write that “EA funders have funded various organisations working on biosecurity and pandemic preparedness”, but I haven’t seen any evidence that EA funders aside from Open Phil have funded biosecurity in any meaningful way. While Open Phil has funded all the organizations you listed, none of them have been funded by the LTFF, Survival and Flourishing Fund, the Centre on Long-Term Risk Fund, or BERI, and nobody in the EA Survey reported giving to any of the organizations.

The LTFF has admittedly made some small biosecurity grants (though as a reference it has granted ~19x more to AI), and FHI (which has relatively broad support from EA and/or longtermist funders) does some biosecurity work. But broadly speaking, I think it’s a (widely held) misconception that EA donors besides Open Phil were materially prioritizing biosecurity grantmaking prior to the pandemic.

This is meta, but I'm sort of hoping the recent Long Term Future Fund grant to CSER on "Ensuring the lessons learnt from COVID-19 improve GCR prevention and mitigation in the future" will generate some answers related to this question. I'd be very interested to see some postmortem work within that project on the actions taken by these major biosecurity organizations.

In a throwback to articles like this one, including one critique that tessa summarized as:

Open Phil is offering large grants to a few carefully-selected parties, giving them outsize impact (and a diversity of perspectives and research focuses is good the the field)

I'm also interested in whether Open Phil funded parties did better or worse than either a) existing big players (as opposed to a possibly different bar of being measured "against reality") or b) a more diverse/"democratic"marketplace of ideas among credentialed biosecurity folks.

Measured against reality, a pandemic happened and a lot of people died for preventable reasons, so in at least one sense clearly we/they didn't do good enough. :/

Vipul Naik asked a similar question near the beginning of the pandemic.

Edit: I've reposted this comment as an answer, and am self-downvoting this.

[This comment is no longer endorsed by its author]Reply

Note that Open Phil funded this project. https://www.nti.org/newsroom/news/nti-launch-global-health-security-index-new-grant-open-philanthropy-project/

[Edit – moved comment to answer above at suggestion of kbog] 

I moved my comment to an answer after learning that the index was directly funded by an Open Phil grant. You'd do better to repost your reply to me there. Sorry about the confusion.