Non-pharmaceutical interventions in pandemic preparedness and response

Anon-biosec-account

Non-pharmaceutical interventions in pandemic preparedness and response

Anon-biosec-account

14 min read · Apr 8, 2021

Comments 16

Sorted by

New & upvoted

Will Bradshaw

I like this post a lot. It's well-written, thoughtful, well-linked and thorough. I especially like the way you've scoped out the different project ideas at the end of the post, and you include several classes of project that I think are important but usually left out of these sorts of lists.

It seems likely to me that this would be a very good use of money for "generalist" (non-GCBR-focused) pandemic prevention. I'd be very interested in hearing counterarguments to that.

For GCBR reduction, I'm less sure how valuable it would be to better evaluate existing NPIs, as opposed to developing (and evaluating) new and improved NPIs targeting key weak points. But I could imagine being persuaded fairly easily on this point, and would like to get other people's takes (though perhaps not in an open forum).

Anon-biosec-account

Thanks a lot for the comment. I was a bit nervous to put my first post up so some positive feedback is very much appreciated.

Davidmanheim

Strongly agree about many of these points. I think it's worth looking at our earlier post, and the paper we wrote on almost exactly this topic which we worked on in 2019 - which obviously aren't focused on post-COVID-19 ideas.

On objections to trials, there is a large literature about the difficulty of assessing the impact of interventions, from Pearl's fundamental argument, here (pdf), to the entire corpus of work on generalizability and transferability in practice.

I think that further development of the suggested potential projects would be valuable - if you agree, I'd be happy to discuss how to turn them into more concrete proposals. Though in fact, many of these have already been done - a literature review (post is strongly recommended reading!) would probably find many pieces like this one that address many of your points.

Linch

Thanks a lot for this! Like willbradshaw I agree that this post is "well-written, thoughtful, well-linked and thorough!"

What are some objections to anything I’ve written here?

If I were to nitpick, I think my biggest objection is that your approach to tackling the problem of NPIs for pandemic preparedness and response appears extremely atheoretical. I think this is fine for a scoping study that tries to estimate the scale of the problem, and fine (perhaps even highly underrated!) for clinical studies. But I think we can get decent results at lower cost with a bit of simple theory.

I believe this because I think the human body in general, and the immune system in particular, is woefully complicated, so it makes sense that we cannot have much faith in biologically plausible mechanisms for treatments, which leads us to necessitate correspondingly greater faith in end-to-end RCTs(and be in a state of radical cluelessness otherwise). But there are other parts of epidemiology that's simpler and more well-understood, such that for transmission we can be reasonably confident in our ability to dice the problem and isolate it into specific confusing subcomponents.

For example, suppose we are worried about a potential respiratory disease pandemic, and we want to figure out whether intervention X (say installing MERV filters for offices) has a sufficiently large impact on an (un)desired endpoint (eg symptomatic disease, hospitalizations). One approach might just be:

Sounds plausible, but we can't know much with confidence in the absence of end-to-end empirical results. What we can do is run an RCT where we install MERV filters in the treatment group and don't install it in the control group with a sufficiently large sample size to power for differences that are big enough for us to care about, and compare results after the study's natural endpoint.

I think this is good, but potentially quite expensive/time-consuming (which is really bad in a fast-moving pandemic!). One way we can potentially do better:

Well, disease transmission isn't magic, and we're reasonably confident in the very high-level theory of respiratory diseases. So we can at least decompose the problem into two parts:
Treat human bodies as a black box function that takes in some combination of scary microbe-laden particles and outputs some probability of undesired endpoints.
Model the world as something that sends scary microbe-laden particles and figure out which interventions reduce such particles to a level that the modeled function in (1) should consider too low to notice.

My decomposition isn't particularly interesting, but I think it's reasonably clean. With it, we can

Tackle 1) with human challenge trials where microbe dose/frequency/timing is variable, to understand what are plausible ranges of parameters for how many droplets are needed to be bad.

Tackle 2) with some combination of

computational fluid dynamics simulations
lab experiments on how much people breathe each other's air, and how fast air need to cycle to reduce that.
field experiments on effect of MERV filters on closely analogous particles (on the physical level)
prior knowledge of the transmission patterns of other similar diseases
???

Now my decomposition is still quite high level, and I'm not sure that my suggested instrumentalizations here aren't dumb. But hopefully what I'm gesturing at makes sense?

Anon-biosec-account

Thanks a lot for the comment. I do think that what your gesturing at makes sense: if I understand correctly you are saying that certain physical interventions can have more predictable effects that ‘biological’ ones because we have a decent idea of exactly how they work. In some cases this is definitely true: as an extreme example, we don’t need RCTs of aeroplane safety as we have a very good understanding of the physical processes and are able to model them well. If we have an airborne pathogen, it’s hardly necessary to run an RCT to see whether or not there is an effect of a stay at home order: there will be one.

In many of the example questions I gave though, I think the fact that there is a large behavioural component pushes us closer to the situation we have with drugs than to the aeroplane. For example, although it could be demonstrated in a laboratory which of mask or shield is actually more effective at blocking exhaled particles, it would be harder to capture the different effects that each has on how often you touch your face, how often it is removed, or other aspects of compliance. These will differ a lot between people, so you’d need to test it on a large group, and the social setting might influence behaviour. I don’t think that we can decompose the often important behavioural component of these interventions in the same way that we can the physical components.

That said, the air filtration question I posed might not have been well chosen. As you point out, it seems reasonable that we can get a good understanding of whether that is likely to be helpful by applying what we know about the filters and viral transmission. Of the questions I posed, RCTs are likely to be the least useful there and may not be useful at all.

However, I do have some thoughts on why an RCT could still be worthwhile. I’m not saying these because I disagree with your points; I’m just providing some possible counterargument.

Learning: by introducing the filters not in an RCT, you are basically doing an experiment but losing the opportunity to learn from it. Even if it has been decided that filters should be introduced in all schools/offices (or whatever unit), it won’t normally be possible to install all of those in parallel. So there is a time where some offices have the filter and some don’t. As long as you can randomise this, you can take advantage of the differences in time for implementation in something like a stepped wedge cluster randomised trial. The effect could be analysed on an ongoing basis in a Bayesian analysis such that if there are large effects they would be detected early in the experiment and implementation of the remaining filters can be accelerated. If you are doing something like this across several interventions, this would help with deciding which to prioritise.
Cost-benefit: There are ~ 137,000 schools in the US. I don’t know how much it costs to install and maintain filtration systems, but I imagine it is not negligible. There are a lot more schools globally. Doing an RCT comparing e.g. air filtration to opening the windows could save quite a bit of money if it turns out that filtration systems don’t provide additional benefit.
Implementation and interaction with behaviour: even assuming that they do work, do people use them? Maybe the filtration is noisy so teachers turn it off; maybe they simply forget to turn it on. In medicine, even with drugs that demonstrably improve the patient’s condition, adherence is (to me) surprisingly low. Perhaps large rooms where people tend to congregate most cannot be adequately filtered, maybe the filtration system gives people a sense of security so they congregate more.

Overall, I think the areas where trials would be most useful are those where we can expect relatively modest effects and where there is a larger behavioural component. The combination of modest effects, if better understood, might be quite important.

Tsunayoshi

There's an additional factor: Marketing and public persuasion. It is one thing to say: Based on a theoretical model, air filters work, and a totally different thing to say: We saw that air filters cut transmission by X% . My hope would be that the certainty and the effect estimate could serve to overcome the collective inaction we saw in the pandemic (in that many people agree that e.g. air filters would probably help, but barely nobody installed them in schools).

Anon-biosec-account

Good point. This is similar to what I was trying to get at when talking about lack of willingness to engage in probabilistic reasoning.

Aaron Gertler 🔸

Just saw some recent results from a cool NPI study (examining the impact of many different interventions on mask-wearing in Bangladesh). See this slideshow for a quick summary of results.

The headline: Mask distribution and promotion tripled mask usage in test locations, and led to sustained mask use for ~10 weeks after the promotion ended.

From the slideshow, here's what worked from the intervention:

The cost-effectiveness numbers aren't too impressive, but I'd think the numbers would look better for a deadlier pandemic:

Anon-biosec-account

Thanks a lot for sharing this. I need to update the post to add this and other research that has been pointed out to me.

rootpi

I liked the post (I have worked a lot on RCTs, and a little on NPIs, but not together alas!). Here's another paper that I didn't see mentioned (although maybe I missed it) which I think roughly falls into the category you're considering?https://www.nber.org/system/files/working_papers/w27496/revisions/w27496.rev0.pdf

I feel like there are also some behavioral econ papers looking at e.g. social distancing in queues, but off the top of my head I'm not certain if there are actual randomized field experiments in that space...

-julian

Anon-biosec-account

For future searching, where/how did you come across that paper?

Anon-biosec-account

Good find - thanks for sharing that paper which I hadn't included. If I update the post I'll add that.

Tsunayoshi

Truly excellent post!

My intuition is that research abouts NPIs on behavioural change might be more tractable and therefore impactful than research where the endpoint is infection. If the endpoint is infection, any study that enrolls the general population will need to have very large sample sizes, as the examples you listed illustrate. I am sure these problems can be overcome, but I assume that one reason we have not seen more of these studies is that it is infeasible to do so without larger coordination.

While it is unfortunate and truly surprising that we have very little research on e.g. the impact of mask wearing and distancing, we do know that certain behavioural, realistic changes would be completely sufficient to squash the pandemic in many regions.

The change does not have to be large: As the reproductive number R is magically hovering around ~1.1 to ~1.3 in most regions in the Western world, it would be sufficient if people would act just a little bit more careful to get R below 1: That could mean reducing private meetings by e.g. one third (or moving them outside), widespread adoption of contact tracing apps, placing air filters in schools, or targeting public health messaging towards people that currently are not reached or persuaded. I have seen some research about vaccine hesitancy, but far less about these other areas. At the very least, a randomized study comparing different kinds of public health messaging seems really easy to do.and fairly useful. This might look differently for the next pandemic though.

More broadly: As you alluded to, fostering and increasing coordination between researchers looking to conduct a study might also be really useful. This applies probably even more to research about drug interventions, but way too much of it is underpowered and badly conducted, and thus pretty much useless before results have even been published. This paper argues that the solutions are already known (e.g. multicenter trials), but not implemented widely due to institutional inertia. Again, it is worth looking into how to facilitate such coordination, I believe that large cash grants by EA aligned institutions conditional on coordination between different trial sites could work.

Linch

Do you have thoughts on pandemic prevention NPIs (eg vector control)? Many of these things are technically non-pharmaceutical interventions, though of course looks very different from mask mandates or social distancing orders!

Davidmanheim

Key thought on vector control: vector control is tricky.
Mostly, we care about mosquitoes, where there is tons of work, and on mammalian carriers, where we know that people like farming / hunting, then eating the animals, so vector control is a bit different. There's lot of work on this, and a literature review for the forum might be a good thing for someone to write.

Anon-biosec-account

I haven't thought much about this so can't add anything useful at the moment. If I think of / come across anything I'll reply again.

Comments

Curated and popular this week

Was Partisanship Good for the Environmental Movement?

Jeffrey Heninger·2y ago·Curated 3d ago·6m read

This is the third in a sequence of posts taken from my recent report: Why Did Environmentalism Become Partisan? Summary Rising partisanship did not make environmentalism more popular or politically effective. Instead, it saw flat or falling overall public opinion, fewer major legislative achievements, and fluctuating executive actions. Public Opinion...

127

Possible mistake EAs are making and shout out to Pause AI UK

Michelle_Hutchinson·4d ago·4m read

I think right now EAs might be making a significant mistake by paying insufficient attention to the political realm. As EAs we tend to figure out what’s most impactful for us to work on and focus hard. That’s great! But there are various actions that are ‘non-delegatable’ - the extent to which an individual can do the action is limited (like voting, going to a protest, making hard money contributions to particular campaigns). It might be useful if we were all more in the habit of doing variou...

102

New Video from AI in Context: The Fall and Rise of Sam Altman

ChanaMessinger, phoebe b, Aric Floyd·6d ago·3m read

New Video from AI in Context: The Fall and Rise of Sam Altman If you want to skip straight to the video, here it is! AI in Context is excited to be back with our fourth video! For those just hearing from us, we make videos for 80,000 Hours, telling stories about transformative AI...

Recent opportunities to take action

127

Possible mistake EAs are making and shout out to Pause AI UK

Michelle_Hutchinson·4d ago·4m read

Build a flourishing EA group at the University of Toronto

Joseph Kostousov, Sophia Wan (navarhontes)·1w ago·1m read

How Africa Can (and Must) Skip the 30-Year Animal Welfare Evolution

Jacob Ayang, Cheslyn Ceaser·2w ago·12m read

Anon-biosec-account

However, I do have some thoughts on why an RCT could still be worthwhile. I’m not saying these because I disagree with your points; I’m just providing some possible counterargument.

Learning: by introducing the filters not in an RCT, you are basically doing an experiment but losing the opportunity to learn from it. Even if it has been decided that filters should be introduced in all schools/offices (or whatever unit), it won’t normally be possible to install all of those in parallel. So there is a time where some offices have the filter and some don’t. As long as you can randomise this, you can take advantage of the differences in time for implementation in something like a stepped wedge cluster randomised trial. The effect could be analysed on an ongoing basis in a Bayesian analysis such that if there are large effects they would be detected early in the experiment and implementation of the remaining filters can be accelerated. If you are doing something like this across several interventions, this would help with deciding which to prioritise.
Cost-benefit: There are ~ 137,000 schools in the US. I don’t know how much it costs to install and maintain filtration systems, but I imagine it is not negligible. There are a lot more schools globally. Doing an RCT comparing e.g. air filtration to opening the windows could save quite a bit of money if it turns out that filtration systems don’t provide additional benefit.
Implementation and interaction with behaviour: even assuming that they do work, do people use them? Maybe the filtration is noisy so teachers turn it off; maybe they simply forget to turn it on. In medicine, even with drugs that demonstrably improve the patient’s condition, adherence is (to me) surprisingly low. Perhaps large rooms where people tend to congregate most cannot be adequately filtered, maybe the filtration system gives people a sense of security so they congregate more.

Non-pharmaceutical interventions in pandemic preparedness and response

Non-pharmaceutical interventions in pandemic preparedness and response

Key points

Feedback that might be useful

What I mean by evaluating NPIs

Background

Neglectedness

Trials

Funding

Discussion in EA

Scale/importance

Tractability

Some possible projects

Immediate/small scale (little/no funding required)

Medium scale (some funding needed)

Longer term (more funding needed)