Hi all,
I thought people might be interested in some of the policy work the Global Priorities Project has been looking into. Below I'm cross-posting some notes on one policy idea. I've talked to several people with expertise in biosafety and had positive feedback, and am currently looking into how best to push further on this (it will involve talking to people in the insurance industry).
In general quite a bit of policy is designed by technocrats and already quite effective. Some other areas are governed by public opinion which makes it very hard to have any traction. When we've looked into policy, we've been interested in finding areas which navigate between these extremes -- and also don't sound too outlandish, so that they have some reasonable chance of broad support.
I'd be interested in hearing feedback on this from EAs. Criticisms and suggestions also very much welcome!
---
Requiring liability insurance for dual-use research with potentially catastrophic consequences
These are notes on a policy proposal aimed at reducing catastrophic risk. They cover some of the advantages and disadvantages of the idea at a general level; they do not yet constitute a proposal for a specific version of the policy.
Research produces large benefits. In some cases it may also pose novel risks, for instance work on potential pandemic pathogens. There is widespread agreement that such 'dual use research of concern' poses challenges for regulation.
There is a convincing case that we should avoid research with large risks if we can obtain the benefits just as effectively with safer approaches. However, there do not currently exist natural mechanisms to enforce such decisions. Government analysis of the risk of different branches of research is a possible mechanism, but it must be performed anew for each risk area, and may be open to political distortion and accusations of bias.
We propose that all laboratories performing dual-use research with potentially catastrophic consequences should be required by law to hold insurance against damaging consequences of their research.
This market-based approach would force researcher institutions to internalise some of the externalities and thereby:
- Encourage university departments and private laboratories to work on safer research, when the benefits are similar;
- Incentivise the insurance industry to produce accurate assessments of the risks;
- Incentivise scientists and engineers to and devise effective safety protocols that could be adopted by research institutions to reduce their insurance premiums.
Current safety records do not always reflect an appropriate level of risk tolerance. For example, the economic damage caused by the escape of the foot and mouth virus from a BSL-3 or BSL-4 lab in Britain in 2007 was high (mostly through trade barriers) and could have been much higher (the previous outbreak in 2001 caused £8 billion of damage). If the lab had known they were liable for some of these costs, they might have taken even more stringent safety precautions. In the case of potential pandemic pathogen research, insurers might require it to take place in BSL-4 or to implement other technical safety improvements such as “molecular biocontainment”.
Possible criticisms and responses
- The potential risks are too large, and nobody would be willing to insure against them.
- We could avoid this by placing an appropriate limit on the amount of insurance that could be required. If it were a sufficiently large sum (perhaps in the billions of dollars), the effect should be more appropriate risk aversion, even if the tail risk for the insurer were not fully internalised.
- The risks are too hard to model, and nobody would be willing to insure against them.
- There are insurance markets for some risks that are arguably harder to model and have equally high potential losses, such as terrorism.
- We already have demanding safety standards, so this wouldn’t reduce risk.
- Much of the current safety standards are focused on occupational health and safety of the lab workers rather than the general public.
- The market-driven approach proposed would focus attention on whichever steps were rationally believed to have the largest effect on reducing risk, and reduce other bureaucratic hurdles.
- Liability has been useful at improving behaviour in other domains, for example in industrial safety.
- It is hard to draw a line around the harmful effects of research. Should we punish research which enables others to perform harmful acts?
- There is a hard question, but we think we would get much of the benefit by using the simple rule that labs are only liable for simple direct consequences of their work. For example, the release – accidental or deliberate – of a pathogenic virus manufactured in that lab.
- Research has positive externalities, and it is unfair if they have to internalise only the negative ones.
- This is true, although research receives funding for precisely this reason.
- If we don’t make an attempt to introduce liability then we are effectively subsidising unsafe research relative to safe research.
- Why require insurance rather than just impose liability? Shouldn’t this be a decision for the individuals?
- Some work may be sufficiently risky that the actors cannot afford to self-insure. In such circumstances it makes sense to require insurance (just as we require car insurance for drivers).
- This will help to ensure that appropriate analysis of the risks is performed.
- Which research should this apply to? How can we draw a line?
- The bureaucracy would be too costly to impose this requirement on all research. We should adapt existing guidelines for which areas need extra oversight. In the first place, potential pandemic pathogen research is an obvious area which should be included.
Excellent idea! The rest of this comment is going to be negative, but my balance of opinion is not reflected in the balance of words.
One potential downside is that dangerous research would move to other countries. However, this effect would be reduced by the dominance of the anglosphere in many areas of research. Additionally, some research with the potential to cause local but not global disasters represents a national but not international externality, in which case other countries are also appropriately incentivised to adopt sensible precautions. So on net this does not seem to be a very big concern.
Another is the lack of any appreciation for public choice in this argument. Yes, I agree this policy would be good if implemented exactly as described. But policies that are actually implemented rarely bare much resemblance to the policies originally advocated by economists. Witness the huge gulf between the sort of financial reform that anyone actually advocates and Dodd-Frank, which as far as I'm aware satisfied basically no-one who was familiar with it. The relevant literature is the public choice literature. So here are some ways this could be misimplimented:
Drivers are generally individuals, whereas research is generally done by institutions. It seems plausible to me that creditworthy institutions/individuals should not have to take out car insurance. If Oxford faced a potential liability in the billions, I'm sure it would insure. I guess the main threat comes from small, limited liability institutions whose only purpose is to do this one kind of research, and are thus unconcerned with the downside. Or large institutions with poor internal governance.
I agree that in general it's fine for creditworthy institutions to self-insure. The issue is that the scale of possible liability is large enough (billions of dollars, perhaps hundreds of billions of dollars) that even institutions which routinely self-insure against all risks as a matter of course may not be creditworthy against the worst outcomes. In some cases they are explicitly or implicitly state-backed, but if nobody in the chain has considered the possible liability you don't get the proper incentive effects. If there were a market so that the risk of the research were priced, I'd expect better governance even at institutions which self-insured.
I agree.
I agree that there are some issues regarding the version of the policy that would actually be implemented. This is a large factor for requiring insurance rather than direct state regulation, and I think this offers a robustness which goes some way towards defusing your concerns.
For example:
If there's just an insurance requirement, it's hard for extra costs to swell much above the true expected externalities (if it's safe, they should be able to find someone willing to insure it cheaply).
Yup, I agree again. Though there is still the risk that the political system might manufacture externalities to accuse the researchers of.
The managers of Harvard's endowment circa 2008 would beg to differ, I think. (It lost about $10 billion, nearly a third of its value.)
It seems like for some of these institutions, how long of a view they take is substantially determined by contingent factors like who's the university president at the time.
I'm sorry, I don't quite understand your point. There's a huge difference between investment risk, for which you are paid the equity risk premium, and the sorts of things people insure against.