Forethought[1] is a new AI macrostrategy research group cofounded by Max Dalton, Will MacAskill, Tom Davidson, and Amrit Sidhu-Brar.
We are trying to figure out how to navigate the (potentially rapid) transition to a world with superintelligent AI systems. We aim to tackle the most important questions we can find, unrestricted by the current Overton window.
More details on our website.
Why we exist
We think that AGI might come soon (say, modal timelines to mostly-automated AI R&D in the next 2-8 years), and might significantly accelerate technological progress, leading to many different challenges. We don’t yet have a good understanding of what this change might look like or how to navigate it. Society is not prepared.
Moreover, we want the world to not just avoid catastrophe: we want to reach a really great future. We think about what this might be like (incorporating moral uncertainty), and what we can do, now, to build towards a good future.
Like all projects, this started out with a plethora of Google docs. We ran a series of seminars to explore the ideas further, and that cascaded into an organization.
This area of work feels to us like the early days of EA: we’re exploring unusual, neglected ideas, and finding research progress surprisingly tractable. And while we start out with (literally) galaxy-brained schemes, they often ground out into fairly specific and concrete ideas about what should happen next. Of course, we’re bringing principles like scope sensitivity, impartiality, etc to our thinking, and we think that these issues urgently need more morally dedicated and thoughtful people working on them.
Research
Research agendas
We are currently pursuing the following perspectives:
* Preparing for the intelligence explosion: If AI drives explosive growth there will be an enormous number of challenges we have to face. In addition to misalignment risk and biorisk, this potentially includes: how to govern the development of new weapons of mass destr
I am a patent attorney and IP/IT legal advisor who is interested in how we can design a more rational system of incentivising the market to develop medical innovations that maximise health impact, by using "financial innovation" or a legal/contractual approach. I wanted to get some feedback from the EA community regarding my social enterprise project, which I hope may be interesting to the members.
This project is based on the principles of EA and my frustration with the lack of clinical trial evidence in support of many therapies that could be viable treatments or cures but lack private incentives to pay for large clinical trials (Phase II+) that would convince the broader medical community of their safety and efficacy (including new uses for off-patent/generic drugs, diets, and lifestyle interventions). I've called these "unmonopolisable therapies" because it is not possible (or uneconomic) to enforce a monopoly price using patents, which is currently the only way that that the private pharmaceutical industry can recover the costs of pre-clinical development and clinical trials. The result is these unmonopolisable therapies often lack clinical trial evidence in support, unrelated to their health impact potential, because grant funding is rarely available for large clinical trials (with some notable exceptions).
The following Medium article is a brief summary of the charitable/social enterprise project and what we are trying to achieve, namely, a matching and facilitation service to establish "pay for success" contracts (e.g. prizes and social impact bonds) for healthcare payers to pay impact investors for successful clinical trials that repurpose generic drugs or other unmonopolisable therapies. We have proposed raising a US$10-50m Covid Prize Fund and/or Social Impact Bond as a pilot because the obvious economic and health burden for payers/govts, which may encourage them to back this "new" approach: https://medium.com/@savvak/can-we-develop-new-affordable-medicines-without-patents-1032399cd428. This project is based on the topic of my LLM thesis which analysed how the current patent system fails to incentivise development of new uses for generic drugs and other "unmonopolisable therapies (see https://ir.canterbury.ac.nz/bitstream/handle/10092/9826/thesis_fulltext.pdf?sequence=1&isAllowed=y).
As far as I am aware, there are only a few non-profit organisations that are trying a similar "financial innovation" approach of using "pay for success contracts" to incentivise impact investors to reduce healthcare costs for healthcare payers by repurposing generic drugs (Cures within Reach in the US (https://cureswithinreach.org/reflections-on-the-approach-and-challenges-of-developing-social-impact-bonds-to-fund-drug-repurposing-clinical-trials-a-conversation-with-dr-rick-thompson), Mission: Cure in the US (https://mission-cure.org/), and Findacure in the UK (https://www.findacure.org.uk/the-rare-disease-drug-repurposing-social-impact-bond/). Nobody is focussed on using pay for success contracts to repurpose generic drugs to treat Covid-19. The main benefit of pay for success vs grant funding is that you are involving the private industry to crowdsource medical innovation which currently lacks market incentives. This would theoretically be more efficient than grants (or at least worth a try as there is also no risk for healthcare payers with the risk of failed clinical trials taken on by impact investors). This could potentially convince healthcare payers to back a much larger Prize Fund or Social Impact Bond than they would otherwise be willing to in return for "de-risking" these Phase II+ clinical trials. This also helps get over the "valley of death" between pre-clinical and applied clinical research. Our main bottleneck is developing a financial model to convince payers to back a pay for success contract and make "outcome payments" for successful clinical trials. We are looking for healthcare economists or anyone that has worked on developing financial models to justify funding by healthcare payers. Once we can convince a healthcare payer (or perhaps UHNWI) to put a price on successful Phase II+ clinical trials on the basis of health savings, setting up a fund of impact investors to repurpose generic drugs (and fund clinical trials for other unmonopolisable therapies) will be relatively easy.
I have recently relaunched my NZ-based charity, the Medical Prize Charitable Trust which has tax-free status (see crowdfundedcures.org). The intention is to incorporate a wholly-owned social enterprise and seek grant funding/investment and scale based on charging management/consulting fees for the matching and facilitation service. I am mostly indifferent as to where this vehicle will be incorporated, but would prefer UK or NZ as I have more experience in those jurisdictions.
Looks interesting! I think you might have some interest in MichaelA's shortform about impact certificates. I saw you mentioned some orgs that are in this space. You may also want to check out Dr. Aidan Hollis' paper, An Efficient Reward System for Pharmaceutical Innovation, and his organization which tries to pay for success, the Health Impact Fund.