Forethought[1] is a new AI macrostrategy research group cofounded by Max Dalton, Will MacAskill, Tom Davidson, and Amrit Sidhu-Brar.
We are trying to figure out how to navigate the (potentially rapid) transition to a world with superintelligent AI systems. We aim to tackle the most important questions we can find, unrestricted by the current Overton window.
More details on our website.
Why we exist
We think that AGI might come soon (say, modal timelines to mostly-automated AI R&D in the next 2-8 years), and might significantly accelerate technological progress, leading to many different challenges. We don’t yet have a good understanding of what this change might look like or how to navigate it. Society is not prepared.
Moreover, we want the world to not just avoid catastrophe: we want to reach a really great future. We think about what this might be like (incorporating moral uncertainty), and what we can do, now, to build towards a good future.
Like all projects, this started out with a plethora of Google docs. We ran a series of seminars to explore the ideas further, and that cascaded into an organization.
This area of work feels to us like the early days of EA: we’re exploring unusual, neglected ideas, and finding research progress surprisingly tractable. And while we start out with (literally) galaxy-brained schemes, they often ground out into fairly specific and concrete ideas about what should happen next. Of course, we’re bringing principles like scope sensitivity, impartiality, etc to our thinking, and we think that these issues urgently need more morally dedicated and thoughtful people working on them.
Research
Research agendas
We are currently pursuing the following perspectives:
* Preparing for the intelligence explosion: If AI drives explosive growth there will be an enormous number of challenges we have to face. In addition to misalignment risk and biorisk, this potentially includes: how to govern the development of new weapons of mass destr
Is it really the case that the UK and US were competing for the gains to reputation that foreign aid brings? I suppose I’d try to answer that question by looking at the history of where the 0.7% target, which I thought was fairly broadly shared among rich countries, originally came from. One history I found said:
> It results from the 1970 United Nations General Assembly Resolution 2626. The 0.7% figure was calculated as a means to boost growth for developing countries. Since 1970, however, only several Nordic countries have met or surpassed this target.
Which makes it look like a potential failed case of cooperation: that no country wants to have the whole cost of international aid fall just on itself, so they try to establish international agreements so that others contribute too. Is that the right model, though? Perhaps countries don’t really experience pressure to adjust their contribution upwards when other countries drop out, and are happy to just continue doing the amount they were doing. I can think of a possible dynamic where ‘preventing bad runaway spillover effects like the spread of war and pandemics’ costs X billion dollars of aid (and has a threshold effect, where providing half of X billion is much less than half as effective as X), which no individual country is willing to stump up, but which is important to every rich country’s interest. In that dynamic, rich countries benefit from such UN-induced cooperation, by spreading the burden and getting the benefits of less instability and disease in the world.
If that’s the dynamic, then your idea of a race to the bottom in aid (a dis-‘alm’-ament, you could say) would not necessarily be what you expect after some country drops out — they’d possibly want to raise their contributions to make up the difference and still be able to reach the threshold of X. Alternatively, the UK now might expect that without the US it will be too difficult to reach that hypothetical threshold, and so there’s no point trying any more.
But if the incentives / dynamic of foreign aid is more about getting prestige for your country, then you could see a broad disalmament when the total pool of funding takes a hit — or alternatively you could see the second or third place players boosting their contribution to try and take the lead. So I’m not sure what determines what one would expect to see given this potential reputation-based dynamic.