Forethought[1] is a new AI macrostrategy research group cofounded by Max Dalton, Will MacAskill, Tom Davidson, and Amrit Sidhu-Brar.
We are trying to figure out how to navigate the (potentially rapid) transition to a world with superintelligent AI systems. We aim to tackle the most important questions we can find, unrestricted by the current Overton window.
More details on our website.
Why we exist
We think that AGI might come soon (say, modal timelines to mostly-automated AI R&D in the next 2-8 years), and might significantly accelerate technological progress, leading to many different challenges. We don’t yet have a good understanding of what this change might look like or how to navigate it. Society is not prepared.
Moreover, we want the world to not just avoid catastrophe: we want to reach a really great future. We think about what this might be like (incorporating moral uncertainty), and what we can do, now, to build towards a good future.
Like all projects, this started out with a plethora of Google docs. We ran a series of seminars to explore the ideas further, and that cascaded into an organization.
This area of work feels to us like the early days of EA: we’re exploring unusual, neglected ideas, and finding research progress surprisingly tractable. And while we start out with (literally) galaxy-brained schemes, they often ground out into fairly specific and concrete ideas about what should happen next. Of course, we’re bringing principles like scope sensitivity, impartiality, etc to our thinking, and we think that these issues urgently need more morally dedicated and thoughtful people working on them.
Research
Research agendas
We are currently pursuing the following perspectives:
* Preparing for the intelligence explosion: If AI drives explosive growth there will be an enormous number of challenges we have to face. In addition to misalignment risk and biorisk, this potentially includes: how to govern the development of new weapons of mass destr
Oh hey, I wrote a blog post (sorta) about this.
The TLDR: Since I was a teenager I've been looking for ways to give effectively, and once GiveWell appeared doing so became a whole lot easier.
I was interested in most of the relevant cause areas in some form from childhood (the global poor, animal welfare, extinction risks), and independently formulated utilitarianism (not uncommon I’m told, both Bertrand Russell and Brian Tomasik apparently did the same), so I was a pretty easy sell.
I was assigned “Famine, Affluence, and Morality” and “All Animals are Equal” for a freshman philosophy course, and decided Peter Singer really got it and did philosophy in the way that seemed most important to me. Later I revisited Singer when working on a long final paper about animal rights and I ran into his TED talk on Effective Altruism.
At first I was sympathetic but not that involved, but gradually realized that it was much more the style of ethics/activism I was interested in promoting than the other things on the table, or at least on top of them. I founded my school’s Effective Altruism club while I still didn’t really know all that much about the movement, and started learning more, especially after a friend (Chris Webster) recommended the 80,000 Hours podcast to me.
Around this same time I read Reasons and Persons, and met my friend and long-time collaborator Nicholas Kross who introduced me to many rationalist ideas and thinkers, and by the end of undergrad, I was basically a pretty doctrinaire, knowledgeable EA. Kind of a long story, but the whole thing was pretty much in fits and starts so I don’t know a great way to compress it.
Here's mine: My Effective Altruism Timeline (2013)
I just find it delightful that that HPMOR is the start of so many people's EA origin story, partly just as a curiosity as I had an opposite path to so many people (AMF > EA > LW > HPMOR)
Presumably there are many people alive today because of a chain of events started with EY writing a fanfic of all things.
That's kind of wonderful to think about - and I think it's hilarious that you have the opposite journey!