I trust my kids and grandkids to solve their own problems in the future.

I don't trust our generation to make sure our kids and grandkids survive.

Avoiding extinction is the urgent priority; all else can wait. (And, life is already getting better at a rapid rate for the vast majority of the world's people. We don't face any urgent or likely extinction risks other than technologies of our own making.)

I don't think this is as clear of a dichotomy as people think it is. A lot of global catastrophic risk doesn't come from literal extinction because human extinction is very hard. A lot of mundane work on GCR policy involves a wide variety of threat models that are not just extinction.

Extinction is not so bad in comparison to futures where the world possibly has lots of intense suffering. We should focus resources on preventing suffering and ensuring wellbeing.

X-risk reduction (especially alinement) is highly neglected and it's less clear how our actions can impact the value of the future. However, I think the impact of both is very uncertain and I still think working on s-risk reduction and long-termest animal work is of high impact. 

Avoiding extinction seems bad to me if we never get our shit together morally-civilizationally, and good otherwise. Conditional on survival, I'm not sure of the likelihood of us getting our shit together to be a force of good in the universe, mostly because I'm uncertain how AGI will play out.

Most people have a strong drive to perpetuate humanity. What makes EAs special is that EAs also care about others' suffering. So EAs should focus on trying to make sure the future isn't full of suffering.

The value of the future conditional on civilization surviving seems positive to me, but not robustly so. I think the main argument for its being positive is theoretical (e.g., Spreading happiness to the stars seems little harder than just spreading), but the historical/contemporary record is ambiguous.

The value of improving the future seems more robustly positive if it is tractable. I suspect it is not that much less tractable than extinction risk work. I think a lot of AI risk satisfies this goal as well as the x-risk goal for reasons Will MacAskill gives in What We Owe the Future. Understanding, developing direct interventions for, and designing political processes for digital minds seem like plausible candidates. Some work on how to design democratic institutions in the age of AI also seems plausibly tractable enough to compete with extinction risk.

working on valuable futures is only worth it if we are not extinct

More tractable, necessary precondition 

I genuinely have no idea. 

I'm a negative utilitarian, meaning I believe reducing suffering is the only thing that truly matters. Other things are important only in so far as they help to reduce suffering. I'm open to debate on this ethical view.

Given this premise:

• There's nothing bad about extinction itself.
• There is, however, a lot of bad if we live in a future full of suffering.

Therefore, we should focus on increasing the quality of of life of living beings, rather than simply prolong our existence.

If we treat digital minds like current animal livestock, the expected value of the future could be really bad. 

I think people overrate how predictable the effect of our actions on the future will be (even though they rate it very low in absolute terms); extinction seem like one of the very few (only?) things that seems like its effects will endure throughout a big part of the future. Still buy the theory that 0-1% of possible value is equally valuable to 98-99%; just about tractability

I'm leaning to disagree because existential risks are a lot broader than extinction risk.

If the question replaced 'extinction' with 'existential' and 'survive' with 'thrive' (retain most value of the future), I would lean towards agree!

The world is pretty good, and the future can be better, if we get the chance to make it so.

The big reason I lean towards disagreeing nowadays is coming to the belief that I expect the AI control/alignment problem to be much less neglected and important to solve, and more generally I've come to doubt the assumption that worlds in which we survive are worlds in which we achieve very large value (under my own value set), such that reducing existential risk is automatically good.

I'm working on trying to describe the environmental future we want to have, so I'll have written up a better answer soon.™️

We know our past and how species have gone extinct. We see the present. It's the future that is uncertain. If we keep looking forward and forget the pot of burning hot tea about to spill the whole table, we will get burned, and badly too. and have a whole ass burnt up body for the future

alien counterfactuals

morally ambivalent about extinction, though reducing chances of extinction could plausibly also have positive flowthrough for S-Risks. 

I roughly feel more comfortable passing the responsibility onto wiser successors. I still like the "positive vs negative longtermism" framework, I think positive longtermism (increasing the value of futures where we survive) risks value lock-in too much. Negative longtermism is a clear cut responsibility with no real downside unless you're presented with a really tortured example about spending currently existing lives to buy future lives or something. 

Id rather have a shitty human society then none at all. A little is better then a lot of progres. 

I think there are a lot of thorny definitional issues here that make this set of issues not boil down that nicely to a 1D spectrum. But overall extinction prevention will likely have a far broader coalition supporting it, while making the future large and amazing is far less popular since most people aren't very ambitious with respect to spreading flourishing through the universe, but I tentatively am.

Far from convinced that continued existence at currently likely wellbeing levels is a good thing

Nonexistence is preferable to intense suffering, and I think there are enough S-risks associated with the array of possible futures ahead of us that we should prioritize reducing S-risks over X-risks, except when reducing X-risks is instrumental to reducing S-risks. So to be specific, I would only agree with this to the extent that "value" == lack of suffering -- I do not think we should build for the utopia that might not come to pass because we wipe ourselves out first, just that it is vastly more important to prevent dystopia 

mostly because of tractability than any other reason

This is a difficult one, and both my thoughts and my justifications (especially the few sources I cite) are very incomplete. 

It seems to me for now that existential risk reduction is likely to be negative, as both human and AI-controlled futures could contain immense orders of magnitude more suffering than the current world (and technological developments could also enable more intense suffering, whether in humans or in digital minds). The most salient ethical problems with the extinction of earth-originating intelligent life seem to be the likelihood... (read more)

Recent advances in LLMs have led me to update toward believing that we live in the world where alignment is easy (i.e. CEV naturally emerge from LLMs, and future AI agents will be based on understanding and following natural language commands by default), but governance is hard (i.e. AI agents might be co-opted by governments or corporations to lock in humanity in a dystopian future, and the current geopolitical environment, characterized by democratic backsliding, cold war mongering, and an increase in military conflicts including wars of aggression, isn't conducive to robust multilateral governance).

I think the expected value of the long-term future, in the “business as usual” scenario, is positive. In particular, I anticipate that advanced/transformative artificial intelligence drives technological innovation to solve a lot of world problems (e.g., helping create cell-based meat eventually), and I also think a decent amount of this EV is contained in futures with digital minds and/or space colonization (even though I’d guess it’s unlikely we get to that sort of world). However, I’m very uncertain about these futures—they could just as easily contain ... (read more)

I'm optimistic about the very best value-increasing research/interventions. But in terms of what would actually be done at the margin, most work that people would do for "value-increasing" reasons would be confused/doomed, I expect (and this is less true for AI safety).

Tepidly disagree — I think the technological developments, like AI, which would raise the spectre of extinction are far more contingent than we would like to believe.

Yes, because it seems like extinction or near-extinction is a major possibility.

Reducing x risk is much less tractable than EAs think

Essentially the Brian Kateman view: civilisation's valence seems massively negative due to farmed animal suffering. This is only getting worse despite people being able to change right now. There's a very significant chance that people will continue to prefer animal meat, even if cultured meat is competitive on price etc. "Astronomical suffering" is a real concern.

I have to push back against the premise, as the dichotomoy seems a bit too forced. There are different ways to ensure survival, such as billionaires building survival shelters and hoarding resources vs. collective international efforts to solve our biggest problems. Working on better futures also usually involves creating more resilient institutions that will be better suited towards preventing extinction; we don't just magically pop into a future.

We can adjust the risk per unit of reward or the reward per unit of risk. 

In the absence of credible, near-term, high-likelihood existential risks and in the absence of being path-locked on an existential trajectory, I would rather adjust the reward per unit of risk. 

I also suspect that the most desirable paths to improving the value of futures where we survive will come with a host of advancements that allow us to more effectively combat risks anyway. Yes, I'm sure there are some really dumb ways to improve the value of futures, such that we're ... (read more)

it's easy to be wrong about what futures are more valuable, but it's pretty clear that extinction closes off all future options of creating value.

Under moral uncertainty, many moral perspectives care much more about averting downsides than producing upsides.

Additionally, tractability is probably higher for extinction-level threats, since they are "absorptive"; decreasing the chance we end up in one gives humanity and their descendants ability to do whatever they figure out is best.

Finally, there is a meaningful sense in which working on improving the future is plagued by questions about moral progress and lock-in of values, and my intuition is that most interventions that take moral progress serious... (read more)

Question seems like a false dichotomy.
For example, Democracy promotion sure seems super important right now and that would help both described causes (and it isn't clear which would be helped more). 

The salient question for me is how much does reducing extinction risk change the long run experience of moral patients? One argument is that meaningfully reducing risk would require substantial coordination, and that coordination is likely to result in better worlds. I think it is as or more likely that reducing extinction risk can result in some worlds where most moral patients are used as means without regard to their suffering.

I think an AI aligned to roughly to the output of all current human coordination would be net-negative. I would shift to thinkin... (read more)

I think that without knowing people's assessment of extinction risk (e.g. chance of extinction over the next 5, 10, 20, 50, 100 years)^[1], the answers here don't provide a lot of information value. 

I think a lot of people on the disagree side would change their mind if they believed (as I do) that there is a >50% chance of extinction in the next 5 years (absent further intervention).

Would be good if there was a short survey to establish such background assumptions to people's votes.

It's a tractability issue. In order for these interventions to be worth funding, they should reduce our chance of extinction not just now, but over the long term. And I just haven't seen many examples of projects that seem likely to do that.

I find it very difficult to determine whether the future will be net-negative or net-positive (when considering humans, factory-farmed animals, wild animals, and possibly artificial sentience). 
This makes it very hard to know whether work on extinction reduction is likely to be positive or not.
I prefer to work on things that aim to move the sign towards "net-positive".

Very worried about AI risk, think short timelines are plausible

Intelligence is the only chance of some redemption to the massive suffering probably associated to the emergence of consciousness. 

This is the age of danger, we are the first species on Earth that has figured out morality, so we shall survive at allmost all cost. 

I think human extinction (from ASI) is highly likely to happen, and soon, unless we stop it from being built^[1]

See my comments in the Symposium for further discussion

It seems even more important to avoid futures full of extreme suffering than to avoid extinction.

Survival feels like a very low bar to me. 

Survival could mean the permanent perpetuation of extreme suffering, human disempowerment, or any number of losses of our civilization's potential.

I really don't have a strong view, but I find myself sympathetic to the idea that the world is not great and is getting worse (if you consider non-human animals).

Broadly Agree

Although I might have misunderstood and missed the point of this entire debate, so correct me if that is the case

I just don't believe  changing the future trajectory is tractable in say 50-100 years from now areas like politics, economics, AI welfare etc. I think its a pipe dream. We cannot predict technological, political and economic changes even in the medium term future. These changes may well quickly render our current efforts meaningless in 10-20 years. I think the effect of work we do now which is future focused diminishes in value... (read more)