Most suffering-focused EAs I know agree about the facts: there's a small chance that AI-powered space colonization will create flourishing futures highly optimized for happiness and other forms of moral value, and this small chance of a vast payoff dominates the expected value of the future on many moral views. I think people generally agree that the typical/median future scenario will be much better than the present (for reasons like this one, though there's much more to say about that), though in absolute terms probably not nearly as good as it could be. 

So in my perception, most of the disagreement comes from moral views, not from perceptions of the likelihood or severity of s-risks.

Great question! I think both moral and factual disagreements play a significant role. David Althaus suggests a quantitative approach of distinguishing between the “N-ratio”, which measures how much weight one gives to suffering vs. happiness, and the “E-ratio”, which refers to one’s empirical beliefs regarding the ratio of future happiness and suffering. You could prioritise s-risk because of a high N-ratio (i.e. suffering-focused values) or because of a low E-ratio (i.e. pessimistic views of the future).

That suggests that moral and factual disagreements are comparably important. But if I had to decide, I’d guess that moral disagreements are the bigger factor, because there is perhaps more convergence (not necessarily a high degree in absolute terms) on empirical matters. In my experience, many who prioritise suffering reduction still agree to some extent with some arguments for optimism about the future (although not with extreme versions, like claiming that the ratio is “1000000 to 1”, or that the future will automatically be amazing if we avoid extinction). For instance, if you were to combine my factual beliefs with the values of, say, Will MacAskill, then I think the result would probably not consider s-risks a top priority (though still worthy of some concern).

In addition, I am increasingly thinking that “x-risk vs s-risk” is perhaps a false dichotomy, and thinking in those terms may not always be helpful (despite having written much on s-risks myself). There are far more ways to improve the long-term future than this framing suggests, and we should look for interventions that steer the future in robustly positive directions.

Strong-upvoted this question. Follow-up question: what kind of research could resolve any factual disagreements?

I'm also interested in answers to this question. I'd add the following nit-picky points: 

• X-risks and s-risks are substantially overlapping categories (in particular, many unrecoverable dystopia scenarios also contain astronomical suffering), so it's possible a more fruitful framing is prioritisation of s-risks vs other x-risks, s-risks in particular vs x-risks as a whole, or s-risks vs extinction risks.
• There could also be other moral or factual disagreements that help explain differences in the extent to which different longtermists prioritise s-risks relative to other x-risks.
  • In particular, I tentatively suspect that there's a weak/moderate correlation between level of prioritisation of s-risks and level of moral concern for nonhuman animals. 
    • If this correlation exists, I expect it'd be partly a factual disagreement about sentience (and thus arguably a factual disagreement about the relative magnitudes of s- and x-risks),  but also partly a moral disagreement about how much moral weight/moral status animals warrant. 
    • And I'd expect the correlation to partly be "mere correlation", and partly a causal factor in these prioritisation decisions.

Concerning how EA views on this compare to the views of the general population, I suspect they aren’t all that different. Two bits of weak evidence:

I.

Brian Tomasik did a small, admittedly unrepresentative and imperfect Mechanical Turk survey in which he asked people the following:

At the end of your life, you'll get an additional X years of happy, youthful, and interesting life if you first agree to be covered in gasoline and burned in flames for one minute. How big would X have to be before you'd accept the deal?

More than 40 percent said that they would not accept it “regardless of how many extra years of life” they would get (see the link for some discussion of possible problems with the survey).

II.

The Future of Life Institute did a Superintelligence survey in which they asked, “What should a future civilization strive for?” A clear plurality (roughly a third) answered “minimize suffering” — a rather different question, to be sure, but it does suggest that a strong emphasis on reducing suffering is very common.

1. Do you know about any good articles etc. that make the case for such views?

I’ve tried to defend such views in chapter 4 and 5 here (with replies to some objections in chapter 8). Brian Tomasik has outlined such a view here and here.

But many authors have in fact defended such views about extreme suffering. Among them are Ingemar Hedenius (see Knutsson, 2019); Ohlsson, 1979 (review); Mendola, 1990; 2006; Mayerfeld, 1999, p. 148, p. 178; Ryder, 2001; Leighton, 2011, ch. 9; Gloor, 2016, II.

And many more have defended views according to which happiness and suffering are, as it were, morally orthogonal.

2. Do you think such or similar views are necessary to prioritize S-Risks?

As Tobias said: No. Many other views can support such a priority. Some of them are reviewed in chapter 1, 6, and 14 here.

3. Do you think most people would/should vote in such a way if they had enough time to consider the arguments?

I say a bit on this in footnote 23 in chapter 1 and in section 4.5 here.

4 For me it seems like people constantly trade happiness for suffering ... Those are reasons for me to believe that most people ... are also far from expecting 1:10^17 returns or even stating there is no return which potentially could compensate any kind of suffering.

Many things to say on this. First, as Tobias hinted, acceptable intrapersonal tradeoffs cannot necessarily be generalized to moral interpersonal ones (cf. sections 3.2 and 6.4 here). Second, there is the point Jonas made, which is discussed a bit in section 2.4 in ibid. Third, tradeoffs concerning mild forms of suffering that a person agrees to undergo do not necessarily say much about tradeoffs concerning states of extreme suffering that the sufferer finds unbearable and is unable to consent to (e.g. one may endorse lexicality between very mild and very intense suffering, cf. Klocksiem, 2016, or think that voluntarily endured suffering occupies a different moral dimension than does suffering that is unbearable and which cannot be voluntarily endured). More considerations of this sort are reviewed in section 14.3, “The Astronomical Atrocity Problem”, here.

4 For me it seems like people constantly trade happiness for suffering (taking drugs expecting a hangover, eating unhealthy stuff expecting health problems or even just feeling full, finishing that show on Netflix instead of going to sleep… ). Those are reasons for me to believe that most people might not want to compensate suffering through happiness 1:1 , but are also far from expecting 1:10^17 returns or even stating there is no return which potentially could compensate any kind of suffering.

One counterargument that has been raised against this is that people just accept suffering in order to avoid other forms of suffering. E.g., you might feel bored if you don't take drugs, might have uncomfortable cravings for unhealthy food if you don't eat it, etc.

I do think this point could be part of an interesting argument, but as it stands, it merely offers an alternative explanation without analyzing carefully which of the two explanations is correct. So on its own, this doesn't seem to be a strong counterargument yet.

I don’t think this view is necessary to prioritise s-risk. A finite but relatively high “trade ratio” between happiness and suffering can be enough to focus on s-risks. In addition, I think it’s more complicated than putting some numbers on happiness vs. suffering. (See here for more details.) For instance, one should distinguish between the intrapersonal and the interpersonal setting - a common intuition is that one man’s pain can’t be outweighed by another’s pleasure.

Another possibility is lexicality: one may contend that only certain particularly bad forms of suffering can’t be outweighed. You may find such views counterintuitive, but it is worth noting that lexicality can be multi-dimensional and need not involve abrupt breaks. It is, for instance, quite possible to hold the view that 1 minute of lava is ‘outweighable’ but 1 day is not. (I think I would not have answered “no amount can compensate” if it was about 1 minute.)

I also sympathise with the view mentioned by Jonas: that happiness matters mostly in so far as an existing being has a craving or desire to experience it. The question, then, is just how strong the desire to experience a certain timespan of bliss is. The poll was just about how I would do this tradeoff for myself, and it just so happens that abstract prospects of bliss does not evoke a very strong desire in me. It’s certainly not enough to accept a day of lava drowning - and that is true regardless of how long the bliss lasts. Your psychology may be different but I don’t think there’s anything inconsistent or illogical about my preferences.

I would refer to this elaborate comment by Magnus Vinding on a very similar question. Like Magnus, I think a common misconception is that suffering-focused views have certain counterintuitive or even dangerous implications (e.g. relating to world destruction), when in fact those problematic implications do not follow.

Suffering-focused ethics is also still sometimes associated with negative utilitarianism (NU). While NU counts as a suffering-focused view, this often fails to appreciate the breadth of possible suffering-focused views, including pluralist and even non-consequentialist views. Most suffering-focused views are not as ‘extreme’ as pure negative utilitarianism and are far more compatible with widely shared moral intuitions. (Cf. this recent essay for an overview.)

Last, and related to this, there is a common perception of suffering-focused views as unusual or ‘fringe’, when they in fact enjoy significant support (in various forms).

Apart from the normative discussions relating to the suffering focus (cf. other questions), I think the most likely reasons are that s-risks may simply turn out to be too unlikely, or too far in the future for us to do something about it at this point. I do not currently believe either of those (see here and here for more), and hence do work on s-risks, but it is possible that I will eventually conclude that s-risks should not be a top priority for one of those reasons.

Paul Christiano talks about this question in his 80,000 Hours podcast, mainly saying that s-risks seem less tractable than AI alignment (but also expressing some enthusiasm for working on them).

We have thought about this, and wrote up some internal documents, but have not yet published anything (though we might do that at some point, as part of a strategic plan). Magnus and I are quite aligned in our thinking about the theory of change. The key intended outcome is to catalyse a research project on how to best reduce suffering, both by creating relevant content ourselves and by convincing others to share our concerns regarding s-risks and reducing future suffering.

One key difference is that there is less money in it, because OpenPhil as the biggest EA grantmaker is not focused on reducing s-risks. In a certain sense, that is good news because work on s-risks is plausibly more funding-constrained than non-suffering-focused longtermism.

In terms of where to donate, I would recommend the Center on Long-Term Risk and the Center for Reducing Suffering (which I co-founded myself). Both of those organisations are doing crucial research on s-risk reduction. If you are looking for something a bit less abstract, you could consider Animal Ethics, the Good Food Institute, or Wild Animal Initiative.

I agree that s-risks can vary a lot (by many orders of magnitude) in terms of severity. I also think that this gradual nature of s-risks is often swept under the rug because the definition just uses a certain threshold (“astronomical scale”). There have, in fact, been some discussions about how the definition could be changed to ameliorate this, but I don’t think there is a clear solution. Perhaps talking about reducing future suffering, or preventing worst-case outcomes, can convey this variation in severity more than the term ‘s-risks’.

Regarding your second question, I wrote up this document a while ago on whether we should focus on worst-case outcomes, as opposed to suffering in median futures or 90th-percentile-badness-futures (given that those are more likely than worst-cases). However, this did not yield a clear conclusion, so I consider this an open question.

One is the Hedonistic Imperative/suffering abolition: using biotechnology to modify sentient life to not suffer (or not suffer significantly, and ensuring artificial sentience does not suffer?). David Pearce, a negative utilitarian, is the founding figure for this.

I think a plausible win condition is that society has some level moral concern for all sentient beings (it doesn’t necessarily need to be entirely suffering-focused) as well as stable mechanisms to implement positive-sum cooperation or compromise. The latter guarantees that moral concerns are taken into account and possible gains from trade can be achieved. (An example for this could be cultivated meat, which allows us to reduce animal suffering while accommodating the interests of meat eaters.)

However, I think suffering reducers in particular should perhaps not focus on imagining best-case outcomes. It is plausible (though not obvious) that we should focus on preventing worst-case outcomes rather than shooting for utopian outcomes, as the difference in expected suffering between a worst-case and the median outcome may be much greater than the difference between the median outcome and the best possible future.

I was exposed to arguments for suffering-focused ethics from the start, since I was involved with German-speaking EAs (the Effective Altruism Foundation / Foundational Research Institute back then). I don’t really know why exactly (there isn’t much research on what makes people suffering-focused or non-suffering-focused), but this intuitively resonated with me.

I can’t point to any specific arguments or intuition pumps, but my views are inspired by writing such as the Case for Suffering-Focused Ethics, Brian Tomasik’s essays, and writings by Simon Knutsson and Magnus Vinding.

Re: 1., there would be many more (thoughtful) people who share our concern about reducing suffering and s-risks (not necessarily with strongly suffering-focused values, but at least giving considerable weight to it). That results in an ongoing research project on s-risks that goes beyond a few EAs (e.g., it is also established in academia or other social movements).

Re: 2., a possible scenario is that suffering-focused ideas just never gain much traction, and consequently efforts to reduce s-risks will just fizzle out. However, I think there is significant evidence that at least an extreme version of this is not happening.

Re: 3., I think the levels of engagement and feedback we have received so far are encouraging. However, we do not currently have any procedures in place to measure impact, which is (as you say) incredibly hard for what we do. But of course, we are constantly thinking about what kind of work is most impactful!

I would guess that actually experiencing certain possible conscious states, in particular severe suffering or very intense bliss, could significantly change my views, although I am not sure if I would endorse this as “reflection” or if it might lead to bias.

It seems plausible (but I am not aware of strong evidence) that experience of severe suffering generally causes people to focus more on it. However, I myself have fortunately never experienced severe suffering, so that would be a data point to the contrary.

Thanks! I've started an email thread with you, me, and David.