• How does XR weigh costs and benefits?
  • I don't think there's a uniform "existential risk reduction movement" answer to what % of X-risk reduction people are willing to trade for large harms today, any more than there's a uniformly "effective altruism movement" answer for what the limits of altruism ought to be. For me personally, all the realistic tradeoffs I make day to day point towards reducing X-risk being of overwhelming importance, aside from model uncertainty.
    • In practice, this means I try to spend my time thinking about longtermism stuff, with mixed success, aside from a) stuff that builds my own skills or other ways that I can be more effective, b) small cooperative actions that I think  are unusually good to values systems otherwise fairly similar to my own, c) entertainment, and d) bullshit.
    • I think probability of existential risk is high enough that I'm not in practice too worried about Pascal's mugging issues. In the moments I doubt whether EA longtermism is stuff I ought to be working on, I'm much more worried about issues in the class of "randomly or systematically deluding myself about the probabilities of specific outcomes I'm worried about." Rather than issues akin to "my entire decision procedure is wrong because expected value calculations are dumb."
• Does XR consider tech progress default-good or default-bad?
  • Basically my view is similar to what JP said. From an XR perspective, the sign of faster tech is pretty hard to tell. There are some weak arguments in favor of it being systematically good (eg that fast technological progress reduces natural risks over the long run, while not being systematically one direction or another for man-made risks). But this is pretty weak if we think the base rate of natural risks is very low. But of course I don't just have an XR perspective, and from other perspectives (as JP mentions) the benefits of technological and economic progress are clearer.
    • As an aside, I think trying to shoehorn in "belief in progress" from an xrisk perspective is kinda dubious. Analogously, I think it'd be dumb to sell kids on the value of learning music because of purported benefits to transfer learning to mathematics standardized tests. Sure there may or may not be some cognitive benefits to learning music, but that's not why I got into music, music is hardly the most efficient way to learn mathematics, and it'd be dubious if you primarily got into music to boost your math test scores.
• What would moral/social progress actually look like?
  • I think a lot of the types of progress that some subset of EAs (including myself) are interested in is "specific technical or organizational solutions to risks that we think of as around the horizon." In terms of high-level moral/social progress, this is something we don't fully understand and still trying to have better clarity of.
    • "it comes across as if tech progress is on indefinite hold until we somehow become better people and thus have sufficiently reduced XR" I think this is assigning more impact/agency/credit/blame to either XR or EAs than is reasonable.  The rest of the world is big, and many people want to (often for selfish or narrowly altruistic reasons) indirectly work on economic progress to better their own lives, or that of people close to them. Some people like yourself also want to more broadly increase economic progress for more global reasons.
• What does XR think about the large numbers of people who don't appreciate progress, or actively oppose it?
  • In practice I don't think much about them.
  • I think they're probably wrong about this but they're probably wrong about many other things too (as am I to be clear!), and I personally feel like (possibly overly arrogantly) that I understand their position well enough to reject them, or at least have made enough attempts at communication that further attempts aren't very useful.
  • Note that this is just a personal take, reasonable people can disagree about how valuable this type of engagement is either in general or in this specific case, and I can also imagine (tho I think it's unlikely) that people much more charitable and/or perceptive and/or diplomatic than myself can gain a lot value from those conversations.
• Re the "highway of progress"/road trip metaphor:
  • I like the idea of a highway of progress.
  • A key distinction to me is whether you think of the highway as more of a road trip or more of a move (less metaphorically, whether the purpose is more to enjoy the journey or to go from Point A to Point B).
  • To extend and hopefully not butcher the metaphor too much, my stereotype of progress studies people is that they treat the highway of progress as pretty much like a road trip. That is, you have limited time and it's really important to have a balance of a) an appropriate fast/steady pace along this highway of progress, along with b) enjoying yourself while you're at it. Sure, it's important to avoid unlikely catastrophes like driving off a cliff, but those things are unusual and the faster you go the more sights you see, etc. So the usual balance of considerations is something like consuming now (stopping and smelling the roses, in this metaphor) vs doing more stuff so we and our near descendants can enjoy more stuff later (traveling faster to see more sights, in this metaphor).
  • My own view** of the highway of progress is that we're actually trying to go somewhere, perhaps with the intention of settling there forever. We're leaving our unpleasant homeland to travel, along a dangerous road, to somewhere glorious and wonderful, perhaps a location we previously identified as potentially great (like New York), perhaps a nice suburb somewhere along the road.
    • (There are nicer and less nice versions of this metaphor. Perhaps we're refugees from a wartorn country. Perhaps we're new college graduates convinced that we can find a better life outside of our provincial hometowns).
  • So to many longtermist EAs, the destination matters much more than the journey.
  • In this regard, I think the story/framing where "humanity is trying to take an exciting but dangerous journey to lands known and unknown" (ie, trying to reach utopia) makes  some sense as a story where exquisite care should be made.
    • Ultimately you have a lifetime to go from point A to point B, but you want to be very careful not to make rash irrevocable moves that means your journey ends prematurely.
      • existential risks can come from death (extinction risk) or from other ways that prematurely ends the journey
        • in the travel metaphor, you can be stuck in a suboptimal town and fool yourself into this being a great place to live
        • in our world, this can be dystopias or astronomical suffering, or just trapped in bad values
          • Notable that there were many past (admittedly non-existential) failures from people attempting to reach utopia
  • if you see the highway of progress as a road trip:
    • haste is really important (limited vacation time!)
    • you'd be a dumbass to be so careful  that you barely visit anywhere.
    • journey matters more than destination
      • not reaching final destination totally fine.
    • safety is important but not critical
  • if you see the highway as a mode of one-way transit
    • fine to take your time
    • have a lifetime to get there
    • when you get there much less important than getting there at all.
      • obviously you'd still want to get there eventually, but since the trade is something on the order of 1% of astronomical waste every 10 million years delay, not too important how fast you are.
• I haven't heard of a satisfying explanation from Progress Studies folks about why urgency of economic growth is so important, at least for people who a) think there are percentage point probabilities of existential risk and b) agree with zero intrinsic discount rate.
  • Possible explanations:
    • There aren't percentage points of existential risk,
    • alternatively, the net probability that" dedicated effort from humanity averting existential risks" can actually reduce xrisks is <<1%.
    • As a special case of the above, probability of all of humanity dying in the next ~1000 years approach 1 (cf Tyler Cowen, H/T Applied Divinity Studies)
      • This is an argument for working on medium-term economic progress over working on existential risk not because the risk is too low, but it's too (unfixably) high.
    • the zero intrinsic discount rate is morally or empirically mistaken
    • temporary stagnation either inevitably or with high probability lead to permanent stagnation (so it is itself an existential risk).
    • Some combination of values pluralism + comparative advantage arguments, such that it makes some sense for some individuals to work on progress studies even if it is overall less overwhelmingly important than xrisk.
      • I find this very plausible but my general anecdotal feeling from you and others in this circle is that you're usually making much stronger claims.
    • Something else
  • Clarifying which position individuals believe may be helpful here.

** to be clear, you don't have to have this view to be a longtermist or an EA, but I do think this is much more common among the modal longtermist EA than the modal Progress studies fan. 

I'd guess the story might be a) 'XR primacy' (~~ that x-risk reduction has far bigger bang for one's buck than anything else re. impact) and b) conditional on a), an equivocal view on the value of technological progress: although some elements likely good, others likely bad, so the value of generally 'buying the index' of technological development (as I take Progress Studies to be keen on) to be uncertain.

"XR primacy"

Other comments have already illustrated the main points here, sparing readers from another belaboured rehearsal from me. The rough story, borrowing from the initial car analogy, is you have piles of open road/runway available if you need to use it, so velocity and acceleration are in themselves much less important than direction - you can cover much more ground in expectation if you make sure you're not headed into a crash first. 

This typically (but not necessarily, cf.) implies longtermism. 'Global catastrophic risk', as a longtermist term of art, plausibly excludes the vast majority of things common sense would call 'global catastrophes'. E.g.:

[W]e use the term “global catastrophic risks” to refer to risks that could be globally destabilizing enough to permanently worsen humanity’s future or lead to human extinction. (Open Phil)

My impression is a 'century more poverty' probably isn't a GCR in this sense. As the (pre-industrial) normal, the track record suggests it wasn't globally destabilising to humanity or human civilisation. Even moreso if the matter is of a somewhat-greater versus somewhat-lower rate in its elimination. 

This makes it's continued existence no less an outrage to human condition. But, across the scales from threats to humankind's entire future, it becomes a lower priority. Insofar as these things are traded-off (which seems to be implicit in any prioritisation given both compete for resources, whether or not there's any direct cross-purposes in activity) the currency of XR reduction has much greater value.

Per discussion, there are a variety of ways the story sketched above could be wrong:

• Longtermist consequentialism (the typical, if not uniquely necessary motivation for the above) is false, so we our exchange rate for common sense global catastrophes (inter alia) versus XR should be higher.
• XR is either very low, or intractable, so XR reduction isn't a good buy even on the exchange rate XR views endorse. 
• Perhaps the promise of the future could be lost with less of a bang but a whimper. Perhaps prolonged periods of economic or stagnation should be substantial subjects of XR concern in their own right, so PS-land and XR-land converge on PS-y aspirations.

I don't see Pascalian worries as looming particularly large apart from these. XR-land typically takes the disjunction of risks and envelope of mitigation to be substantial/non-pascalian values. Although costly activity that buys an absolute risk reduction of 1/trillions looks dubious to common sense, 1/thousands + (e.g.) is commonplace (and commonsensical) when stakes are high enough. 

It's not clear how much of a strike that Pascalian counter-examples are constructable from the resources of a given view, and although the view wouldn't endorse them, it doesn't have a crisp story of decision theoretic arcana why not. Facially, PS seems susceptible to the same (e.g. a PS-ers work is worth billions per year, given the yield if you compound an (in expectation) 0.0000001% marginal increase in world GDP growth for centuries).


Buying the technological progress index?

Granting the story sketched above, there's not a straightforward upshot on whether this makes technological progress generally good or bad. The ramifications of any given technological advance for XR are hard to forecast; aggregating over all of them to get a moving average harder still. Yet there seems a lot to temper fairly unalloyed enthusiasm around technological progress I take as the typical attitude in PS-land.

• There's obviously the appeal to the above sense of uncertainty: if at least significant bits of the technological progress portfolio credibly have very bad dividends for XR, you probably hope humanity is pretty selective and cautious in its corporate investments. It'd also generally surprise for what is best for XR to also be best for 'progress' (cf.)
• The recent track record doesn't seem greatly reassuring. The dual-use worries around nuclear technology remain profound 70+ years after their initial development, and 'derisking' these downsides remain remote. It's hard to assess the true ex ante probability of a strategic nuclear exchange during the cold war, nor exactly how disastrous it would have been, but pricing in reasonable estimates of both probably takes a large chunk out of the generally sunny story of progress we observe ex post over the last century.
• Insofar as folks consider disasters arising from emerging technologies (like AI) to represent the bulk of XR, this supplies concern against their rapid development in particular, and against exuberant technological development which may generate further dangers in general.

Some of this may just be a confusion of messaging (e.g. even though PS folks portray themselves as more enthusiastic and XR folks less so, both would actually be similarly un/enthusiastic for each particular case). I'd guess more of it is more substantive around the balance of promise and danger posed by given technologies (and the prospects/best means to mitigate the latter), which then feeds into more or less 'generalized techno-optimism'.

But I'd guess the majority of the action is around the 'modal XR account' of XR being a great moral priority, which can be significantly reduced, and is substantially composed of risks from emerging technology. "Technocircumspection" seems a fairly sound corollary from this set of controversial conjuncts.   

Imagine we can divide up the global economy into natural clusters. We'll refer to each cluster as a "Global Project." Each Global Project consists of people and their ideas, material resources, institutional governance, money, incentive structures, and perhaps other factors.

Some Global Projects seem "bad" on the whole. They might have directly harmful goals, irresponsible risk management, poor governance, or many other failings. Others seem "good" on net. This is not in terms of expected value for the world, but in terms of the intrinsic properties of the GP that will produce that value.

It might be reasonable to assume that Global Project quality is normally distributed. One point of possible difference is the center of that distribution. Are most Global Projects of bad quality, neutral, or good quality?

We might make a further assumption that the expected value of a Global Project follows a power law, such that projects of extremely low or high quality produce exponentially more value (or more harm). Perhaps, if Q is project quality and V is value, $V=Q^N$. But we might disagree on the details of this power law.

One possibility is that in fact, it's easier to destroy the world than to improve the world. We might model this with two power laws, one for Q > 0 and one for Q < 0, like so:

• $V=Q^3$, Q >= 0
• $V=Q^7$, Q < 0

In this case, whether or not progress is good will depend on the details of our assumptions about both the project quality distribution and the power law for expected value:

• The size of N, and whether or not the power law is uniform or differs for projects of various qualities. Intuitively, "is it easier for a powerful project to improve or destroy the world, and how much easier?"
• How many standard deviations away from zero the project quality distribution is centered, and in which direction. Intuitively, "are most projects good or bad, and how much?"

In this case, whether or not average expected value across many simulations of such a model is positive or negative can hinge on small alterations of the variables. For example, if we set N = 7 for bad projects and N = 3 for good projects, but we assume that the average project quality is +0.6 standard deviations from zero, then average expected value is mildly negative. At project quality +0.7 standard deviations from zero, the average expected value is mildly positive.

Here's what an X-risk "we should slow down" perspective might look like. Each plotted point is a simulated "world." In this case, the simulation produces negative average EV across simulated worlds.

And here is a Progress Studies "we should speed up" perspective might look like, with positive average EV.

The joke is that it's really hard to tell these two simulations apart. In fact, I generated the second graph by altering the center point of the project quality distribution 0.01 standard deviations to the right relative to the first graph. In both case, a lot of the expected value is lost to a few worlds in which things go cataclysmically wrong.

One way to approach a double crux would be for adherents of the two sides to specify, in the spirit of "if it's worth doing, it's worth doing with made up statistics," their assumptions about the power law and project quality distribution, then argue about that. Realistically, though, I think both sides understand that we don't have any realistic way of saying what those numbers ought to be. Since the details matter on this question, it seems to me that it would be valuable to find common ground.

For example, I'm sure that PS advocates would agree that there are some targeted risk-reduction efforts that might be good investments, along with a larger class of progress-stimulating interventions. Likewise, I'm sure that XR advocates would agree that there are some targeted tech-stimulus projects that might be X-risk "security factors." Maybe the conversation doesn't need to be about whether "more progress" or "less progress" is desirable, but about the technical details of how we can manage risk while stimulating growth.

How does XR weigh costs and benefits?
Does XR consider tech progress default-good or default-bad?

The core concept here is differential intellectual progress. Tech progress can be bad if it reorders the sequence of technological developments to be worse, by making a hazard precede its mitigation. In practice, that applies mainly to gain-of-function research and to some, but not all, AI/ML research. There are lots of outstanding disagreements between rationalists about which AI/ML research is good vs bad, which, when zoomed in on, reveal disagreements about AI timeline and takeoff forecasts, and about the feasibility of particular AI-safety research directions.

Progress in medicine (especially aging- and cryonics-related medicine) is seen very positively (though there's a deep distrust of the existing institutions in this area, which bottoms out in a lot of rationalists doing their own literature reviews and wishing they could do their own experiments).

On a more gut/emotional level, I would plug my own Petrov Day ritual as attempting to capture the range of it: it's a mixed bag with a lot of positive bits, and some terrifying bits, and the core message is that you're supposed to be thinking about both and not trying to oversimplify things.

What would moral/social progress actually look like?

This seems like a good place to mention Dath Ilan, Eliezer's fictional* universe which is at a much higher level of moral/social progress, and the LessWrong Coordination/Cooperation tag, which has some research pointing in that general direction.

What does XR think about the large numbers of people who don't appreciate progress, or actively oppose it?

I don't think I know enough to speak about the XR community broadly here, but as for me personally: mostly frustrated that their thinking isn't granular enough. There's a huge gulf between saying "social media is toxic" and saying "it is toxic for the closest thing to a downvote button to be reply/share", and I try to tune out/unfollow the people whose writings say things closer to the former.

You mention that some EAs oppose progress / think that it is bad. I might be wrong, but I think these people only "oppose" progress insofar as they think x-risk reduction from safety-based investment is even better value on the margin. So it's not that they think progress is bad in itself, it's just that they think that speeding up progress incurs a very large opportunity  cost. Bostrom's 2003 paper outlines the general reasoning why many EAs think x-risk reduction is more important than quick technological development.

Also, I think most EAs interested in x-risk reduction would say that they're not really in a Pascal's mugging as the reduction in probability of an existential catastrophe occurring that can be achieved isn't astronomically small. This is partly because x-risk reduction is so neglected that there's still a lot of low-hanging fruit.

I'm not super certain on either of the points above but it's the sense I've gotten from the community.

To be honest this parsing of these two communities that have a ton in common reminds me of this great scene from Monty Python's Life of Brian: 

Regarding your question:

What would moral/social progress actually look like?

This is a big and difficult question, but here are some pointers to relevant concepts and resources:

• Moral circle expansion (MCE) - MCE is "the attempt to expand the perceived boundaries of the category of moral patients." For instance, this could involve increasing the moral concern in the wider public (or, more  targeted, among societal decision-makers) for non-human animals or future people. Arguably,  MCE could help reduce the risk of societies committing further atrocities like factory farming and also increase the resources spent on existential risk mitigation (as there is a greater concern for the welfare of future people).
• Improving institutional decision-making - this covers a very broad range of interventions, including, for instance, voting reform. The case for it is that "Improving the quality of decision-making in important institutions could improve our ability to solve almost all other problems. It could also help society’s ability to identify “unknown unknowns” – problems we haven’t even thought of yet – and to mitigate all global catastrophic risks".
• See also this list of resources on differential progress (and in particular, Differential Intellectual Progress as a Positive-Sum Project)
• Global priorities research (GPR) - arguably, a key priority for GPR is to provide answers to the above question.  For instance, this might involve rigorously investigating the plausibility of longtermism in the light of objections (such as the epistemic objection).
• The Possibility of an Ongoing Moral Catastrophe - a very interesting paper arguing "for believing that our society is unknowingly guilty of serious, large-scale wrongdoing.". The paper ends by making two suggestions relevant to the above question: "The article then discusses what our society should do in light of the likelihood that we are doing something seriously wrong: we should regard intellectual progress, of the sort that will allow us to find and correct our moral mistakes as soon as possible, as an urgent moral priority rather than as a mere luxury; and we should also consider it important to save resources and cultivate flexibility, so that when the time comes to change our policies we will be able to do so quickly and smoothly."

[Likely not a crux]

EA often uses an Importance - Neglectedness - Tractability framework for cause prioritization.  I would expect things producing progress to be somewhat less neglected than working on XR; it is still somewhat possible to capture some of the benefits. 

We do indeed see vast amounts of time and money being spent on research and development, in comparison to the amount being spent on XR concerns. Possibly you'd prefer to compare with PS itself, rather than with all R&D? (a) I'm not sure how justified that is; (b) it still feels to me like it ought to be possible to capture some of the benefits from many of PS's proposed changes; (c)  my weak impression is that PS (or things similar to PS- meta-improvements to progress) is still less neglected, and in particular that lots of people who don't explicitly  identify as being part of PS are still working on related concerns.

For those of us who are unfamiliar with Progress Studies, I think it would help if you clarified what exactly that community thinks or advocates.

Is the idea simply to prioritize economic growth? Is it about increasing the welfare of people alive today / in the near future? Would distributing malaria bed nets count as something that a Progress Studies person might advocate qua Progress Studies advocate? Or is it about technological development specifically? (If bed nets don't count as Progress Studies, would development of CRISPR technologies to eradicate malaria count? If yes, why? (Assume the CRISPR technology has the same estimated cost-effectiveness for preventing malaria deaths as bed nets over the next century.))

(Context note: I read this post, all the comments, then Ben Todd's question on your AMA, then your Progress Studies as Moral Imperative post. I don't really know anything about Progress Studies besides this context, but will offer my thoughts now below in the hope it will help with identifying the crux.)

None of the comments so far have engaged with your road trip metaphor, so I'll bite:

In your Progress Studies as Moral Imperative post it sounds like you're concerned that humanity might just slow the car down, stop, and just stay there indefinitely or something due to a lack of appreciation or respect for progress. Is that right?

Personally I think that sounds very unlikely and I don't feel concerned at all about that. I think nearly all other longtermists would probably agree.

This first thing your Moral Imperative post made me think of is Factfulness by Rosling et al. Before reading the book in 2019 I had often heard the idea that roughly "people don't know how much progress we've made lately." I felt like I heard several people say this for a few years without actually encountering the people who were ignorant about the progress.

In the beginning of Factfulness Rosling talks about how a bunch of educated people on a UN council (or something) were ignorant of basic facts of humanity's progress in recent decades. I defer to his claim and yours that these people who are ignorant of the progress we've made exist.

That said, when I took the pre-test quiz at the beginning of the book about the progress we've made I got all of his questions right, and I was quite confident in essentially all of the answers. I recall thinking that other people I know (in the EA community, for example) would probably also get all the questions correct, despite the poor performance on the same quiz by world leaders and other audiences that Rosling spoke to over the years.

I say all this to suggest that maybe Progress Studies people are reactionary to some degree and longtermists (what you're calling "EA/XR" people) aren't? (Maybe PS people are used to seeing a lot of people in society (including some more educated and tech people) being ignorant of progress and opposed to it, while maybe EA people have experienced less of this or just don't care to be as reactionary to such people?) Could this be a crux? Longtermists just aren't very concerned that we're going to stop progressing (besides potentially crashing the car--existential risk or global catastrophic risk), whereas Progress Studies people are more likely to think that progress is slowing and coming to a stop?