I just want to say this

I do my best to reply to everyone who reaches out about careers at CG, even if sometimes this is just a quick note celebrating their pursuit of impactful work and sharing career resources with them. At the time of writing, this has worked out to roughly 1,200 messages and 600 calls/in-person meetings.

is really inspiring. It sounds exhausting, but as someone who's often struggled to get feedback from decisionmakers in the EA space, I'm sure those 1800 people very much appreciated it - and I'd be amazed if it didn't lead to some substantially positive counterfactual changes in their plans :) 

(and maybe I'll reach out to you for some feedback soon!)

A relatively simple way of making the repugnant conclusion more intuitive to me is to recognise that individual selves are largely an illusion, i.e. that empty individualism (or, its better marketed/more spiritual sounding but functional equivalent open individualism) is correct.

Suppose you've set up your parameters such that pinpricks actually involve negative utility - because (trigger warning of slightly graphical image for the second of these links) in many cases it obviously isn't actually negative, which muddies our intuitions. Then for empty individualism a tiny amount of torture either is actually closely analogous to a large number of pinpricks (that just happen to be locally clustered). The OI equivalent is that a small amount of torture is analogous to a pinprick on the cosmic entity

Even in the case of humans, we can imagine how pinpricks could add up. A single superficial pinprick isn't that bad - and the difference between a 1mm and 2mm insertion would be very slight. But if you insert hundreds or thousands of pins 10+mm deep, gradually you move towards an experience that seems as bad as any other torture. 

To put it simply, sufficiently many pinpricks, of sufficient depth to be unpleasant, are torture. And they can be torture at a degree of virtually any level of pain a human is capable of experiencing - so I don't see a need to introduce dramatic discontinuities in our moral evaluation to explain why other tortures are somehow still morally worse.

That was fascinating - I really like the idea of reframing EA ideas as a way of saving on future research costs.

I'm a bit unclear what we're currently to make of the the '3–5x' estimate - you say it's illustrative, but also 'plausible'. Assuming that is your current best guess, could you say how you reached it?

Have you had any takers (who you can publicly name) other than Greg?

I don't encounter many people who still identify as longtermist, but as someone who does, I recently wrote these arguments for why longtermists should be less extinction-focused.

The tl;dr is that I think that other than extinction there are predictable patterns, with perhaps the most prominent related to entropy, and that those patterns provide more nuanced ways to estimate the cost of lesser catastrophes - and that while assessing the costs of lesser catastrophes precisely is infeasible, that's not a basis for thinking they would be negligible compared to extinction.

Nice post :) This helps clarify some of my own continuing scepticism about likely speed and degree of change in the near term. 

I've had a longer-term analogous view that AI doom futures involving interstellar expansion seem unlikely on the grounds that, after wiping out all humans such an AI would find itself in an extremely novel scenario, in which anything resembling deep learned training on the world to that point would be useless.

>I think it's historically pretty incorrect that the grounding in cost-effectiveness is what made EA good

FWIW the reasons you're giving here are closely related to the reasons why I'm sceptical that modern AI-focused EA is in fact as good. I don't think it's unreasonable to support AI safety work, but I think doing so to the extent the movement is being guided to is throwing away most of the epistemics that could make EA a long-term robustly positive influence. EA's original tagline used to be 'using evidence and reason', but the extreme AI safety focus seems to drop the 'evidence' part.

To believe you should focus on AI safety, you approximately need to believe all of

• short timelines; and
• either
  • no trend of convergence between intelligence and morality or
  • the view that convergence wouldn't matter or wouldn't be enough to avoid moral disaster; and
• either
  • long timelines on other GCRs or
  • other GCRs not really mattering to humanity's long term prospects and
    • zero discounting on future people; and
• that a flourishing human future is +EV; and
• that trying to improve average welfare in a flourishing future is less good than trying to increase the probability of a flourishing future; and
• reasonable confidence that AI safety work has learned from its past mistakes and will be reliably +EV; and
• there won't be a sufficient public shift towards AI safety to make it low enough leverage that less; and
• you personally have more comparative advantage working on AI safety than any other cause; and

and surely some further assumptions I've missed, and many ways to further unpack these premises. To advocate work on AI safety as the primary EA cause you need to believe that the final bullet applies to the majority of your audience.

But I think there's plenty in that list of assumptions that's easy to disagree with, and a lot of entangled assumptions whose entanglement to my knowledge hasn't really been explored (e.g. I find it hard to credit both that there's no convergence between intelligence and morality and that there's a long term equilibrium which is both stable and in some nontrivial sense positive or desirable).

So I semiagree with @MichaelDickens's original comment's in-principle scepticism while wondering whether in practice int/a might end up promoting causes that feel closer to the what I view as the original spirit of the movement.

There's also some practical concerns in the OP that I think EA has dropped the ball on, such as building the sort of real community that would have retained greater support/membership over the years (my impression is the substantial majority of EAs who joined the movement more than 6 or 7 years ago have largely disengaged with it). 

So I guess I'm noncommittally hopeful that this becomes something valuable - and remains, and Euan said, symbiotic with EA. If it just gives people who would have been somewhat supportive but felt too constrained a way to stay engaged with an encouraging community, that seems like it could be high value.

I'm confused by the strong negative reaction to this comment. I guess it's about the CoGi funding, which does sound like I was wrong. But it seems to be true that there's no option to directly apply for funding for a new project (NickLaing mentions the GH funding circle, but they completed one round last year and their website doesn't currently imply there would be any more). 

I think this helps explain the decline of GHD in the OP - AIM's charity list notwithstanding, no-one in the movement is incentivised to come up with practical ideas in the field.

Last I heard it was something like 10% of their GCR budget.

It's also basically impossible to apply for GHD funding. I recently decided to put my money where my mouth is and get involved in an early stage GHD project, but there's basically no EA-aligned funder who's willing to let you approach them. 

SFF are exclusively longtermist, EA GHD as mentioned basically shut down, and Givewell and CoGi don't accept unsolicited applications. So as far as I can see if you think you have an idea in the GHD space and need funding for it you basically have to look outside the EA world (someone tell me if I missed something!)

It seems like, considering how intelligent and creative our species is, we should expect that, even in very dire conditions, we would be able to re-build civilization.

That shouldn't necessarily be the primary concern. Though it also seems that people who've studied our ability to rebuild civilisation are substantially more pessimistic.