Not OP, but I'm guessing it's at least unclear for the non-safety positions at OpenAI listed but it depends a lot on what a person would do in those positions. (I think they are not necessarily good "by default", so the people working in these positions would have to be more careful/more proactive to make it positive. Still think it could be great.) Same for many similar positions on the sheet but pointing out OpenAI since a lot of roles there are listed. For some of the roles, I don't know enough about the org to judge.

Haha, no, it took me quite a bit longer to phrase what I wrote but I didn't have dedicated non-writing thinking time, e.g. the claim about the expected ratio of future assets seems like something I could sanity check + get a better number for with a pen and pencil and a few minutes but I was too lazy to do that :)

(And I can't let false praise of me stand)

edit to also comment on the substantial part of your comment: Yes, that takeaway seems good to me!

edit edit: Although I'd caveat that s-risk is less mature than general longtermism (more "pre-paradigmatic" for people who like that word), so there might be less (obvious) to do for founders/leaders right now and that can be very frustrating. We still always want to hear about such people.

last edit?: And as in general longtermism, if somebody is interested in s-risk and has really high EtG potential, I might sometimes prefer that. Especially given what I said above about founder/leader type people. Something within an order of magnitude or two of FTX F for s-risk reduction would obviously be a huge win for the space and I don't think it's crazy to think that people could achieve that.

I didn't run this by anyone else in the s-risk funding space, so please don't hold others to these numbers/opinions.
 

Tl;dr: I think this is probably right in direction but with lots of caveats. In particular, it's still the case that s-risk has a lot of money (~low hundreds $m) compared to ideas/opportunities at least right now and at least possibly more so than general longtermism. I think this might change soon since I expect s-risk money to grow less than general longtermist money.

edit: I think s-risk is ideas constrained when it comes to small grants and funding (and ideas) constrained for large grants/investments.

I'd estimate s-risk to have something in the low hundreds $m in expected value (not time-discounted) of current assets specifically dedicated to it. Your question is slightly hard to answer since I'm guessing OpenPhil and FTXF would fund at least some s-risk projects if there were more proposals/more demand for money in s-risk. Also, a lot of funded people and projects who don't work directly on s-risk still care about s-risk. Maybe that should be counted somehow. Naively not counting these people and OpenPhil/FTXF money at all and comparing current total assets in general longtermism vs. s-risk:

In absolute terms: Yup, general longtermism definitely has much more money (~two orders of magnitude.) My guess is that this ratio will grow bigger over time and that it will in expectation grow bigger over time. (~70% credence for each of the claims? Again confused about how to count OpenPhil and FTX F money and how they'll decide to spend money in the future. If I stick to not counting them as s-risk money at all, then >70% credence.)

Per person working on s-risk/general longtermism: Would still say yes although I don't have a good way to count s-risk people and general longtermist people. Could be closer to even and probably not (much) more than an order of magnitude difference. Again, quick and wild guess is that the difference will in expectation grow larger over time, but less confident in this than my guess about how the ratio of absolute money will develop. (55%?)

Per quality-adjusted idea/opportunity to spend money: Unsure. I'd (much) rather have more money-eating ideas/opportunities to reduce s-risk than more money to reduce s-risk but I'm not sure if this is more or less the case compared to general longtermism (s-risk has both fewer ideas/opportunities and less money). Also don't know how this will develop. Arguably, the ratio between money and idea/opportunity also isn't a great metric because you might care more about absolutes here. I think some people might argue that s-risk is less funding constrained compared to ideas-constrained than general longtermism. This isn't exactly what you've asked for but still seems relevant. OTOH, having less absolute money does mean that the s-risk space might struggle to fund even one really expensive project.

edit: I do think if we had significantly more money right now, we would be spending more money now-ish.

Per "how much people in the EA community care about this issue": Who knows :) I'm  obviously both biased and in a position that selects for my opinion.

Funding infrastructure: Funding in s-risk is even more centralized than in general longtermism, so if you think diversification is good, more s-risk funders are good :) There are also fewer structured opportunities for funding in s-risk and I think the s-risk funding sources are generally harder to find. Although again, I assume one could easily apply with an s-risk motivated proposal to general longtermist places, so it's kind of weird to compare the s-risk funding infrastructure to the general longtermist funding infrastructure.

I wrote this off the cuff and in particular, might substantially revise my predictions with 15 minutes of thought.

If you or someone you know are seeking funding to reduce s-risk, please send me a message. If it's for a smaller amount, you can also apply directly to CLR Fund. This is true even if you want funding for a very different type of project than what we've funded in the past.

I work for CLR on s-risk community building and on our CLR Fund, which mostly does small-scale grantmaking, but I might also be able to make large-scale funding for s-risk projects ~in the tens of $ millions (per project) happen. And if you have something more ambitious than that, I'm also always keen to hear it :)

Thanks for asking! We would definitely consider later starts if people aren't available earlier and I would be surprised if we rejected a strong candidate just on the basis that they are only available a month later. There's some chance we would shorten the default fellowship length (not necessarily by the same number of weeks that they would start later) for them, though but we would discuss this with them first. I think if they would only accept the fellowship if it starts later and is the original 9 weeks long, this would increase the threshold for accepting them somewhat, but again, I would be surprised if we rejected a very strong candidate just on the basis. (I think it would only matter for edge cases.) It also depends a bit on what other applications we get: E.g. if we get many strong applications for Germans who can only start later, we would probably be much more happy to accommodate all of them.

Thanks for the question! It's unclear whether we'll run an S-Risk Intro Fellowship in this precise format again. We are fairly likely to run intro events with similar content in the future though. I think this will most likely happen on an annual or semi-annual basis.

Some data that I didn't formally write up and put in the post (mostly for time reasons) on how past fellows evaluated the fellowship:

2021

10 out of 14 fellows filled in the fellowship feedback survey:

• 10 of 10 respondents answered "Are you glad that you participated in the fellowship" with 5/5 ("hell yeah")
• 9 of 10 respondents answered "If the same program happened next year, would you recommend a friend (with similar background to you before the fellowship) to apply?" with 10/10 ("strongly yes).
  • 1 of 10 fellows rated the question with 9/10.
• 4 of 10 respondents answered "To what extent was the fellowship a good use of your time compared to what you would otherwise have been doing" with "notably more valuable (3-10x the counterfactual)"
  • 3 of 10 respondents answered "To what extent was the fellowship a good use of your time compared to what you would otherwise have been doing" with "Much more valuable (10-30x the counterfactual)"
  • 1 of 10 respondents answered "To what extent was the fellowship a good use of your time compared to what you would otherwise have been doing" with "Far more valuable (>30x the counterfactual)"
  • 1 of 10 respondents answered "To what extent was the fellowship a good use of your time compared to what you would otherwise have been doing" with "Somewhat more valuable (1-3x the counterfactual)"
  • 1 of 10 respondents did not answer the question

It's possible that the respondents were anchored by the possible options for the last question: There was one option "about as valuable" and 4 options each in the directions more and less valuable. The lowest respondents could go was "not at all valuable (<10% of counterfactual)"

The survey was not anonymous (although the name field was optional and one respondent chose not to enter their name) and several of the respondents were either in employment, on a grant, or on a trial with us at the time of responding.

2020

7 out of 9 fellows filled in the fellowship feedback survey:

• 6 of 7 respondents answered "Are you glad that you participated in the fellowship" with 5/5 ("hell yeah")
  • 1 of 7 respondents did not answer this question
• 3 of 7 respondents answered "To what extent was the fellowship a good use of your time compared to what you would otherwise have been doing" with "Much more valuable (10-30x the counterfactual)"
  • 3 of 7 respondents answered  "To what extent was the fellowship a good use of your time compared to what you would otherwise have been doing" with "notably more valuable (3-10x the counterfactual)"
  • 1 of 7 respondents answered "To what extent was the fellowship a good use of your time compared to what you would otherwise have been doing" with "much more valuable (>30x the counterfactual)"
• We did not ask the question of whether they would recommend the program to someone similar to them this year.

The survey was anonymous in 2020. Several of the respondents were either in employment, on a grant, or on a trial with us at the time of responding.

Again, I haven't actually read this, but this article discusses intransitivity in asymmetric person-affecting views, i.e. I think in the language you used: The value of pleasure is contingent in the sense that creating new lives with pleasure has no value. But the disvalue of pain is not contingent in this way. I think you should be able to directly apply that to other object-list theories that you discuss instead of just hedonistic (pleasure-pain) ones.

An alternative way to deal with intransitivity is to say that not existing and any life are incomparable. This gives you the unfortunate situation that you can't straightforwardly compare different worlds with different population sizes. I don't know enough about the literature to say how people deal with this. I think there's some long work in the works that's trying to make this version work and that also tries to make "creating new suffering people is bad" work at the same time.

I think some people probably do think that they are comparable but reject that some lives are better than neutral. I expect that that's rarer though?

person-affecting view of ethics, which longtermists reject

I'm a longtermist and I don't reject (asymmetric) person(-moment-)affecting views, at least not those that think necessary ≠ only present people. I would be very hard-pressed to give a clean formalization of necessary people though. I think it's bad if effective altruists think longtermism can only be justified with astronomical waste-style arguments and not at all if someone has person-affecting intuitions. (Staying in a broadly utilitarian framework. There are, of course, also obligation-to-ancestor-type justifications for longtermism or similar.) The person-affecting part of me just pushes me in the direction of caring more about trajectory change than extinction risk.

Since I could only ever give very handwavey defenses of person-affecting views and even handwaveier explanations of my overall moral views: Here's a paper by someone that AFAICT is at least sympathetic to longtermism and discusses asymmetric person-affecting views. (I have to admit I never got around to read the paper.) (Writing a paper on how an asymmetric person-affecting view obviously also doesn't necessarily mean that the author doesn't actually reject person-affecting views)

Big fan of what you describe in the end or something similar.

It's still not great, and it would still be hard to distinguish the people who opted-in and received the codes but decided not to use them from the people who just decided to not receive their codes in the first place

Not sure whether you mean it's hard from the technical side to track who received their code and who didn't (which would be surprising) or whether you mean distinguishing between people who opted out and people who opted in but decided not to see the code. If the latter: Any downside to just making it clear in the email that not receiving your code is treated as opting out? People who don't read the email text should presumably not count anyway.

On the trust-building and adding to the voices in favor of making it opt-in: I like many aspects of this game, including the fact that doing the right thing is at least plausibly "do nothing and don't tell anyone you've done/you're doing the right thing." But currently, the combination of no opt-in/opt-out and that it's not anonymous doesn't really make it feel like a trust-building exercise to me. It feels more like "Don't push the button because people will seriously hate you if you do" and also "people will also get angry if you push the button because of an honest mistake, so it's probably best to just protect yourself from information for a day" (see last year - although maybe people were more upset about the wording in some of the messages the person who pushed sent rather than being tricked into pushing itself?), which isn't great. So, I think the lack of opt-in/out makes lots of people upset + it ruins the original purpose of this event IMO, and everyone is unhappy.