It would be more useful to compare AI safety work vs. other longtermist interventions, since it's unlikely that donations to GiveWell would beat longtermist interventions from a longtermist POV

I agree that would be incredibly useful; maybe I'll do that next (20% chance). The same model can be used for pandemics and nuclear risk -- I'd just need to update (1) P(doom) for each, (2) tractability (for AI, that's the 'AI safety decreases risk by 7% per doubling of staff'), and (3) personal contribution. It could be a quick tool for anyone to realize how impactful longtermist careers are and, based on their beliefs about the world and their own ability, choose the career with the highest EV, though I'd only recommend acting on that comparison if the difference is quite large (my hunch is 5x or higher) given the uncertainty involved.

It would also force people to hold self-consistent beliefs. If there's a separate calculator for AI safety and one for biosecurity, someone could claim that non-[x-risk at hand] is much higher than [x-isk at hand] in each case, but that wouldn't be consistent across the two, cause each [x-risk at hand] would factor into the other's non-[x-risk at hand]. In other words, it can be used as a tool to calibrate beliefs about existential risks (I think it would do that for me, at least).

The biggest thing missing from the model is the possibility that safety research is net harmful.

This is quite interesting; I hadn't thought of this. Do you think it should be approximated as "% chance that AI safety is actually bad" and "increase in AI risk per doubling of staff"? e.g. it would look like this:

1. 90% chance AI safety reduces AI risk, decreasing it by 10% per doubling of staff
2. 10% chance AI safety increases AI risk, increasing it by 10% per doubling of staff

Or is that too rudimentary, you think?

I compared GiveWell to convince people who believe in global dev but are skeptical of AI risk. I could have kept the explanation of why dollars to GiveWell should be discounted and instead said "Donate [smaller yet still big amount] to AI safety / longtermist solutions" which would be equivalent to "Donate $5M to GiveWell" (assuming my discounting is accurate), but I feared it would sound circular. Some people's natural response would be "I already don't believe AI risk is that big of a deal!" even though the two framings are logically equivalent

Oh that was a complete misunderstanding on my part. Thank you

Your post (along with Anthropic's statement, and Sam Altman's tweet) convinced me to switch from ChatGPT Plus to Claude Pro.

I am not particuarly convinced by Bregman's reasoning. I think the frequent prompts (by your post, Anthropic's, and Altman's) to consider boycotting ChatGPT is probably what actually convinced me, which is not very rational to best honest.

My "rational" justification is that I think moving $20/month from OpenAI to Anthropic is most likely non-negative (and probably positive). I'm not sure it why I never thought of this before haha

Thank you!

I might be biased (same school, tried something similar in physics), but great work! I recommend promoting the Olympiads nationally once they're stable, so more students can benefit. I believe (not super confident about this, though) that the private tutoring market in Egypt can absorb a lot of demand from students wanting to participate in Science Olympiads, so you shouldn't (at least in my opinion) try to train students en masse; just create the incentive. No idea how you can do that, though-- I assume social media is the cheapest method?

Also, not trying to bring you down, but Egypt's bureaucracy is hell (you probably know this already). If you manage to launch just one Olympiad and keep it stable for 1-2 years, that's fantastic. If you ever feel like you're hitting too many walls, it's totally fine to rethink the whole project and invest your time in other (more impactful) ones

Good luck!