Cost-effectiveness estimates generally suggest that, for most reasonable assumptions about the moral weight and degree of suffering of animals, animal welfare interventions are most cost-effective
Animal welfare is more neglected than global health, but not (again for reasonable assumptions about how much animal wellbeing matters) proportionally less important
I think focusing on AI explosive growth has grown in status over the last two years. I don't think many people were focusing on it two years ago except Tom Davidson. Since then, Utility Bill has decided to focus on it full-time, Vox has written about it, it's a core part of the Situational Awareness model, and Carl Shulman talked about it for hours in influential episodes on the 80K and Dwarkesh podcasts.
One of the weaker parts of the Situational Awareness essay is Leopold's discussion of international AI governance.
He argues the notion of an international treaty on AI "fanciful", claiming that:
It would be easy to "break out" of treaty restrictions
There would be strong incentives to do so
So the equilibrium is unstable
That's basically it - international cooperation gets about 140 words of analysis in the 160 page document.
I think this is seriously underargued. Right now it seems harmful to propagate a meme like "International AI cooperation is fanciful".
This is just a quick take, but I think it's the case that:
It might not be easy to break out of treaty restrictions. Of course it will be hard to monitor and enforce a treaty. But there's potential to make it possible through hardware mechanisms, cloud governance, inspections, and other mechanisms that we haven't even thought of yet. Lots of people are paying attention to this challenge and working on it.
There might not be strong incentives to do so. Decisionmakers may take the risks seriously and calculate the downsides of an all-out race exceed the potential benefits of winning. Credible benefit-sharing and shared decision-making institutions may convince states they're better off cooperating than trying to win a race.
International cooperation might not be all-or-nothing. Even if we can't (or shouldn't!) institute something like a global pause, cooperation on more narrow issues to mitigate threats from AI misuse and loss of control could be possible. Even in the midst of the Cold War, the US and USSR managed to agree on issues like arms control, non-proliferation, and technologies like anti-ballistic missile tech.
(I critiqued a critique of Aschenbrenner's take on international AI governance here, so I wanted to clarify that I actually do think his model is probably wrong here.)
Vasco, how do your estimates account for model uncertainty? I don't understand how you can put some probability on something being possible (i.e. p(extinction|nuclear war) > 0), but end up with a number like 5.93e-12 (i.e. 1 in ~160 billion). That implies an extremely, extremely high level of confidence. Putting ~any weight on models that give higher probabilities would lead to much higher estimates.
One of the most common lessons people said they learned from the FTX collapse is to pay more attention to the character of people with whom they're working or associating (e.g. Spencer Greenberg, Ben Todd, Leopold Aschenbrenner, etc.). I agree that some update in this direction makes sense. But it's easier to do this retrospectively than it is to think about how specifically it should affect your decisions going forward.
If you think this is an important update, too, then you might want to think more about how you're going to change your future behaviour (rather than how you would have changed your past behaviour). Who, exactly, are you now distancing yourself from going forward?
Remember that the challenge is knowing when to stay away basically because they seem suss, not because you have strong evidence of wrongdoing.
I'm usually very against criticizing other people's charitable or philanthropic efforts. The first people to be criticized should be those who don't do anything, not those who try to do good.
But switching from beef to other meats (at least chicken, fish, or eggs, I'm less sure for other meats) is so common among socially- and environmentally-conscious people, and such a clear disaster on animal welfare grounds, that it's worth discussing.
Even if we assume the the reducitarian diet emits the same GHGs as a plant-based diet, you'll save about 0.4 tonnes of CO2e per year, the equivalent of a $4 donation (in expectation) to Founders Pledge's climate fund. Meanwhile, for every beef meal you replace with chicken, 200x more animals have to be slaughtered.
I'd bet that for ~any reasonable estimate of the damages of climate change and the moral value of farmed animal lives, this math does not work out favourably.
Your first job out of college is the hardest to get. Later on you'll be able to apply for jobs while working, which is less stressful, and you'll have a portfolio of successful projects you can point to. So hopefully it's some small comfort that applying for jobs will probably never suck as much as it does for you right now. I know how hard it can be though, and I'm sorry. A few years ago after graduating from my Master's, I submitted almost 30 applications before getting an offer and accepting one.
I do notice that the things you're applying to all seem very competitive. Since they're attractive positions at prestigious orgs, the applicant pool is probably unbelievably strong. When there are hundreds of very strong applicants applying for a handful of places, many good candidates simply have to get rejected. Hopefully that's some more small comfort.
It may also be worth suggesting, though, for anyone in a similar position who may be reading this, that it's also fine to look for less competitive opportunities (particularly early on in your career). Our lives will be very long and adventurous (hopefully), and you may find it easier to get jobs at the MITs and Horizons and GovAIs of the world after getting some experience at organisations which may seem somewhat less prestigious.
To speak on my own experience, among those ~30 places that rejected me were some of the same orgs you mention (e.g. GovAI, OpenPhil, etc.). The offer I ended up accepting was from Founders Pledge. I was proud to get that offer and the FP research team there was and is very strong, but I do think it's probably the case that it was a somewhat less competitive application process. But ultimately I loved working at FP. I got to do some cool and rigorous research, and I've had very interesting work opportunities since. It's probably even the case that, at that point in my career, FP was a better place for me to end up than some of the other places I applied.
I think focusing on AI explosive growth has grown in status over the last two years. I don't think many people were focusing on it two years ago except Tom Davidson. Since then, Utility Bill has decided to focus on it full-time, Vox has written about it, it's a core part of the Situational Awareness model, and Carl Shulman talked about it for hours in influential episodes on the 80K and Dwarkesh podcasts.
Tweet away! 🫡
One of the weaker parts of the Situational Awareness essay is Leopold's discussion of international AI governance.
He argues the notion of an international treaty on AI "fanciful", claiming that:
That's basically it - international cooperation gets about 140 words of analysis in the 160 page document.
I think this is seriously underargued. Right now it seems harmful to propagate a meme like "International AI cooperation is fanciful".
This is just a quick take, but I think it's the case that:
(I critiqued a critique of Aschenbrenner's take on international AI governance here, so I wanted to clarify that I actually do think his model is probably wrong here.)
Vasco, how do your estimates account for model uncertainty? I don't understand how you can put some probability on something being possible (i.e. p(extinction|nuclear war) > 0), but end up with a number like 5.93e-12 (i.e. 1 in ~160 billion). That implies an extremely, extremely high level of confidence. Putting ~any weight on models that give higher probabilities would lead to much higher estimates.
This is beautiful, Teps. Thanks for sharing.
One of the most common lessons people said they learned from the FTX collapse is to pay more attention to the character of people with whom they're working or associating (e.g. Spencer Greenberg, Ben Todd, Leopold Aschenbrenner, etc.). I agree that some update in this direction makes sense. But it's easier to do this retrospectively than it is to think about how specifically it should affect your decisions going forward.
If you think this is an important update, too, then you might want to think more about how you're going to change your future behaviour (rather than how you would have changed your past behaviour). Who, exactly, are you now distancing yourself from going forward?
Remember that the challenge is knowing when to stay away basically because they seem suss, not because you have strong evidence of wrongdoing.
I'm usually very against criticizing other people's charitable or philanthropic efforts. The first people to be criticized should be those who don't do anything, not those who try to do good.
But switching from beef to other meats (at least chicken, fish, or eggs, I'm less sure for other meats) is so common among socially- and environmentally-conscious people, and such a clear disaster on animal welfare grounds, that it's worth discussing.
Even if we assume the the reducitarian diet emits the same GHGs as a plant-based diet, you'll save about 0.4 tonnes of CO2e per year, the equivalent of a $4 donation (in expectation) to Founders Pledge's climate fund. Meanwhile, for every beef meal you replace with chicken, 200x more animals have to be slaughtered.
I'd bet that for ~any reasonable estimate of the damages of climate change and the moral value of farmed animal lives, this math does not work out favourably.
You should probably also blank their job title (which would make it easy to work out who they are) and their phone number (!)
Your first job out of college is the hardest to get. Later on you'll be able to apply for jobs while working, which is less stressful, and you'll have a portfolio of successful projects you can point to. So hopefully it's some small comfort that applying for jobs will probably never suck as much as it does for you right now. I know how hard it can be though, and I'm sorry. A few years ago after graduating from my Master's, I submitted almost 30 applications before getting an offer and accepting one.
I do notice that the things you're applying to all seem very competitive. Since they're attractive positions at prestigious orgs, the applicant pool is probably unbelievably strong. When there are hundreds of very strong applicants applying for a handful of places, many good candidates simply have to get rejected. Hopefully that's some more small comfort.
It may also be worth suggesting, though, for anyone in a similar position who may be reading this, that it's also fine to look for less competitive opportunities (particularly early on in your career). Our lives will be very long and adventurous (hopefully), and you may find it easier to get jobs at the MITs and Horizons and GovAIs of the world after getting some experience at organisations which may seem somewhat less prestigious.
To speak on my own experience, among those ~30 places that rejected me were some of the same orgs you mention (e.g. GovAI, OpenPhil, etc.). The offer I ended up accepting was from Founders Pledge. I was proud to get that offer and the FP research team there was and is very strong, but I do think it's probably the case that it was a somewhat less competitive application process. But ultimately I loved working at FP. I got to do some cool and rigorous research, and I've had very interesting work opportunities since. It's probably even the case that, at that point in my career, FP was a better place for me to end up than some of the other places I applied.