SC

Stephen Clare

Senior Research Associate @ Center for International Governance Innovation
4353 karmaJoined Working (6-15 years)

Bio

Previously I've been a Research Fellow at the Forethought Foundation, where I worked on What We Owe The Future with Will MacAskill; an Applied Researcher at Founders Pledge; and a Program Analyst for UNDP.

Comments
227

Thanks for writing this, it's clearly valuable to advance a dialogue on these incredibly important issues. 

I feel an important shortcoming of this critique is that it frames the choice between national securitization vs. macrosecuritization in terms of a choice between narratives, without considering incentives. I think Leopold gives more consideration to alternatives than you give him credit for, but argues that macrosecuritization  is too unstable of an equilibrium:

Some hope for some sort of international treaty on safety. This seems fanciful to me. The world where both the CCP and USG are AGI-pilled enough to take safety risk seriously is also the world in which both realize that international economic and military predominance is at stake, that being months behind on AGI could mean being permanently left behind. If the race is tight, any arms control equilibrium, at least in the early phase around superintelligence, seems extremely unstable. In short, ”breakout” is too easy: the incentive (and the fear that others will act on this incentive) to race ahead with an intelligence explosion, to reach superintelligence and the decisive advantage, too great.

I also think you underplay the extent to which Leopold's focus on national security is instrumental to his goal of safeguarding humanity's future. You write: "It is true that Aschenbrenner doesn’t always see himself as purely protecting America, but the free world as a whole, and probably by his own views, this means he is protecting the whole world. He isn’t, seemingly, motivated by pure nationalism, but rather a belief that American values must ‘win’ the future." (emphasis mine.)

First, I think you're too quick to dismiss Leopold's views as you state them. But what's more, Leopold specifically disavows the specific framing you attribute to him:

To be clear, I don’t just worry about dictators getting superintelligence because “our values are better.” I believe in freedom and democracy, strongly, because I don’t know what the right values are [...] I hope, dearly, that we can instead rely on the wisdom of the Framers—letting radically different values flourish, and preserving the raucous plurality that has defined the American experiment.

Both of these claims -- that international cooperation or a pause is an unstable equilibrium, and that the West maintaining an AI lead is more likely to lead to a future with free expression and political experimentation -- are empirical. Maybe you'd disagree with them, but then I think you need to argue that this model is wrong, not that he's just chosen the wrong narrative.

This is beautiful, Teps. Thanks for sharing.

One of the most common lessons people said they learned from the FTX collapse is to pay more attention to the character of people with whom they're working or associating (e.g. Spencer Greenberg, Ben Todd, Leopold Aschenbrenner, etc.). I agree that some update in this direction makes sense. But it's easier to do this retrospectively than it is to think about how specifically it should affect your decisions going forward.

If you think this is an important update, too, then you might want to think more about how you're going to change your future behaviour (rather than how you would have changed your past behaviour). Who, exactly, are you now distancing yourself from going forward?

Remember that the challenge is knowing when to stay away basically because they seem suss, not because you have strong evidence of wrongdoing.

I'm usually very against criticizing other people's charitable or philanthropic efforts. The first people to be criticized should be those who don't do anything, not those who try to do good.

But switching from beef to other meats (at least chicken, fish, or eggs, I'm less sure for other meats) is so common among socially- and environmentally-conscious people, and such a clear disaster on animal welfare grounds, that it's worth discussing.

Even if we assume the the reducitarian diet emits the same GHGs as a plant-based diet, you'll save about 0.4 tonnes of CO2e per year, the equivalent of a $4 donation (in expectation) to Founders Pledge's climate fund. Meanwhile, for every beef meal you replace with chicken, 200x more animals have to be slaughtered. 

I'd bet that for ~any reasonable estimate of the damages of climate change and the moral value of farmed animal lives, this math does not work out favourably.

You should probably also blank their job title (which would make it easy to work out who they are) and their phone number (!)

Your first job out of college is the hardest to get. Later on you'll be able to apply for jobs while working, which is less stressful, and you'll have a portfolio of successful projects you can point to. So hopefully it's some small comfort that applying for jobs will probably never suck as much as it does for you right now. I know how hard it can be though, and I'm sorry. A few years ago after graduating from my Master's, I submitted almost 30 applications before getting an offer and accepting one.

I do notice that the things you're applying to all seem very competitive. Since they're attractive positions at prestigious orgs, the applicant pool is probably unbelievably strong. When there are hundreds of very strong applicants applying for a handful of places, many good candidates simply have to get rejected. Hopefully that's some more small comfort. 

It may also be worth suggesting, though, for anyone in a similar position who may be reading this, that it's also fine to look for less competitive opportunities (particularly early on in your career). Our lives will be very long and adventurous (hopefully), and you may find it easier to get jobs at the MITs and Horizons and GovAIs of the world after getting some experience at organisations which may seem somewhat less prestigious.

To speak on my own experience, among those ~30 places that rejected me were some of the same orgs you mention (e.g. GovAI, OpenPhil, etc.). The offer I ended up accepting was from Founders Pledge. I was proud to get that offer and the FP research team there was and is very strong, but I do think it's probably the case that it was a somewhat less competitive application process. But ultimately I loved working at FP. I got to do some cool and rigorous research, and I've had very interesting work opportunities since. It's probably even the case that, at that point in my career, FP was a better place for me to end up than some of the other places I applied.

Your steelman doesn't seem very different from "I didn’t have strong views on whether either of these opinions were true. My aim was just to introduce the two of them, and let them have a conversation and take it from there."

Many organizations I respect are very risk-averse when hiring, and for good reasons. Making a bad hiring decision is extremely costly, as it means running another hiring round, paying for work that isn't useful, and diverting organisational time and resources towards trouble-shooting and away from other projects. This leads many organisations to scale very slowly.

However, there may be an imbalance between false positives (bad hires) and false negatives (passing over great candidates). In hiring as in many other fields, reducing false positives often means raising false negatives. Many successful people have stories of being passed over early in their careers. The costs of a bad hire are obvious, while the costs of passing over a great hire are counterfactual and never observed.

I wonder  whether, in my past hiring decisions, I've properly balanced the risk of rejecting a potentially great hire against the risk of making a bad hire. One reason to think we may be too risk-averse, in addition to the salience of the costs, is that the benefits of a great hire could grow to be very large, while the costs of a bad hire are somewhat bounded, as they can eventually be let go.

Sam said he would un-paywall this episode, but it still seems paywalled for me here and on Spotify. Am I missing something? (The full thing is available on youtube)

Load more