I wrote this as an answer to a question which I think has now been deleted, so I copied it to my shortform in order to be able to link it in future, and found myself linking to it often enough that it seemed worth making a top-level post, in particular because if there are important counterarguments I haven't considered I'd like to come across them sooner rather than later! I'd usually put more thought into editing a top-level post, but the realistic options here were not post it at all, or post it without editing.
Epistemic status: I've thought about both how people should thinking about PhDs and how people should think about timelines a fair bit, both in my own time and in my role as an advisor at 80k, but I wrote this fairly quickly. I'm sharing my take on this rather than intending to speak on behalf of the whole organisation, though my guess is that the typical view is pretty similar.
BLUF:
- Whether to do a PhD is a decision which depends heavily enough on personal fit that I expect thinking about how well you in particular are suited to a particular PhD to be much more useful than thinking about the effects of timelines estimates on that decision.
- Don’t pay too much attention to median timelines estimates. There’s a lot of uncertainty, and finding the right path for you can easily make a bigger difference than matching the path to the median timeline.
Going into a bit more detail - I think there are a couple of aspects to this question, which I’m going to try to (imperfectly) split up:
- How should you respond to timelines estimates when planning your career?
- How should you think about PhDs if you are confident timelines are very short?
In terms of how to think about timelines in general, the main advice I’d give is to try to avoid the mistake of interpreting median estimates as single points. Taking this metaculus question as an example, which has a median of July 2027, that doesn’t mean the community predicts that AGI will arrive then! The median just indicates the date by which the community thinks there’s a 50% chance the question will have resolved. To get more precise about this, we can tell from the graph that the community estimates:
- Only a 7% chance that AGI is developed in the year 2027
- A 25% chance that AGI will be developed before August of next year.
- An 11% chance that AGI will not be developed before 2050
- A 9% chance that the question has already resolved.
- A 41% chance that AGI will be developed after January 2029 (6 years from the time of writing).
Taking these estimates literally, and additionally assuming that any work that happens post this question resolving is totally useless (which seems very unlikely), you might then conclude that delaying your career by 6 years would cause it to have 41/91 = 45% of the value. If that’s the case, if the delay increased the impact you could have by a bit more than a factor of 2, the delay would be worth it.
Having done all of that work (and glossed over a bunch of subtlety in the last comment for brevity), I now want to say that you shouldn’t take the metaculus estimates at face value though. The reason is that (as I’m sure you’ve noticed, and as you’ve seen in the comments) they just aren’t going to be that reliable for this kind of question. Nothing is - this kind of prediction is really hard.
The net effect of this increased uncertainty should be (I claim) to flatten the probability distribution you are working with. This basically means it makes even less sense than you’d think from looking at the distribution to plan for AGI as if timelines are point estimates.
Ok, but what does this mean for PhDs?
Before I say anything about how a PhD decision interacts with timelines, it seems worth mentioning that the decision whether to do a PhD is complicated and highly dependent on the individual who’s considering it and the specific situation they are in. Not only that, I think it depends on a lot of specifics about the PhD. A quick babble of things that can vary a lot (and that not everyone will have the same preferences about):
- How much oversight/direction your supervisor provides.
- How supportive the supervisor is.
- How many other people are in the research group and how closely they work together.
- How many classes you’ll have to take and what they will consist of.
- A bunch of other stuff.
- Whether the job you want to be doing afterwards requires/benefits from a PhD.
When you then start thinking about how timelines affect things, it’s worth noting that a model which says ‘PhD students are students, so they are learning and not doing any work, doing a PhD is therefore an n-year delay in your career where n is the number of years it takes’ is badly wrong. I think it usually makes sense to think of a PhD as more like an entry-level graduate researcher job than ‘n more years of school’, though often the first year or two of a US-style PhD will involve taking classes, and look quite a lot like ‘more school’, so “it’s just a job” is also an imperfect model. As a couple of examples of research output during a PhD, Alex Turner’s thesis seems like it should count for more than nothing, as does Collin Burns's recent paper (did you know he was only in the second year of his PhD)!
The second thing to note is that some career paths require a PhD, and other paths come very close to requiring it. For these paths, choosing to go into work sooner isn’t giving you a 6 year speedup on the same career track - you’re just taking a different path. Often, the first couple of years on that path will involve a lot learning the basics and getting up to speed, certainly compared to 6 years in, which again pushes in the direction of the difference that timelines makes being smaller than it first seems. Importantly though, the difference between the paths might be quite big, and point in either direction. Choosing a different path to the PhD will often be the correct decision for reasons that have nothing to do with timelines.
Having said that, there are some things that are worth bearing in mind:
- Flatter distributions might also put more weight on even sooner timelines as well, and to the extent that you are waiting/delaying this clearly does have a downside.
- Shorter timelines are probably correlated with ‘something like current state of the art (SoTA) scales to AGI’, and it’s harder to work on SoTA models in academia compared to in industry.
- Whether you’ll be able to work on something relevant to alignment isn’t guaranteed (I’m not saying here that people should never do PhDs for ‘pure’ skillbuilding purposes, but I do think that option looks worse with very short timelines).
- Many paths don’t require a PhD, so doing a PhD before checking whether you can go straight in does look much more straightforwardly like a mistake.
- Many people shouldn’t do a PhD regardless of timelines. PhDs can be extremely challenging, emotionally as well as intellectually. Personal fit is important for many paths, but I suspect it’s much more important for deciding whether to do a PhD than on average.
I think you raise some good considerations but want to push back a little.
I agree with your arguments that
- we shouldn't use point estimates (of the median AGI date)
- we shouldn't fully defer to (say) Metaculus estimates.
- personal fit is important
But I don't think you've argued that "Whether you should do a PhD doesn't depend much on timelines."
Ideally as a community we can have a guess at the optimal number of people in the community that should do PhDs (factoring in their personal fit etc) vs other paths.
I don't think this has been done, but since most estimates of AGI timelines have decreased in the past few years it seems very plausible to me that the optimal allocation now has fewer people doing PhDs. This could maybe be framed as raising the 'personal fit bar' to doing a PhD.
I think my worry boils down to thinking that "don't factor in timelines too much" could be overly general and not get us closer to the optimal allocation.
I think "different timelines don't change the EV of different options very much" plus "personal fit considerations can change the EV of a PhD by a ton" does end up resulting in an argument for the PhD decision not depending much on timelines. I think that you're mostly disagreeing with the first claim, but I'm not entirely sure.
In terms of your point about optimal allocation, my guess is that we disagree to some extent about how much the optimal allocation has changed, but that the much more important disagreement is about whether some kind of centrally planned 'first decide what fraction of the community should be doing what' approach is a sensible way of allocating talent, where my take is that it usually isn't.
I have a vague sense of this talent allocation question having been discussed a bunch, but don't have write-up that immediately comes to mind that I want to point to. I might write something about this at some point, but I'm afraid it's unlikely to be soon. I realise that I haven't argued for my talent allocation claim at all, which might be frustrating, but it seemed better to highlight the disagreement at all than ignore it, given that I didn't have the time to explain in detail.
Yep, that's right that I'm disagreeing with the first claim. I think one could argue the main claim either by:
I think (1) is false, and think that (2) should be qualified by how one's advice would change depending on timelines. (You do briefly discuss (2), e.g. the SOTA comment).
To put my cards on the table, on the object level, I have relatively short timelines and that fewer people should be doing PhDs on the margin. My highly speculative guess is that this post has the effect of marginally pushing more people towards doing PhDs (given the existing association of shorter timelines => shouldn't do a PhD).
I agree with many of the points, especially that personal fit is a big deal and that doing a PhD is also in part useful research (rather than pure career capital), and what matters is time until the x-risk rather than random definitions of AGI, but I'm worried this bit understates the reasons for urgency quite a bit:
This is on a model in which work becomes moot after a transition point. But it's assuming work before the transition is equally valuable no matter the year.
However, the AI safety community is probably growing at 40%+ per year, and (if timelines are short) it'll probably still be growing at 10-20%+ when the potential existential risk arrives. This roughly means that moving a year of labour invested in AI safety community building one year earlier makes it 10-20% more valuable. This would mean an extra year of labour now is worth 3-10x one in 10 years, all else equal.
Or to turn to direct work, there are serial dependencies i.e. 100 people working for 1 year won't achieve anywhere near as much as 10 people working for 10 years. This again could make extra labour on alignment now many times more valuable work in 10 years.
Another argument is that since the community can have more impact in world with short timelines, people should act as if they're shorter than they are.
This could mean, for instance, if your best guess is 33% timelines under 10yr, 33% medium timelines and 33% longer timelines, it might be optimal for people to allocate something like 70%, 15%, 15%. Yes in this world some people focus on long-term career capital, but it would be less than normal.
Estimating the size of these effects is hard – my main point is that they can be very large, especially as timelines get short. (Many of these effects feel like they go up non-linearly as timelines shorten.)
So, while I agree that if someone's median timeline estimate changes from, say, 25 years to 20 years, that's not going to have much effect on the question; I think how much to focus on career capital could be pretty sensitive to, say, your probability on <10 year timelines.
This is a useful consideration to point out, thanks. I push back a bit below on some specifics, but this effect is definitely one I'd want to include if I do end up carving out time to add a bunch more factors to the model.
I don't think having skipped the neglectedness considerations you mention is enough to call the specific example you quote misleading though, as it's very far from the only thing I skipped, and many of the other things point the other way. Some other things that were skipped:
Work after AGI likely isn't worth 0, especially with e.g. Metaculus definitions.
While in the community building examples you're talking about, shifting work later doesn't change the quality of that work, this is not true wrt PhDs (doing a PhD looks more like truncating the most junior n years of work than shifting all years of work n years later).
Work that happens just before AGI can be done with a much better picture of what AGI will look like, which pushes against the neglectedness effect.
Work from research leads may actually increase in effectiveness as the field grows, if the growth is mostly coming from junior people who need direction and/or mentorship, as has historically been the case.
And then there's something about changing your mind, but it's unclear to me which direction this shifts things:
Good point there are reasons why work could get more valuable the closer you are – I should have mentioned that.
Also interesting points about option value.
The pay difference between working in industry and doing a PhD was a big factor for me to avoid getting a PhD a few years ago.
These days it still plays a role, though as an independent researcher I’d like to connect with more academics so that I can get better at doing research with more rigour and publish more papers. Avoiding the PhD has made this hard and I kind of have to have a lot more initiative to develop these skills that PhD students typically develop. That said, being able to selectively learn the skills that are actually useful for solving alignment is worth the tradeoff.
EDIT: Oh, and the lower level of prestige/credibility I have (from not doing a PhD) may get in the way of some of my plans so I’m trying to be creative about how to gain that prestige without having to do a PhD.