There are at least four ways of thinking about replaceability:
Key takeaways:
You want to do some good in the world. But to do good you need resources, and some of those will be in limited supply. If you use scarce resources, that means someone else won't. Now they can't do as much good as they could've. So how should you decide which scarce resources to use, or whether to use them at all?
Replaceability is a partial answer to this. Replaceability is usually taken to mean something like "the extent to which someone else would do what you'd do in a job if you don't take the job". I interpret it more generally, as "the extent to which other people would've produced similar value with a scarce resource had you not used it".[1]
The paradigmatic example is choosing a career: if you're thinking of becoming a doctor to save hundreds of lives, you should perhaps keep in mind that someone else might've saved those lives had you chosen a different career. If so, the doctor profession is highly replaceable. But this is just one of many situations where replaceability may be a factor:
| Decision | Scarce resource |
|---|---|
| Choosing a job | Salary, opportunities for direct impact and support from employer and colleagues |
| Asking someone to be a mentor | Mentor's time and energy |
| Applying for a grant | Grant, grantmaker's time and grantmaker's connections |
| Marketing a resource to the community | Prestige and community attention |
We can look at impact in at least four different ways:
Benjamin Todd, Paul Christiano and others have thought and written about replaceability, but I think it's fair to say no one has reached any definitive conclusion: it's a hard problem. Todd recommends focusing on other, more robustly predictable factors, like personal fit and scale and solvability, when choosing a career. In fact, there's been remarkably little discussion about replaceability in the past few years, I think partly because people have realised that replaceability differs less across career options than those other things (personal fit, etc.), and partly because people have tactically retreated from a difficult problem.
I think this is somewhat unfortunate, as replaceability seems to me to be decision-relevant and somewhat tractable, even if not as important as the problems Todd and Christiano have moved on to.
There are ~1M licensed physicians in the US alone, which is ~2 orders of magnitude more than the number of effective altruists globally. I'd expect there to be a difference of ≥1 order of magnitude in the number of people working on different problems within effective altruism too. If the number of people working in a field affects replaceability in a way that matters when estimating counterfactual impact, it seems important to know how it matters.
To imagine a rather extreme scenario, I think it's easy for someone who's choosing between two jobs, one in a field with lots of people and one that basically only they can do, to have heard of replaceability and think, "Though this thing that only I can do seems less important than this thing that many other people are doing, if I don't work on the thing that many people are doing, someone else nearly as good as me would take my place, so instead I should work on this thing that only I can do." But if something like the replacement view is a better representation of the truth than the single comparison view, this person may be making a grievous error!
Sports seems somewhat analogous here. Sports teams are also making decisions about how to allocate scarce resources (e.g. playing time), and they do so by explicitly considering replaceability. (It's easier for them; they have good measures of how teams and players perform.)
For example, suppose a hockey team is deciding whether to sign a star left winger or equally talented star right winger. Suppose currently its top left winger is near star calibre, whereas its top right winger is middling. Suppose its worst left winger is so bad as to be a liability, whereas its worst right winger is pretty good. If the team only compared each potential addition to the player whose spot they'd take, they'd go with the star right winger (who's much better than the currently best right winger on the team, who's middling). But it may be better for the team to sign the star left winger (to get rid of the marginal left winger, who's a liability).[2]
I think it's currently unclear how we are looking or should look at replaceability. My impression is that the single comparison view was circulated once long ago and that, ever since its flaws became apparent, there's been something of a vacuum. With what do people fill this vacuum? I have no idea. But we make decisions, so it must be something.
I've run Monte Carlo simulations with the aim of seeing how these views perform in four simple, idealised scenarios. The scenarios pit two fields (as in, spheres of activity) against each other. (For more on how the simulations were run, see the Appendix.) The percentages signify how often you'd choose Field A if you took a given view:
| # | Field A | Field B | Naive | Single Comparison | Replacement | God |
|---|---|---|---|---|---|---|
| 1 | 1K ppl (medium talent), 100 jobs | 1K ppl (medium talent), 100 jobs | 49.9% | 50.7% | 50.0% | 50.1% |
| 2 | 1K ppl (high talent), 100 jobs | 1K ppl (medium talent), 100 jobs | 69.6% | 62.3% | 61.8% | 62.4% |
| 3 | 1K ppl (medium talent), 100 jobs | 100 ppl (medium talent), 10 jobs | 50.7% | 26.0% | 53.4% | 49.4% |
| 4 | 1K ppl (medium talent), 100 jobs | 200 ppl (medium talent), 100 jobs | 67.9% | 55.8% | 56.2% | 56.5% |
I interpret the results thusly:
Like any model, this one rests on a number of simplifying assumptions. None of these results are guaranteed to hold outside the model's world. Still ...
Though I don't feel sure enough to actually make recommendations to people faced with career decisions, I'm tentatively bullish on the replacement view. Arguments for:
Arguments against:
Replaceability is also more complicated than this post makes it out to be. For example, in the real world:
The problem that replaceability is meant to address is a pretty rare one. There doesn't seem to be much research on it. There aren't that many situations where people (1) pursue the same goal[5], (2) don't usually coordinate, (3) use shared scarce resources with substantial supply and demand and (4) are able to use those resources to varying degrees of efficiency. But effective altruists are in this rare situation.
Here is the procedure I used to simulate career choices based on the four replaceability views:
Page, Scott E. 2018. The Model Thinker: What You Need to Know to Make Data Work for You. Basic Books.
Pearl, Judea, Madelyn Glymour, and Nicholas P Jewell. 2016. Causal Inference in Statistics: A Primer. John Wiley & Sons.
Replaceability is different from counterfactuals. Pearl, Glymour, and Jewell (2016) describes a counterfactual as "an 'if' statement in which the 'if' portion is untrue or unrealized". This involves tallying up all the ways a thing would've gone differently. Replaceability is a special kind of counterfactual reasoning, dealing only with the use (or non-use) of a scarce resource. ↩︎
True, ice time makes this a more subtle calculation. Signing the star left winger means the near-star-calibre left winger gets pushed down to the second line, meaning their (considerable) impact is reduced. But I think it serves as an example of these kinds of considerations mattering in practice. ↩︎
I frame it as "how much effort you add", not "how much impact you have", because impact also depends on other things, in particular the problem areas' relative scale (defined, after 80,000 Hours, as Good Done ÷ % of Problem Solved) and solvability (% of Problem Solved ÷ % Increase in Effort). Focusing on effort alone is cleaner as we can bracket those other concepts. As far as this post is concerned, all problems have the same scale and solvability.
NB. "Increase in Effort" is called by 80,000 Hours "Increase in Resources", but since I'm already using the word "resource" to refer to labour, time and money, I'm calling it "Increase in Effort" instead. ↩︎
Some posts, like this one, point to income and researcher citation count as evidence of this (emphasis mine): "If job performance is like income, or the number of citations people have on academic papers, it is more like a log normal distribution[.] That is, most aspiring academics have few citations, while some have thousands, tens of thousands, or even hundreds of thousands. [...] We're very unsure about this question, and would like to see more research into it. Some evidence we've seen suggests that output is normally distributed even in 'complex' jobs, like being a doctor. However, for the most difficult and creative work, like academic research, we suspect that the variance is high in the tails. Even there, it's hard to be confident since many measures of output (such as citation count) are likely to overstate differences in productivity."
As alluded to in the quoted passage, I think income and citation count aren't good evidence. Even if talent is normally distributed (i.e. follows a bell curve), salary and citations could well have heavy tails due to nonlinear effects later in the causal chain. The Matthew effect – where having an advantage gets you further advantages – applies here too, as well-cited papers are more likely to get further citations independently of quality, and richer people are more likely to get more money regardless of talent. ↩︎
Benjamin Todd calls this a "shared aim community". ↩︎
This model implicitly takes neglectedness and personal fit into account – neglectedness (and importance) is captured by a job's impact level, and personal fit is captured by a person's talent level. ↩︎
Page (2018) writes: "In some cases, we may know the mean of the distribution and also know that all values must be positive. Given those constraints, the maximal entropy distribution must have a long tail, and as we spread the distribution across more values, we must balance high values with many low-value outcomes."
I don't know the means of these talent distributions, but it does seem likely to me that (a) that talent can't be negative and (b) the distance between the average talent and zero talent is smaller than the distance between the average talent and the greatest talent. That seems like a pretty good justification for a heavy-tailed distribution. ↩︎
Note that this means that, if the number of people exceeds the number of jobs, you'll tend to have an above average talent level. If there are 10x as many people as jobs, for example, you're randomly selected from the 90th percentile. ↩︎