This is a special post for quick takes by Jamie_Harris. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Sorted by Click to highlight new quick takes since: Today at 3:52 AM

The value of graduate training for EA researchers: researchers seem to think it is worthwhile

Imagine the average "generalist" researcher employed by an effective altruist / longtermist nonprofit with a substantial research component (e.g. Open Philanthropy, Founders' Pledge, Rethink Priorities, Center on Long-Term Risk). Let's say that, if they start their research career with an undergraduate/bachelor's degree in a relevant field but no graduate training, each year of full-time work, they produce one "unit" of impact.

In a short Google Form, posted on the Effective Altruism Researchers and EA Academia Facebook groups,  I provided the above paragraph and then asked: "If, as well as an undergraduate/bachelor's degree, they start their research career at EA nonprofits with a master's degree in a relevant field, how many "units" of impact do you expect that they would produce each year for the first ~10 years of work?"* The average response, from the 8 respondents, was 1.7. 

I also asked: "If, as well as an undergraduate/bachelor's degree, they start their research career at EA nonprofits with a PhD in a relevant field, how many "units" of impact do you expect that they would produce each year for the first ~10 years of work?"* The average response was 3.9. 

I also asked people whether they were a researcher at a nonprofit, in academia, or neither, and whether they had graduate training themselves or not.** Unsurprisingly, researchers in academia rated the value of graduate training more highly than researchers in nonprofits (2.0 and 4.3 for each year with a master's and a PhD, respectively, compared to 1.2 and 1.7), as did respondents with graduate training themselves, relative to respondents without graduate training (2.0 and 5.2 compared to 1.2 and 1.7).

I asked a free-text response question: "Do you think that the value of graduate training would increase/compound, or decrease/discount, the got further into their career?" 4 respondents wrote that the value of graduate training would decrease/discount the got further into their career, but didn't provide any explanations for this reasoning. This was also my expectation; my reasoning was that one or more years' of graduate training, which would likely only be partly relevant to the nonprofit work that you would be doing, would become relatively less important later on, since your knowledge, skills, and connections would have increased through your work in nonprofits. 

However, two respondents argued that the value of graduate training would increase/compound. One added: "People without PhDs are sadly often overlooked for good research positions and also under-respected relative to their skill. If they don't have a PhD they will almost never end up in a senior research position." The other noted that it would "increase/compound, particularly if they do things other than anonymous research, e.g. they build an impressive CV, get invited to conferences because of their track record. If one doesn't have a PhD, the extent of this is limited, mostly unless one fits a high-credibility non-academic profile, e.g. founded an organization."

I did some simple modelling / back of the envelope calculations to estimate the value of different pathways, accounting for 1) the multipliers on the value of your output as discussed in the questions on the form and 2) the time lost on graduate education.*** Tldr; with the multiplier values suggested by the form respondents, graduate education clearly looks worthwhile for early career researchers working in EA nonprofits, assuming they will work in an EA research nonprofit for the rest of their career. It gets a little more complex if you try to work it out in financial terms, e.g. accounting for tuition fees.

For my own situation (with a couple of years of experience in an EA research role, no graduate training), I had guessed multipliers of 1.08 and 1.12 on the value of my research in the ~10 years after completing graduate training, for a master's and PhD, respectively. For the remaining years of a research career after that, I had estimated 1.01 and 1.02. Under these assumptions, the total output of a nonprofit research career with or without a master's looks nearly identical for me; the output after completing a PhD looks somewhat worse. However, with the average values from the Google form then the output looks much better with a master's than without and with a PhD than with just a master's. Using the more pessimistic values from other EA nonprofit researchers, or respondents without graduate training, the order is still undergrad only < master's < PhD, though the differences are smaller. In my case, tuition fees seem unlikely to affect these calculations much (see the notes on the rough models sheet).

Of course, which option is best for any individual also depends on numerous other career strategy considerations.  For example, let's think about "option value." Which options are you likely to pursue if research in EA nonprofits doesn't work out or you decide to try something else? Pursuing graduate training might enable you to test your fit with academia and pivot towards that path if it seems promising, but if your next best option is some role in a nonprofit that is unrelated to research (e.g. fundraising), then graduate education might not be as valuable.

I decided to post here partly in case others would benefit, and partly because I'm interested in feedback on/critiques of my reasoning, so please feel free to be critical in the comments!

*For both questions, I noted: "There are many complications and moderating factors for the questions below, but answering assuming the "average" for all other unspecified variables could still be helpful.)" and "1 = the same as if they just had a bachelor's; numbers below 1 represent reduced impact, numbers above 1 represent increased impact."
**These questions were pretty simplified, not permitting people to select multiple options.
*** Here, for simplicity,  I assumed that: 
- You would produce no value while doing your graduate training, which seems likely to be false, especially during (the later years of) a PhD. 
- The value of 1 year after your graduate education was the same as 1 year before retirement, which seems likely to be false.

In a short Google Form, posted on the Effective Altruism Researchers and EA Academia Facebook groups,  I provided the above paragraph and then asked: "If, as well as an undergraduate/bachelor's degree, they start their research career at EA nonprofits with a master's degree in a relevant field, how many "units" of impact do you expect that they would produce each year for the first ~10 years of work?"* The average response, from the 8 respondents, was 1.7.

The mechanism may not be causal. If you're conditioning on type of person who can get accepted into graduate programs + get funding + manage to stick with a PhD program, you are implicitly drawing on a very different pool of people than if you don't condition on this.

That's a good point -- my intention was that it would be the same individual in each instance, just with or without the training, but I didn't word the survey question clearly to reflect that.

It's an interesting analysis. Just a thought - since the value of 1 unit is up to the responder if I've understood correctly, it might be more meaningful to calculate ratios of the responses for each person and average these rather than average the responses to each part - for the latter, if any responder picked small "unit" sizes and correspondingly gave large numerical values, they would make an outsized contribution. Calculating ratios first cancels out whatever "unit" people have decided on. Though it should only matter much if people's "units" differ considerably in size.

Hey Jamie, thanks for doing this, I find the results interesting. Just want to point out what I think are two small typos that made it harder to understand what you wrote:

I asked a free-text response question: "Do you think that the value of graduate training would increase/compound, or decrease/discount, the got further into their career?" 4 respondents wrote that the value of graduate training would decrease/discount the got further into their career

Could you correct what you put above?

Also, I'm curious on 

1. What Master's or Ph.D degrees  are you considering to take?

2. What do you think would be a good Master's or Ph.D degree to take for the average "generalist" researcher at an EA / longtermist non-profit (if this is different from what you personally would take)?

Thanks!

Oops, I meant "the further they got"

Psychology, sociology, (history), (political science). I imagine that that's an unusually broad range to be considering, but I didn't want to rule anything out prematurely. My undergraduate was in history but my research in nonprofits has been much more social science-y, and a bit more quantitative.

I imagine that there's a very broad range that could be on the table. I haven't thought about this question in general that much for "EA / longtermist" research orgs. For effective animal advocacy research organisations, my main guesses would be the same as the list above, plus economics. But there could be others that I haven't thought about, related to those options, or an unusually good fit for some individuals etc.

How did Nick Bostrom come up with the "Simulation argument"*? 

Below is an answer Bostrom gave in 2008. (Though note, Pablo shares a comment below that Bostrom might be misremembering this, and he may have taken the idea from Hans Moravec.)

"In my doctoral work, I had studied so-called self-locating beliefs and developed the first mathematical theory of observation selection effects, which affects such beliefs. I had also for many years been thinking a lot about future technological capabilities and their possible impacts on humanity. If one combines these two areas – observation selection theory and the study of future technological capacities – then the simulation argument is only one small inferential step away.

Before the idea was developed in its final form, I had for a couple of years been running a rudimentary version of it past colleagues at coffee breaks during conferences. Typically, the response would be “yeah, that is kind of interesting” and then the conversation would drift to other topics without anything having been resolved.

I was on my way to the gym one evening and was again pondering the argument when it dawned on me that it was more than just coffee-break material and that it could be developed in a more rigorous form. By the time I had finished the physical workout, I had also worked out the essential structure of the argument (which is actually very simple). I went to my office and wrote it up.

(Are there any lessons in this? That new ideas often spring from the combining of two different areas or cognitive structures, which one has previously mastered at sufficiently a deep level, is a commonplace. But an additional possible moral, which may not be as widely appreciated, is that even when we do vaguely realize something, the breakthrough often eludes us because we fail to take the idea seriously enough.)"

 

Context for this post:

  • I'm doing some research on "A History of Robot Rights Research," which includes digging into some early transhumanist / proto-EA type content. I stumbled across this.
  • I tend to think of researchers as contributing either more through being detail oriented -- digging into sources or generating new empirical data -- or being really inventive and creative. I definitely fall into the former camp, and am often amazed/confused by the process of how people in the latter camp do what they do. Having found this example, it seemed worth sharing quickly.

 

*Definition of the simulation argument: "The simulation argument was set forth in a paper published in 2003. A draft of that paper had previously been circulated for a couple of years. The argument shows that at least one of the following propositions is true: (1) the human species is very likely to go extinct before reaching a “posthuman” stage; (2) any posthuman civilization is extremely unlikely to run a significant number of simulations of their evolutionary history (or variations thereof); (3) we are almost certainly living in a computer simulation. It follows that the belief that there is a significant chance that we will one day become posthumans who run ancestor-simulations is false, unless we are currently living in a simulation. A number of other consequences of this result are also discussed. The argument has attracted a considerable amount of attention, among scientists and philosophers as well as in the media."

Note that Hans Moravec, an Austrian-born roboticist, came up with essentially the same idea back in the 1990s. Bostrom was very familiar with Moravec's work, so it's likely he encountered it prior to 2003, but then forgot it by the time he made his rediscovery.

It's quite common:

"Cryptomnesia occurs when a forgotten memory returns without its being recognized as such by the subject, who believes it is something new and original. It is a memory bias whereby a person may falsely recall generating a thought, an idea, a tune, a name, or a joke,[1] not deliberately engaging in plagiarism but rather experiencing a memory as if it were a new inspiration."

https://en.wikipedia.org/wiki/Cryptomnesia

I haven't read Moravec's book very thoroughly, but I ctrl+f'd for "simulation" and couldn't see anything very explicitly discussing the idea that we might be living in a simulation. There are a number of instances where Moravec talks about running very detailed simulations (and implying that these would be functionally similar to humans). It's possible (quite likely?) Bostrom didn't ever see the 1995 article where Moravec "shrugs and waves his hand as if the idea is too obvious." 

Either way, it seems true that (1) the idea itself predates Bostrom's discussion in his 2003 article, (2) Bostrom's discussion of this specific idea is more detailed than Moravec's.

Bostrom (2003) cited Moravec (1988), but not for this specific idea -- it's only for the idea that "One estimate, based on how computationally expensive it is to replicate the functionality of a piece of nervous tissue that we have already understood and whose functionality has been replicated in silico, contrast enhancement in the retina, yields a figure of ~10^14 operations per second for the entire human brain."

But yeah, his answer to the question "How did you come up with this?" in the 2008 article I linked to in the original post seems misleading, because he doesn't mention Moravec at all and implies that he came up with the idea himself.

Oh, nice, thanks very much for sharing that. I've cited Moravec in the same research report that led me to the Bostrom link I just shared, but hadn't seen that article and didn't read Mind Children fully enough to catch that particular idea.

Buying a house will probably save you lots of money, which you can later donate, but it might not make much difference (and may work out as negative) in terms of your ability to do good.

It seems like common sense that buying a house saves you from wasting money on rent and works out better, financially, in the long term. But earlier this year, John Halstead wrote a blogpost providing a bunch of reasons not to buy a house.

I had another look at John's calculations. I kept the basic calculations the same, but added a few considerations and re-checked the appropriate numbers for London (where I live). I also added various different tabs of the spreadsheet to compare things like variations in interest rates, property prices, timeframes for buying and selling, and other costs. In every scenario, unless there's a housing crash shortly after you buy, it looks like buying comes out as far, far better, from a financial perspective. In the best guess, realistic scenario, buying came out as about £550,000 better after 10 years. John has also had another look at his calculations since his post and seems more optimistic about buying. I haven't looked at figures and costs for countries other than the UK, but the differences are so large that I'd quite surprised if investing and renting came out as more favourable in (m)any countries.

This doesn't address the concerns about buying in John's blog post (e.g. that you will only be able access the money when you're older). But if you're interested in patient philanthropy, and are happy to donate more accumulated wealth in several decades' time (when you downsize or die) rather than having a strong preference for donating less sooner, then buying a house looks better. (For discussion, see "Giving now vs giving later" and "How becoming a ‘patient philanthropist’ could allow you to do far more good")

Despite the large raw difference between buying vs. renting and investing, these differences might mean surprisingly little, in terms of ability to do good in the world, if you apply a discount to the value of future money to calculate its net present value. If you apply a high discount rate, then the gains are practically zero. Indeed, some EA orgs express a strong preference for money sooner rather than later. I haven't worked this bit out properly, but if you take these numbers literally (and reject patient philanthropy) it might be better to just donate sooner rather than to save up for a deposit.

I also live in London, and bought a house in April 2016. So I've thought about these calculations a fair bit, and happy to share some thoughts here:

One quick note on your calculations is that stamp duty has been massively, but temporarily, cut due to COVID. You note it's currently £3k on a £560k flat. Normally it would be £18k. You can look at both sets of rates here.

When I looked at this, the calculation was heavily dependent on how often you expect to move. Every time you sell a home and buy a new one you incur large fixed costs; normally 2-4% of purchase price in stamp duty, 1-3% in estate agent fees, and a few other fixed costs which are minor in the context of the London property market but would be significant if you were looking at somewhere much cheaper (legal fees etc.). All of this seems well accounted for in your spreadsheet, but it means that if you expect to move every 1-3 years then the ongoing saving will be swamped by repeatedly incurring these costs.

There's also a somewhat fixed time cost; when I bought a home I estimate I spent the equivalent of 1 week of full-time work on the process (not the moving itself), most of which was spent doing things I wouldn't have needed to do for rented accomodation.

All told, for my personal situation in 2016 I thought I should only buy if I expected to stay in that flat for at least 5 years, and to make the calculation clear I would have wanted that to be more like 10 years. As a result, buying looks much better if you have outside factors already tying you down; a job that is very unlikely to be beaten, kids, a city you and/or your partner loves, etc.

This is a much closer calculation that will come out with your numbers, because I don't think a 7.5% housing return is a sensible average to use going forward. I had something like a 2% real (~4% nominal, but I generally prefer to think in terms of real) estimate pencilled in for housing, and more like a 5% real (7% nominal) rate pencilled in for stocks. There's a longer discussion there, but the key point I would make is that interest rates fallen dramatically in recent decades, boosting the value of assets which pay out streams of income, i.e. rent/dividends. It's unclear to me that the recent trend towards ever lower rates can go much further, and markets don't expect it to, so I didn't want to tacitly assume that.

So far, that conservative estimate was much closer, London house prices rose by roughly 1.5% annualised between April 2016 and March 2020. Then a pandemic hit, but happy to exclude that from 'things I could have reasonably expected'.

Does this include how it might limit your ability to move for work, which might be the most important factor in salary/impact?

Good point although I guess there's always the possibility of moving and renting out your home (and then renting yourself in the place you move to)

No, I didn't list the "other" pros and cons, this is just the financial perspective.

I don't have a good sense of how difficult it is to move houses. But my guess is that a decision to move for work or not wouldn't be that dependent on selling a house. E.g. you either want to stay, come what may, because of reasons like friends, family, partners etc, or you're personally happy to move, and wouldn't mind selling then renting?

Thanks for this Jamie. Useful to know that the outcome can differ according to person/location. I reckon I'll do this exercise for myself at some point. A few quick questions/comments (I haven't looked at this in detail so apologies if I've missed anything):

  • Have you identified the key difference(s) between your calculation and John's calculation that leads to the different result? It might be helpful to call this out
    • E.g. is it mainly driven by higher rental costs in London / the fact that you've assumed a smaller deposit for the house etc.
  • Pretty minor point, but the 3.5% discount rate should decline over time and it doesn't seem you've factored this in (it shouldn't really change much though as you're not looking over a very long time scale)
  • I'm not really sure how useful the 3.5% discount rate is for philanthropists, in particular EA philanthropists. It includes a discount of future utility on account of the future being less morally valuable, which is something that philosophers have pretty much rejected and is quite counter to EA philosophy. There are good reasons for EA philanthropists to discount (more on that here and here) but I don't there's a good reason for us to expect it to lead to a 3.5% rate. It could actually be higher or lower depending on an individual's preferred cause area/underlying ethical views. The general point that you're making that buying a house only provides access to money when older, and therefore that this becomes subject to discounting is a very useful one though.
  • Doesn't John's calculation also say buying is better? Or am I missing something?
Have you identified the key difference(s) between your calculation and John's calculation that leads to the different result? It might be helpful to call this out

No, I haven't gone through and done that. Actually, John's calculations still come out in favour of buying from a financial perspective, albeit by a much smaller margin than in my calculations; I think he was put off for other reasons.

Pretty minor point, but the 3.5% discount rate should decline over time and it doesn't seem you've factored this in (it shouldn't really change much though as you're not looking over a very long time scale)

I'm probably doing the maths completely wrong on that bit... suggestions for correct formula to use are welcome. Commenting on the sheet is currently on if you want to comment on directly.

It could actually be higher or lower depending on an individual's preferred cause area/underlying ethical views. The general point that you're making that buying a house only provides access to money when older, and therefore that this becomes subject to discounting is a very useful one though

Yeah I haven't got my head very thoroughly round the various arguments on this, so thanks for sharing. My impression was also that using 3.5% didn't make much sense and should probably either go lower than that (for "patient" reasons) or much higher (if you think opportunities for cost-effective giving will diminish rapidly for various reasons.


Some relevant context I probably should have added to the post was that I did this calculation because I was very surprised at John's overall conclusion and wanted to check it, and, despite this not being very thorough or anywhere near my research "expertise", I thought other people might benefit from these rough and ready efforts, so decided to share.