Estimating the marginal impact of outreach

This post is an entry for  $5k challenge to quantify the impact of 80,000 hours' top career paths

One of the career paths evaluated by 80000 hours is helping to build the effective altruism movement: this is currently ranked in fifth position, behind AI technical research, AI governance, biorisk and organisation entrepeneurship. This post presents a model for the marginal number of additional individuals an individual devoted to outreach would attract, and finds with very high confidence that on the margin outreach to build effective altruism further is higher impact than working on existential misk mitigation directly.

Discount rate estimation

The value of outreach compared to immediately working on existential risk heavily depends on our discount rate. This is because the benefits in terms of basis points of existential risk reduced are entirely in the future for outreach (in the form of additional researchers) while immediately working on research will bring gains sooner. Two factors seem relevant for our discount rate: the probability that humanity ceases to exist before the additional researchers can have an impact and the marginal value of labour over time^[1]. The best database we are aware of for total existential risk is here: of these, the only bounded annual (or annual-equivalent assuming constant risk over time) estimates for the risk are 0.19%, 0.21%, 0.11% and  0.2%. These have a geometric mean 0f 0.17%, and a standard deviation of 0.046%, which will be used here.

For the latter component, if constant returns to scale and variable exogeneity are assumed, a Cobb-Douglas production function can be taken, with $Y$ as output, however defined, $A$ as labour productivity, $L$ as labour, $K$ as capital and $\alpha$ as the capital elasticity of output:

If differentiated with respect to labour, this then yields:

Thus:

So if capital allocated to EA is growing at a faster rate than labour ($\beta >\gamma$), our discount rate should be negative with respect to time: if labour is growing faster, it should be positive, making the simplifying assumption that EAs are the only group in the world producing moral value . Intuitively, this occurs because capital and labour are varying at some rates exogenously and we wish our level of capital per worker to be as close to constant over time as possible due to diminishing marginal returns to all inputs.

Is capital or labour growing faster? This 80000 hours article from 2021 estimated at the time that the total capital allocation was growing by around 37% per year, and labour by 10-20%, which would imply deeply negative discount rates if these figures still held. However, the two largest components of the increase in capital allocated were primarily in the form of FTX and Facebook stock. With the former now worthless, and the latter having declined to less than one third of its value at the time of the writing of the article, the overall capital stock allocated to EA has plunged in value while the labour stock remains almost unaffected. Using the same methodology as the above article, noting that Dustin Moskovitz’s net worth now only stands at $6.5 billion at the time of writing, reduces the increase in funding over the past 7 years to $2.95 billion nominally (from a starting level of ~$10 billion), or a real terms increase of only $0.35 billion, or 2.8%: 0.39% annually. Capital growth will be modelled as lognormal, as a heavier tail distribution feels appropriate due to the possibility of another FTX, with mean $e^{ln\left(0.39\right)}$ and log standard deviation of 10%, due to the potential for capital growth of 37% absent FTX's collapse and Facebook's decline in stock value. Labour growth is considerably more stable than capital growth, but still volatile, so will be assumed to be a constant rate of 10% with a standard deviation of 5%, with the lower bound taken as the mean due to difficulties in higher growth rates (30% would imply that 4% of the world would be engaged in EA-relevant work by 2060, which seems highly implausible). $\alpha$ will be assumed to be the same level as the economy as a whole, at 0.4.

The model continues with the uncertainties above, but they give a mean discount rate of 1.1%, which values an asset that returns $1 every year to infinity at $93.42. Interestingly, as the annual existential risk probability is low in comparison to the potential difference between capital and labour growth rates, in worlds with very high capital growth compared to labour the discount rate should be negative, so an EA tomorrow should be preferred to one today, as is reflected in the distribution above possessing a long tail with negative discount rates.

Other parameter estimation

The remaining estimation of marginal value depends on three variables, if individuals are assumed to be recruiting individuals of identical value to themselves: the average number of additional members of EA the typical person devoted to outreach can acquire, the path of individual effectiveness at this over time and the time interval between making an individual available for EA and them providing an impact. 

For the first of these, three programmes seem to provide relevant data: Leaf 2022 is estimated to have produced 9.2 HEAs ^[2] (Highly Engaged EAs: 90% 0.77 to 22) per FTE (Full-Time Equivalent) year ^[3], while EA Stanford produced perhaps 6.7 HEAs per FTE year, assuming 1 FTE year from the organiser in question and 0.5 from others: however, as it also worked towards accelerating the careers of those already interested in EA. EA Malaysia appears to have produced 1 additional HEA (Cat >3) with 0.2 FTEs worth of inputs, and thus a cost-effectiveness of 5 HEAs per FTE, absent consideration of individuals it moved to below HEA but still impactful level. However, all of these are of course average not marginal costs, so need adjusting to calculate marginal benefit. If the capital elasticity of output is the same as above, then the marginal effect of any given increase in the labour stock are 60% of the current average. However, as EA Stanford had substantial impacts beyond HEA recruitment, raising its value to 10 seems reasonable to model strict optimisation for the task in question here seems reasonable, which assuming the same uncertainty ratios as in Leaf provides an arithmetic mean value of 12.1 HEAs/FTE (90% 0.68 to 18.5) distributed roughly normally. This model assumes a normal distribution of mean 7.26 and standard deviation 2.673 ^[4]. 

For the path of facilitator effectiveness over time, much research, such as Rice 2003 and Rockoff 2004 find that teacher effectiveness, likely a decent proxy for how convincing facilitators are, is correlated with experience, but with minimal gains after about two years of teaching, implying that facilitator effectiveness is near-flat for the longer-horizons view for the purposes of this blog post, which it will be treated as such for this model. The third of these depends heavily on the type of outreach conducted. If the outreach is assumed to be that which maximises elasticity of career adjustment with respect to inputs, namely to pre-university students, then the impact delay is likely at least 5 years, depending on exact student ages. For the purposes of the model here, a delay of 10 years with a standard deviation of 2 seems reasonable (absent the usage of potentially more effective strategies). 

Results

Table 1: Distribution of estimates of present value of HEAs produced over 30 year-career in outreach

Table 2: Correlation coefficients between input variables and outputs

These results counterintuitively imply that the current marginal individual would be substantially higher marginal impact working to expand effective altruism than working on maximising the reduction in existential risk today, with 99.7% confidence The most significant uncertainty comes in the form of the capital growth rate which, as the results of the past few months have shown, is extremely volatile and path dependent, and reducing the uncertainty in the long-run path of this would do most for reducing the range of the estimates above. The principal shortcoming of this post is a lack of estimation of the range of personal fit levels, which is vital for determining decisions regarding the allocation of heterogeneous individuals rather than homogenous capital: given the scale of the marginal effects above, this would be a promising avenue for further investigation, and would be desireable to undertake before substantial transfers of labour and capital between projects, as the median AI alignment researcher may well be an order of magnitde or more more productive in their current role than outreach. 

Code appendix

The code for this model is below, and was done directly in the R programming language.

library(ggplot2)
e <- 2.7182818284590452353602874713527
samples <- 10^4
discount_rate_XR <- rnorm(samples,(0.0011*0.0019*0.0020*0.0021)^0.25, sd(c(0.0011,0.0019,0.002,0.0021)))
for(i in 1:samples){discount_rate_XR[i] <- max(0,discount_rate_XR[i])} #existential risk is strictly non-negative
labourgrowth <- rnorm(samples,0.10,0.05)
capitalgrowth <- e^rnorm(samples,log(0.039),log(10^0.5))
alpha <- 0.4 #capital elasticity of output, assumes constant returns to scale
discount_rate_scalereturns <- 1-((1+labourgrowth)^(-alpha))*((1+capitalgrowth)^alpha)
discount_factor <- 1-(discount_rate_scalereturns+discount_rate_XR)
averageimpactdelay <- rnorm(samples, 10, 2)
averageoutreach <- rnorm(samples,0.6*(5+10+9.2)/3, 2.673)
for(i in 1:length(averageoutreach)){if(averageoutreach[i] < 0){averageoutreach[i] <- sample(averageoutreach,1)}}
effectivenessfunction <- seq(1,1.0000001,0.0000001/30) #relates increase in relative effectiveness over time
presenteffectivenessvalue <- c()
for(i in 1:samples){temp_sum <- 0
  for(j in 1:length(effectivenessfunction)){temp_sum <- temp_sum + averageoutreach[i]*effectivenessfunction[j]*discount_factor[i]^(j+averageimpactdelay[i])}
  presenteffectivenessvalue <- append(presenteffectivenessvalue, temp_sum)}
presenteffectivenessvalue[is.na(presenteffectivenessvalue)] <- sample(presenteffectivenessvalue,1)
correctedpresent <- c()
for(i in 1:length(presenteffectivenessvalue)){correctedpresent <- append(correctedpresent, max(1,presenteffectivenessvalue[i]))} #removing negative values to allow for logarithms to be taken
hist(log(presenteffectivenessvalue))
hist(discount_factor)
mean(discount_factor)
1/(1-mean(discount_factor)) #valuation of a $1 asset to infinity
sortedpresenteffectivenessvalue <- sort(presenteffectivenessvalue)
sortedpresenteffectivenessvalue[samples/1000]
sortedpresenteffectivenessvalue[samples/20]
sortedpresenteffectivenessvalue[samples/10]
sortedpresenteffectivenessvalue[samples/4]
sortedpresenteffectivenessvalue[samples/2]
sortedpresenteffectivenessvalue[3*samples/4]
sortedpresenteffectivenessvalue[9*samples/10]
sortedpresenteffectivenessvalue[19*samples/20]
sortedpresenteffectivenessvalue[999*samples/1000]
cor(discount_rate_XR,presenteffectivenessvalue)
cor(labourgrowth,presenteffectivenessvalue)
cor(capitalgrowth, presenteffectivenessvalue)
cor(averageimpactdelay,presenteffectivenessvalue)
cor(averageoutreach,presenteffectivenessvalue)

1. ^{^}

   The survival of EA seems of more limited relevance as even worlds without anything officially EA could persist in outreach for specific sub-fields such as AI risk, albeit with more limited ability to enact degree change among pre-university students. To the extent that fields such as AI safety were destroyed, this would also prevent those who had tried to work in such fields directly from having an impact: although this would still change the ratio of benefits due to the impact delay variable above, this would not be sufficient to overturn the fundamental result of an overwhelming result in favour of outreach, especially as unlike teaching research productivity is likely a function with at least one time-increasing component.

2. ^{^}

   Appropriate Guesstimate available upon request from [email protected].

3. ^{^}

   The same estimation model uses facilitator estimates that the median student attracted to Leaf was substantially higher impact (2.4, 90% 0.67 to 6.6) than the median EA generally: both in the interests of keeping this model conservative, with the headline result not requiring this assumption, and also due to participant impact on a smaller, more selective version of the program (only 2/3 of eventual student numbers) than eventually ran being forecast as lower ex post (acquisition process for Guesstimate similar to Footnote 2), it will be assumed the median individual attracted by the outreach efforts above is as productive as the median current HEA. Additionally, it seems likely that the higher impact individuals were also more likely to have heard about EA regardless, due to greater likelihood of entering environments such as top universities where such ideas would be discussed, meaning that the median counterfactual student may be lower impact than the median student, and likely considerably closer to the mean. 

^{^}

Although this considers constant returns to scale effects, implicitly this neglects personal fit effects. As outreach to individuals below the age of 18 is still extremely small-scale, this seems not especially egregious, but will result in a slight overestimate of marginal impact, and would become substantial if EA outreach for younger age groups expanded considerably further.

Show all footnotes