alex lawsen
Bio
I work on AI Grantmaking at Open Philanthropy. Comments here are posted in a personal capacity.
Posts 16
Comments363
- Use more examples
- Start with voice transcription
- Get a subscription which allows you to use projects (and then use them)
I think that this:
> but the intuition that calls this model naive is driven by a sense that it's going to turn out to not "actually" be 2 additional people, that additionality is going to be lower than you think, that the costs of getting that result are higher than you think, etc. etc.
is most of the answer. Getting a fully counterfactual career shift (that person's expected career value without your intervention is ~0, but instead they're now going to work at [job you would otherwise have taken, for at least as long as you would have]) is a really high bar to meet. If you did expect to get 2 of those, at equal skill levels to you, then I think the argument for 'going meta' basically goes through.
In practice, though:
- People who fill [valuable role] after your intervention probably had a significant chance of finding out about it anyway.
- They also probably had a significant chance of ending up in a different high-value role had they not taken the one you intervened on.
How much of a discount you want to apply for these things is going depend a lot on how efficiently you expect the [AI safety] job market to allocate talent. In general, I find it easier to arrive at reasonable-seeming estimates for the value of career/trajectory changes by modelling them as moving the the change earlier in time rather than causing it to happen at all. How valuable you expect the acceleration to be depends on your guesses about time-discounting, which is another can of worms, but I think is plausibly significant, even with no pure rate of time preference.
(This is basically your final bullet, just expanded a bit.)
I thought it seemed worth flagging that Open Philanthropy recently recommended a grant to Palisade Research. I investigated the grant, and am happy to see that Michael is also excited about their work and included them in his top five.
Inspect is open-source, and should be exactly what you're looking for given your stated interest in METR
Why do you think superforecasters who were selected specifically for assigning a low probability to AI x-risk are well described as "a bunch of smart people with no particular reason to be biased"?
For the avoidance of doubt, I'm not upset that the supers were selected in this way, it's the whole point of the study, made very clear in the write-up, and was clear to me as a participant. It's just that "your arguments failed to convince randomly selected superforecasters" and "your arguments failed to convince a group of superforecasters who were specifically selected for confidentiality disagreeing with you" are very different pieces of evidence.
The smart people were selected for having a good predictive track record on geopolitical questions with resolution times measured in months, a track record equaled or bettered by several* members of the concerned group. I think this is much less strong evidence of forecasting ability on the kinds of question discussed than you do.
*For what it's worth, I'd expect the skeptical group to do slightly better overall on e.g. non-AI GJP questions over the next 2 years, they do have better forecasting track records as a group on this kind of question, it's just not a stark difference.
The first bullet point of the concerned group summarizing their own position was "non-extinction requires many things to go right, some of which seem unlikely".
This point was notably absent from the sceptics summary of the concerned position.
Both sceptics and concerned agreed that a different important point on the concerned side was that it's harder to use base rates for unprecedented events with unclear reference classes.
I think these both provide a much better characterisation of the difference than the quote you're responding to.