In case it's helpful to people considering the one working with me, I wrote a bit about why I'm personally excited about it here. My colleague Catherine also wrote this comment about what working with me is like.
I think your suggested format is a significant upgrade on the (much more common, unfortunately) "group brainstorm" set up that Ollie is criticising, for roughly the reasons he outlines; It does much better on "fidelity per person-minute".
Individual brainstorming is obviously great for this, for the reasons you said (among others).
Commenting on a doc (rather than discussing in groups of 6-8) again allows many more people to be engaging in a high-quality/active way simultaneously.
It also seems worth saying that choosing questions well, which means they are:
worth answering
difficult or contextual enough that multiple people's thought is required to get to a good answer
scoped well enough that progress can actually be made by a group in the relevant time
is a) necessary for group discussion to be worthwhile, b) difficult, and c) significantly more difficult for a group of mixed ability and context, whom you don't know well. c, of course, applies much more strongly in the context Ollie is primarily concerned with (EAGs and similar events), to the one you're describing (research team meeting).
I think that, almost without exception, if event sessions want to incorporate some discussion, they should start with the 'individual silent thought' exercise you mention, and then expand to pairs (with some 3s to stop people needing to do a lot of rearranging odd numbers of people). There are lots of reasons that this works better than larger groupings, but again Ollie's heuristic of 'fidelity per person-minute' is one. A less obvious one is that minimising the distance between speaker and listener allows conversation volume to be much quieter, and if you think about how volume scales with distance, this more than outweighs the effect of having more people talking simultaneously.
Feeding back to the whole group from these discussions can happen (and be great), but is worth parallelising where possible, e.g. by commenting on a central gdoc as you suggest, or with a Slido, where people are encouraged to submit questions that their paired/small group discussions did not resolve, which the facilitator can then answer, or suggest steps to answer, at the end.
*I spent a ~decade as a teacher, and have facilitated many highly-reviewed workshops during and since, so I feel like I do have reasonable grounds to claim authority, but this is a joke, it just seemed like a funny epistemic status.
I think that this: > but the intuition that calls this model naive is driven by a sense that it's going to turn out to not "actually" be 2 additional people, that additionality is going to be lower than you think, that the costs of getting that result are higher than you think, etc. etc.
is most of the answer. Getting a fully counterfactual career shift (that person's expected career value without your intervention is ~0, but instead they're now going to work at [job you would otherwise have taken, for at least as long as you would have]) is a really high bar to meet. If you did expect to get 2 of those, at equal skill levels to you, then I think the argument for 'going meta' basically goes through.
In practice, though: - People who fill [valuable role] after your intervention probably had a significant chance of finding out about it anyway. - They also probably had a significant chance of ending up in a different high-value role had they not taken the one you intervened on.
How much of a discount you want to apply for these things is going depend a lot on how efficiently you expect the [AI safety] job market to allocate talent. In general, I find it easier to arrive at reasonable-seeming estimates for the value of career/trajectory changes by modelling them as moving the the change earlier in time rather than causing it to happen at all. How valuable you expect the acceleration to be depends on your guesses about time-discounting, which is another can of worms, but I think is plausibly significant, even with no pure rate of time preference.
(This is basically your final bullet, just expanded a bit.)
I thought it seemed worth flagging that Open Philanthropy recently recommended a grant to Palisade Research. I investigated the grant, and am happy to see that Michael is also excited about their work and included them in his top five.
Why do you think superforecasters who were selected specifically for assigning a low probability to AI x-risk are well described as "a bunch of smart people with no particular reason to be biased"?
For the avoidance of doubt, I'm not upset that the supers were selected in this way, it's the whole point of the study, made very clear in the write-up, and was clear to me as a participant. It's just that "your arguments failed to convince randomly selected superforecasters" and "your arguments failed to convince a group of superforecasters who were specifically selected for confidentiality disagreeing with you" are very different pieces of evidence.
I'm excited about these roles!
In case it's helpful to people considering the one working with me, I wrote a bit about why I'm personally excited about it here. My colleague Catherine also wrote this comment about what working with me is like.
FAR AI posts recordings of talks from the events they organise on YouTube.
Many, perhaps most people, would be well advised to beware surprising and suspicious convergence.
The rest would be well advised to beware excessive wariness of surprising and suspicious convergence.
[Epistemic status: argument from authority*]
I think your suggested format is a significant upgrade on the (much more common, unfortunately) "group brainstorm" set up that Ollie is criticising, for roughly the reasons he outlines; It does much better on "fidelity per person-minute".
Individual brainstorming is obviously great for this, for the reasons you said (among others).
Commenting on a doc (rather than discussing in groups of 6-8) again allows many more people to be engaging in a high-quality/active way simultaneously.
It also seems worth saying that choosing questions well, which means they are:
is a) necessary for group discussion to be worthwhile, b) difficult, and c) significantly more difficult for a group of mixed ability and context, whom you don't know well. c, of course, applies much more strongly in the context Ollie is primarily concerned with (EAGs and similar events), to the one you're describing (research team meeting).
I think that, almost without exception, if event sessions want to incorporate some discussion, they should start with the 'individual silent thought' exercise you mention, and then expand to pairs (with some 3s to stop people needing to do a lot of rearranging odd numbers of people). There are lots of reasons that this works better than larger groupings, but again Ollie's heuristic of 'fidelity per person-minute' is one. A less obvious one is that minimising the distance between speaker and listener allows conversation volume to be much quieter, and if you think about how volume scales with distance, this more than outweighs the effect of having more people talking simultaneously.
Feeding back to the whole group from these discussions can happen (and be great), but is worth parallelising where possible, e.g. by commenting on a central gdoc as you suggest, or with a Slido, where people are encouraged to submit questions that their paired/small group discussions did not resolve, which the facilitator can then answer, or suggest steps to answer, at the end.
*I spent a ~decade as a teacher, and have facilitated many highly-reviewed workshops during and since, so I feel like I do have reasonable grounds to claim authority, but this is a joke, it just seemed like a funny epistemic status.
I think that this:
> but the intuition that calls this model naive is driven by a sense that it's going to turn out to not "actually" be 2 additional people, that additionality is going to be lower than you think, that the costs of getting that result are higher than you think, etc. etc.
is most of the answer. Getting a fully counterfactual career shift (that person's expected career value without your intervention is ~0, but instead they're now going to work at [job you would otherwise have taken, for at least as long as you would have]) is a really high bar to meet. If you did expect to get 2 of those, at equal skill levels to you, then I think the argument for 'going meta' basically goes through.
In practice, though:
- People who fill [valuable role] after your intervention probably had a significant chance of finding out about it anyway.
- They also probably had a significant chance of ending up in a different high-value role had they not taken the one you intervened on.
How much of a discount you want to apply for these things is going depend a lot on how efficiently you expect the [AI safety] job market to allocate talent. In general, I find it easier to arrive at reasonable-seeming estimates for the value of career/trajectory changes by modelling them as moving the the change earlier in time rather than causing it to happen at all. How valuable you expect the acceleration to be depends on your guesses about time-discounting, which is another can of worms, but I think is plausibly significant, even with no pure rate of time preference.
(This is basically your final bullet, just expanded a bit.)
I thought it seemed worth flagging that Open Philanthropy recently recommended a grant to Palisade Research. I investigated the grant, and am happy to see that Michael is also excited about their work and included them in his top five.
I've seen people wear a very wide range of things at the EAGs I've been to.
Inspect is open-source, and should be exactly what you're looking for given your stated interest in METR
Why do you think superforecasters who were selected specifically for assigning a low probability to AI x-risk are well described as "a bunch of smart people with no particular reason to be biased"?
For the avoidance of doubt, I'm not upset that the supers were selected in this way, it's the whole point of the study, made very clear in the write-up, and was clear to me as a participant. It's just that "your arguments failed to convince randomly selected superforecasters" and "your arguments failed to convince a group of superforecasters who were specifically selected for confidentiality disagreeing with you" are very different pieces of evidence.