"To see the world as it is, rather than as I wish it to be."

I work for the EA research nonprofit Rethink Priorities. Despite my official title, I don't really think of the stuff I do as "research." In particular, when I think of the word "research", I think of people who are expanding the frontiers of the world's knowledge, whereas often I'm more interested in expanding the frontiers of my knowledge, and/or disseminating it to the relevant parties.

I'm also really interested in forecasting.

People may or may not also be interested in my comments on Metaculus and Twitter:




Why did EA organizations fail at fighting to prevent the COVID-19 pandemic?

(Going entirely from Twitter etc and not having read the original papers or grant proposals myself) 

I don't think what the WIV did was central to "gain-of-function" research, at least according to Marc Lipsitch. My understanding is that Shi Zhengli (Obviously not an unbiased source) from WIV claims that their work isn't gain-of-function because they were studying intermediate hosts, rather than deliberately trying to make pathogens more virulent or transmissible.* 

My own opinion is that GoF has become ill-defined and quite political, especially these days, so we have to be really careful about precisely what we mean when we say "GoF"

I realize that this sound like splitting hairs, but the definitional limits are important, because Lipsitch's 2014 paper(s) about the dangers of GoF were predicated on a narrow definition/limits of GoF (the clearest-cut cases/worst offenders), while the claims about lab escape, if true, comes from a broader model of GoF. 

(Two caveats

1) I want to be clear that I personally think that whether it's called GoF or not, studying transmission from intermediate hosts is likely a bad idea at current levels of lab safety.
2) I don't feel particularly qualified to judge this ). 

*I wanted to find the source but couldn't after 3 minutes of digging. Sorry.

EA Forum feature suggestion thread

Do people not find it viable to post under a pseudonym? Is your worry about coming across as dishonest? 

Linch's Shortform

I've started trying my best to consistently address people on the EA Forum by username whenever I remember to do so, even when the username clearly reflects their real name (eg Habryka). I'm not sure this is the right move, but overall I think this creates slightly better cultural norms since it pushes us (slightly) towards pseudonymous commenting/"Old Internet" norms, which I think is slightly better for pushing us towards truth-seeking and judging arguments by the quality of the arguments rather than be too conscious of status-y/social monkey effects.

 (It's possible I'm more sensitive to this than most people). 

I think some years ago there used to be a belief that people will be less vicious (in the mean/dunking way) and more welcoming if we used Real Name policies, but I think reality has mostly falsified this hypothesis.

Linch's Shortform

Thanks for the karmically beneficial tip! 

I've now posted this question in its own right.


2018-2019 Long Term Future Fund Grantees: How did they do?

I assumed that you'd be one of the people in the 

5 grants worth a total of $195,000 which I [NunoSempere] tagged didn’t evaluate because of a perceived conflict of interest

bucket, fwiw.

2018-2019 Long Term Future Fund Grantees: How did they do?

Conversely, if I do see them getting some successes, I would update upwards on the mean and the standard deviation of the power law distribution from which their impact is drawn.

It makes sense to update upwards on the mean, but why would you update on the standard deviation from n of 1? (I might be missing something obvious)

Taboo "Outside View"

I mostly use outside views to mean reference classes, but I agree that this term has expanded to mean more than is originally denoted. I'm not sure how big a problem this is in practice; I think  by default phrases in natural language expands to mean more than their technical beginnings (consider phrases like "modulo", "pop the stack," etc). My intuition is that zealously guarding against this expansion by specifying new broader words (rather than being precise in-context) seems quite doomed as an overall enterprise, though it might buy you a few years. 

A related point is that if we do go with "reference classes" as the preferred phrase, we should be cognizant that for most questions there's a number of different relevant reference classes, and saying that a particular reference class we've picked is the best/only reference class is quite a strong claim, and (as EliezerYudkowsky alludes to) quite susceptible to motivated reasoning. 

Taboo "Outside View"

All those experimental results on people doing well by using the outside view are results on people drawing a new sample from the same bag as previous samples.  Not "arguably the same bag" or "well it's the same bag if you look at this way", really actually the same bag: how late you'll be getting Christmas presents this year, based on how late you were in previous years

Hmm, I'm not convinced that this is meaningfully different in kind rather than degree. You aren't predicting a randomly chosen holdout year, so saying that 2021 is from the same distribution as 2011-2020 is still a take. "X thing I do in the future is from the same distribution of all my attempts in past years*" is still a judgement call, albeit a much easier one than AI timelines. 

I agree with (part of) your broader point that incareful applications of the outside view and similar vibes is very susceptible to motivated reasoning (including but not limited to the absurdity heuristic), but I guess my take here is that we should just be more careful individually and more willing to point out bad epistemic moves in others (as you've often done a good job of!) as a community. 

All our tools are limited and corruptible, and I don't think on balance reference class forecasting is more susceptible to motivated reasoning than other techniques.

*are you using your last 10 years? since you've been an adult? all the years you've been alive? 

Load More