Holden Karnofsky


My current impressions on career choice for longtermists

Thanks for the thoughtful comments, Linch.

Response on point 1: I didn't mean to send a message that one should amass the most impressive conventional credentials possible in general - only that for many of these aptitudes, conventional success is an important early sign of fit and potential.

I'm generally pretty skeptical by default of advanced degrees unless one has high confidence that one wants to be on a track where the degree is necessary (I briefly give reasons for this skepticism in the "political and bureaucratic aptitudes" section). This piece only mentions advanced degrees for the "academia," "conceptual and empirical research" and "political and bureaucratic" aptitudes. And for the latter two, these aren't particularly recommended, more mentioned as possibilities.

More generally, I didn't mean to advocate for "official credentials that anyone could recognize from the outside." These do seem crucial for some aptitudes (particularly academia and political/bureaucratic), but much less so for other aptitudes I listed. For org running/building/boosting, I emphasized markers of success that are "conventional" (i.e., they're not contrarian goals) but are also not maximally standardized or legible to people without context - e.g., raises, promotions, good relationships.

Response on point 2: this is interesting. I agree that when there is some highly neglected (often because "new") situation, it's possible to succeed with a lot less time invested. Crypto, COVID-19, and AI safety research of today all seem to fit that bill. This is a good point.

I'm less sure that this dynamic is going to be reliably correlated with the things that matter most by longtermist lights. When I picture "crunch time," I imagine that generalists whose main asset is their "willingness to drop everything" will have opportunities to have impact, but I also imagine that (a) their opportunities will be better insofar as they've developed the sorts of aptitudes listed in this piece; (b) a lot of opportunities to have impact will really rely on having built aptitudes and/or career capital over the long run. 

For example, I imagine there will be a lot of opportunities for (a) people who are high up in AI labs and government; (b) people who know how to run large projects/organizations; (c) people with large existing audiences and/or networks; (d) people who have spent many years working with large AI models; (e) people who have spent years developing rich conceptual and empirical understanding of major potential risk factors, and that these opportunities won't exist for generalists.

"Drop everything and work hard" doesn't particularly seem to me like the sort of thing one needs to get practice with (although it is the sort of thing one needs to be prepared for, i.e., one needs not to be too attached to their current job/community/etc.) So I guess overall I would think most people are getting "better prepared" by building the sorts of aptitudes described here than by "simulating crunch time early." That said, jumping into areas with unusually "short climbs to the top" (like the examples you gave) could be an excellent move because of the opportunity to build outsized career capital and take on outsized responsibilities early in one's career. And I'll reiterate my reservations about "advice," so wouldn't ask you to defer to me here!

My current impressions on career choice for longtermists

I like this; I agree with most of what you say about this kind of work.

I've tried to mostly list aptitudes that one can try out early on, stick with if they're going well, and pretty reliably build careers (though not necessarily direct-work longtermist careers) around. I think the aptitude you're describing here might be more of later-career/"secondary" aptitude that often develops as someone moves up along an "organization building/running/boosting" or "political/bureaucratic" track. But I agree it seems like a cluster of skills that can be intentionally developed to some degree and used in a lot of different contexts.

My current impressions on career choice for longtermists

Thanks for the thoughtful comments!

On your first point: the reason I chose to emphasize longtermism is because:

  • It's what I've been thinking about the most (note that I am now professionally focused on longtermism, which doesn't mean I don't value other areas, but does mean that that's where my mental energy goes).
  • I think longtermism is probably the thorniest, most frustrating area for career choice, so I wanted to focus my efforts on helping people in that category think through their options.
  • I thought a lot of what I was saying might generalize further, but I wasn't sure and didn't want to claim that it would. And I would have found it harder to make a list of aptitudes for all of EA without having noticeable omissions.

With all of that said, I hear you on why this felt unwelcoming, and regret that. I'll add a link to this comment to the main post to help clarify.

On your second point, I did try to acknowledge the possibility of for-profit startups from a learning/skill-building point of view (paragraph starting with "I do think that if you have any idea for an organization that you think could succeed ...") though I do agree this sort of entrepreneurship can be useful for making money and having impact in other ways (as noted by MichaelA, below), not just for learning, and should have been clearer about that.

My current impressions on career choice for longtermists

I think a year of full-time work is likely enough to see the sort of "signs of life" I alluded to, but it could take much longer to fulfill one's potential. I'd generally expect a lot of people in this category to see steady progress over time on things like (a) how open-ended and poorly-scoped of a question they can tackle, which in turn affects how important a question they can tackle; (b) how efficiently and thoroughly they can reach a good answer; (c) how well they can communicate their insights; (d) whether they can hire and train other people to do conceptual and empirical research, and hence "scale" what they're doing (this won't apply to everyone).

There could also be an effect in the opposite direction, though - I expect some people in this category to have their best insights relatively early on, and to have more trouble innovating in a field as the field becomes better-developed.

Overall this track doesn't seem like the one most likely to offer a steady upward trajectory, though I think some people will experience that. (I'd guess that people focused on "answering questions" would probably have more of a steady upward trajectory than people focused on "asking new questions / having totally original ideas.")

My current impressions on career choice for longtermists

I didn't mean to express a view one way or the other on particular current giving opportunities; I was instead looking for something a bit more general and timeless to say on this point, since especially in longtermism, giving opportunities can sometimes look very appealing at one moment and much less so at another (partly due to room-for-more-funding considerations). I think it's useful for you to have noted these points, though.

Hi, I'm Holden Karnofsky. AMA about jobs at Open Philanthropy

This is still a common practice. The point of it isn't to evaluate employees by # of hours worked; the point is for their manager to have a good understanding of how time is being used, so they can make suggestions about what to go deeper on, what to skip, how to reprioritize tasks, etc.

Several employees simply opt out from this because they prefer not to do it. It's an optional practice for the benefit of employees rather than a required practice used for performance assessment.

Hi, I'm Holden Karnofsky. AMA about jobs at Open Philanthropy

I'm referring to the possibility of supporting academics (e.g. philosophers) to propose and explore different approaches to moral uncertainty and their merits and drawbacks. (E.g., different approaches to operationalizing the considerations listed at https://www.openphilanthropy.org/blog/update-cause-prioritization-open-philanthropy#Allocating_capital_to_buckets_and_causes , which may have different consequences for how much ought to be allocated to each bucket)

Hi, I'm Holden Karnofsky. AMA about jobs at Open Philanthropy

Keep in mind that Milan worked for GiveWell, not OP, and that he was giving his own impressions rather than speaking for either organization in that post.

That said:

*His "Flexible working schedule" point sounds pretty consistent with how things are here.

*We continue to encourage time tracking (but we don't require it and not everybody does it).

*We do try to explicitly encourage self-care.

Does that respond to what you had in mind?

Hi, I'm Holden Karnofsky. AMA about jobs at Open Philanthropy

GiveWell's CEA was produced by multiple people over multiple years - we wouldn't expect a single person to generate the whole thing :)

I do think you should probably be able to imagine yourself engaging in a discussion over some particular parameter or aspect of GiveWell's CEA, and trying to improve that parameter or aspect to better capture what we care about (good accomplished per dollar). Quantitative aptitude is not a hard requirement for this position (there are some ways the role could evolve that would not require it), but it's a major plus.

Load More