Hi EA Forum,

I'm Holden Karnofsky I'm here to answer any questions about jobs at Open Philanthropy. I'll be here today from 9:30am to 12:30pm Pacific time  (with some breaks) and will likely respond to comments later on as well. 

We'd hate to miss out on strong applicants because of misconceptions about the roles, so I hope people will ask whatever is on their mind, on topics from office environment to day-to-day work to the likely long-term trajectory of the role. I think Open Philanthropy jobs are among the best possible ways for effective altruists to have impact, and I hope anyone who could imagine performing well in these jobs will at least consider applying!

Please post different questions as separate comments, for discussion threading.

Looking forward to it!

Added 12:32pm Pacific time: This concludes the "official" portion of the AMA, but feel free to post more questions; we may respond to them later on!

New Comment
113 comments, sorted by Click to highlight new comments since: Today at 5:57 PM
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

What are things that previous Research Analysts have struggled with at Open Phil? What are reasons others have found it not to be a good fit?

7Holden Karnofsky5y
A few things that come to mind: 1. The work is challenging, and not everyone is able to perform at a high enough level to see the career progression they want. 2. The culture tends toward direct communication. People are expected to be open with criticism, both of people they manage and of people who manage them. This can be uncomfortable for some people (though we try hard to create a supportive and constructive context). 3. The work is often solitary, consisting of reading/writing/analysis and one-on-one checkins rather than large-group collaboration. It's possible that this will change for some roles in the future (e.g. it's possible that we'll want more large-group collaboration as our cause prioritization team grows), but we're not sure of that.

This is still a common practice. The point of it isn't to evaluate employees by # of hours worked; the point is for their manager to have a good understanding of how time is being used, so they can make suggestions about what to go deeper on, what to skip, how to reprioritize tasks, etc.

Several employees simply opt out from this because they prefer not to do it. It's an optional practice for the benefit of employees rather than a required practice used for performance assessment.

I’m pretty confused about the work of the RA role - it seems to include everything from epidemiological literature reviews to philosophical work on population ethics to following up on individual organisations you’ve funded.

Could you give some concrete info about how you and the RA determine what the RA works on?

7Holden Karnofsky5y
The role does include all three of those things, and I think all three things are well served by the job qualifications listed in the posting. A common thread is that all involve trying to deliver an informative, well-calibrated answer to an action-relevant question, largely via discussion with knowledgeable parties and critical assessment of evidence and arguments. In general, we have a list of the projects that we consider most important to complete, and we look for good matches between high-ranked projects and employees who seem well suited to them. I expect that most entry-level Research Analysts will try their hand at both cause prioritization and grant investigation work, and we'll develop a picture of what they're best at that we can then use to assign them more of one or the other (or something else, such as the work listed at https://www.openphilanthropy.org/get-involved/jobs/analyst-specializing-potential-risks-advanced-artificial-intelligence [https://www.openphilanthropy.org/get-involved/jobs/analyst-specializing-potential-risks-advanced-artificial-intelligence] ) over time.
1Ben Pace5y
Thanks Holden!

If someone can't apply right now due to other commitments, do you expect there to be new roles for generalist research analysts next year as well? What are the best ways one could make oneself a better candidate meanwhile?

4Holden Karnofsky5y
There will probably be similar roles in the future, though I can't guarantee that. To become a better candidate, one can accomplish objectively impressive things (especially if they're relevant to effective altruism); create public content that gives a sense for how they think (e.g., a blog); or get to know people in the effective altruism community to increase the odds that one gets a positive & meaningful referral.

I'm graduating from the University of Michigan (Ann Arbor) with an Honors History degree this year. This gave me a nonstandard amount of experience with literature reviews and research for an undergraduate student. However, I've seen that OpenPhil generally prefers its candidates to have non-humanities majors.

1) Is the latter claim true?

2) In general, how does OpenPhil rate research experience that is not in the field(s) currently being explored?

Thanks for the question - I have wondered the same, as I also studied History at undergraduate level. A slight detour from your question, but maybe of interest. There is currently is no community / FB group for people with backgrounds or research interests in History within EA that I know of. There have been quite a few times when discussions around the usefulness of historical studies has come up and it might be good to share ideas and collaborate. I don't have time to try and coordinate this at the moment, but it seems like trying to establish some sort of discussion forum (or organisation?) for using the study of history to advance our understanding of (and strategy towards) cause areas which are often prioritised within EA. If this is something you (or anyone else seeing this) has an interest in me developing, it's something to bear in mind? People should feel free to contact me at james_a_harris [at] hotmail.co.uk if you want to talk about it further. Examples: I'm thinking primarily within Effective Animal Advocacy (Sentience Institute's study of the British antislavery movement; ACE discontinuing their social studies project; technology adoption being considered as a precedent for clean meat e.g. by Sentience Institute and Paul Shapiro) but this would also apply to other fields. The systematic approach described in the post linked at [1] seems to correlate more closely with the approach Holden and others took at OPP than it does the studies done in the Effective Animal Advocacy sphere. [1] http://effective-altruism.com/ea/1lz/why_we_should_be_doing_more_systematic_research/ [http://effective-altruism.com/ea/1lz/why_we_should_be_doing_more_systematic_research/]
just emailed you.
Michael, apologies for this. I just came back to check this post. I didn't ever receive the email because the formatting of the EA forum removed the underscore from my email, and I didn't notice at the time. If you can find the email that you sent from your sent box in April, and could forward it to me at jamie@sentienceinstitute.org [jamie@sentienceinstitute.org] that would be great!
OpenPhil is big on history, I would apply.
(I work for Open Phil.) As Holden said elsewhere, "Quantitative aptitude is not a hard requirement for this position (there are some ways the role could evolve that would not require it), but it's a major plus." In that sense we prefer candidates with quantitative/technical majors, but also note that in most cases we pay more attention to an applicant's performance on the work tests that are part of our application process than we do to their CV. Research experience in a current Open Phil focus area is a bonus, but less important than general quantitative aptitude and performance on our work tests.

A quick Google search gave me the impression that it isn't very easy to get a work visa in the US even though it will be sponsored. Is this correct, and do you have stronger requirements for non-US applicants because they'll be less likely to actually be able to work for you? (I'm completely unfamiliar with the visa system)

2Holden Karnofsky5y
We don't control the visa process and can't ensure that people will get sponsorship. We don't expect sponsorship requirements to be a major factor for us in deciding which applicants to move forward with.
Thanks for the response. I understand OPP doesn't control the visa process, but do you have a rough sense of how likely a successful applicant would be to get a visa after being sponsored, or is it a complete unknown?
Unfortunately the likelihood is still pretty unclear to us at this point, and the available options vary a fair bit by applicant, depending on which country they're from, whether they recently graduated from undergrad or graduate school, and other factors.

Here's a question I received via email, which I'll answer here so others can benefit from the answer.


Would you be able to give examples of how the research has been used for decision making around giving money/grants?


Sure, here are a few examples:

  • Our research on the history of philanthropy helped us decide to be more ambitious: "we are more interested in working on daunting problems over long periods of time after learning about some of philanthropy’s past contributions." This has increased the amount of attention and funding we'v
... (read more)

Keep in mind that Milan worked for GiveWell, not OP, and that he was giving his own impressions rather than speaking for either organization in that post.

That said:

*His "Flexible working schedule" point sounds pretty consistent with how things are here.

*We continue to encourage time tracking (but we don't require it and not everybody does it).

*We do try to explicitly encourage self-care.

Does that respond to what you had in mind?

Thanks Holden and Luke for answering so many questions <3

Thinking in terms of broad generalisations/approximations - if you had to draw a graph depicting the value provided to OP by a new research analyst over time, what kind of shape would this graph have?

Or to ask the question in a different way: Are your efforts to hire for new OP roles motivated more by a desire to make better grants in the next couple of years, or by an intention to have a strong team in place several years from now which does high quality work in the future?

Responding to your second formulation of the question, the answer is "more the latter than the former." We intend to invest heavily in training and mentoring new hires, and we hope that research analysts will end up being long-term core contributors at Open Phil — as research analysts, as grant investigators, and as high-level managers, among other roles — or, in some cases, in important roles that require similar skills outside Open Phil.

Thanks, Holden, for the AMA and everyone for their questions!

When looking at the application form initially, there was a mentioning of an applied task as part of the application -- as far as I can tell this has now been removed.

Could someone confirm this change? Does that mean that the application now only consists of the Google Form plus resume in the first stage?

Thanks in advance!

Yes, we recently removed a 2-question timed test from the job ad page. We wanted to lower the cost of submitting an initial application, and also we want to filter some applications before we request that applicants take the timed test. However, we are still using this timed test in an early part of our application process, and we still think it is diagnostic of who will be a good fit for our research analyst positions. (I work for Open Phil.)
Thanks, Luke!

Who is likely to manage new Grants Associates, Operations Associates, and Research Analysts?

2Holden Karnofsky5y
All bios here: https://www.openphilanthropy.org/about/team [https://www.openphilanthropy.org/about/team] Grants Associates and Operations Associates are likely to report to Derek or Morgan. Research Analysts are likely to report to people who have been in similar roles for a while, such as Ajeya, Claire, Luke and Nick. None of this is set in stone though.

What are the working hours like for a position like Research Analyst? Strict/flexible? 40 hours/week or other? What is the overtime like on average, and what is it like on peaks?

8Holden Karnofsky5y
We're flexible. People don't clock in or out; we evaluate performance based on how much people get done on a timescale of months. We encourage people to work hard but also prioritize work-life balance. The right balance varies by the individual. Most people here work more than one would in a traditional 9-5 job. (A common figure is 35-40 "focused" hours per week.) I think that reflects that they're passionate about their work rather than that they feel pressure from management to work a lot. We regularly check in with people about work-life balance and encourage them to work less if it seems this would be good for their happiness.

It seems that OpenPhil wants a more satisfactory answer to moral uncertainty than just worldview diversification before ramping up the amount of grants per year. Is this part of why you are hiring new Research Analysts, and if so, how much will they work on this problem? (This seems like a very interesting but hard problem)

5Holden Karnofsky5y
We could certainly imagine ramping up grantmaking without a much better answer. As an institution we're often happy to go with a "hacky" approach that is suboptimal, but captures most of the value available under multiple different assumptions. If someone at Open Phil has an idea for how to make useful progress on this kind of question in a reasonable amount of time, we'll very likely find that worthwhile and go forward. But there are lots of other things for Research Analysts to work on even if we don't put much more time into researching or reflecting on moral uncertainty. Also note that we may pursue an improved understanding via grantmaking rather than via researching the question ourselves.
I'm very curious about how that improved understanding would come about via grantmaking. Any write-up you have about this? I can see how you'd learn about tractability, and maybe about neglectedness, but I wonder how you incorporate this in your decision-making. Anyway, this might go a little too off-topic so I'd understand if you replied to other questions first :)
1Holden Karnofsky5y
I'm referring to the possibility of supporting academics (e.g. philosophers) to propose and explore different approaches to moral uncertainty and their merits and drawbacks. (E.g., different approaches to operationalizing the considerations listed at https://www.openphilanthropy.org/blog/update-cause-prioritization-open-philanthropy#Allocating_capital_to_buckets_and_causes [https://www.openphilanthropy.org/blog/update-cause-prioritization-open-philanthropy#Allocating_capital_to_buckets_and_causes] , which may have different consequences for how much ought to be allocated to each bucket)

The job description for Research Analyst says that the best candidates will have "comfort thinking in terms of expected value and using systematic, quantitative frameworks." How quantitative should a candidate be to apply? For example, if a person feels comfortable with basic expected value concepts but finds GiveWell's CEA overwhelming (and probably could never produce something similar to GiveWell's CEA), is that not quantitative enough?

5Holden Karnofsky5y
GiveWell's CEA was produced by multiple people over multiple years - we wouldn't expect a single person to generate the whole thing :) I do think you should probably be able to imagine yourself engaging in a discussion over some particular parameter or aspect of GiveWell's CEA, and trying to improve that parameter or aspect to better capture what we care about (good accomplished per dollar). Quantitative aptitude is not a hard requirement for this position (there are some ways the role could evolve that would not require it), but it's a major plus.
3Chris Leong5y
What's Givewell's CEA?
Cost-Effectiveness Analysis :)

Is there any possibility for the research analyst (or some other) role to be taken up completely remotely? (i.e. while living in another country)

Jumping in here for Holden (I'm an Open Phil RA)… We would like our Research Analysts to work in the SF office most of the time, with 'partly remote' in the job ad meaning that they can occasionally work from home/different locations. Despite our policy, we aren't totally closed to the idea of hiring someone who mostly works remotely, but they'd have to meet a higher bar.
Is it possible to switch to remote work (in my case in Europe) after working from office in SF e.g. for one year?
Possibly. If a research analyst excels at the job after being hired, and seems likely to continue excel even while remote, we'd certainly consider it.

What key metrics do research analysts pay attention to in the course of their work? More broadly, how do employees know that they're doing a good job?

2Holden Karnofsky5y
We do formal performance reviews twice per year, and we ask managers to use their regular (~weekly) checkins with reports to sync up on performance such that nothing in these reviews should be surprising. There's no unified metric for an employee's output here; we set priorities for the organization, set assignments that serve these priorities, set case-by-case timelines and goals for the assignments (in collaboration with the people who will be working on them), and compare output to the goals we had set.

I'm under the impression that holidays, sick days, and parental leave are on average much less generous in the US compared to Europe or Canada. What are your policies on time off work?

3Holden Karnofsky5y
We're in the process of reviewing our policies, but we're likely to settle on something like 25 paid days off (including sick days), 10 holiday days (with the option to work on holidays and use the paid time off elsewhere), several months of paid parental leave, and a flexible unpaid leave policy for people who want to take more time off. We are also flexible with respect to working from home.
To clarify, are the 10 holiday days part of the 25 paid days off, or in addition to?
Or by 'holiday days' do you mean statutory holidays like Thanksgiving?
1Holden Karnofsky5y
Yes, I mean statutory holidays like Thanksgiving.
It is still unclear to me whether the statutory holidays are supposed to be included in the 25 days paid days off or in addition to.
In addition to, 35 days total. (I work at Open Phil.)

In my current field – biomedical research – there’s a tendency for the best researchers to be drawn towards important problem areas, but to then detour towards interesting problems instead of important ones when choosing a specific research topic. I think this is especially true for generalists, who often have a wider perspective on why the Really Cool problem is Really Cool. I’m guessing similar things happen in most fields. Does the Open Philanthropy Project try to combat this? As a Research Analyst, will the aim always be Important over Interesting?

Thanks! :)

3Holden Karnofsky5y
We're certainly not using the same standards as academia! In general, we aim to base assignments on a combination of 1. How we judge what's most important to do (in terms of accomplishing as much good as possible) 2. What the employees themselves are motivated and interested to work on (including their own judgments of how to do as much good as possible).

How is doing research at Open Philanthropy different from doing research in academia? To make things concrete, how would the work of someone doing economic modelling or ML research in academia differ from typical research at OpenPhil?

(I work for Open Phil.) Comparing to academia, I'd say that research at Open Phil is (1) consistently targeted at what will help us do as much good as possible rather than what is most intellectually interesting or prestigious, (2) aimed at informing our actions as cheaply as possible, meaning that we cut corners when we don't think doing so will change our bottom-line conclusions, rather than trying to live up to the standards for thoroughness etc. expected in academia, and (3) only aimed at what academia would consider "novel research" when that's what is required to help us do the most good.
Wow. Working at Open Phil sounds like a dream compared to academia. You've identified three things I spend huge amounts of time doing as part of my research and find intensely irritatingly.

How much collaboration exists between research analysts (or operations associates, for that matter)?

I decided against working in academic research because I do much better in a team environment (short feedback loops, bouncing ideas off peers, sense that my work contributes to shared purpose and projects) than I do working independently. I prefer the industry side of basically all of Philip Guo's industry vs. academia comparisons. Would it still make sense for me to apply for an OpenPhil job? I think I have relevant skills, but I'm worried that I wouldn't be effective in a research environment, even if it is non-academic.

3Holden Karnofsky5y
Most of the roles here involve a lot of independent work, consisting of reading/writing/analysis and one-on-one checkins rather than large-group collaboration. It’s possible that this will change for some roles in the future (e.g. it’s possible that we’ll want more large-group collaboration as our cause prioritization team grows), but we’re not sure of that. I think you should probably be prepared for a fair amount of work along the lines of what I've described here.

Hi Holden,

the job description for Research Analysts says:

Research Analysts will receive intensive training and mentorship, and over time will become highly experienced with our approach to reasoning transparency, cost-effectiveness analysis, critical evidence assessment, grant investigation, and balancing thoroughness with efficiency."

Could you please elaborate a bit more on the training and mentorship part? E.g.: How much time is reserved for training? Who would be the mentors? What would the relation with the mentor(s) be like?

Thanks for offering this Q&A!

3Holden Karnofsky5y
I answered a similar question here: http://effective-altruism.com/ea/1mf/hi_im_holden_karnofsky_ama_about_jobs_at_open/dpl [http://effective-altruism.com/ea/1mf/hi_im_holden_karnofsky_ama_about_jobs_at_open/dpl] In general, people who have been in the Research Analyst role for a while will be the managers and primary mentors of new Research Analysts. There will be regular (~weekly) scheduled checkins as well as informal interaction as needed (e.g., over Slack). There's no hard line between training and "just doing the work" - every assignment should have some direct value and some training value. We expect to lean pretty hard toward the training end of the spectrum for people's first few months, then gradually move along the spectrum to where assignments are more optimized for direct value.

Hey Holden, thanks for doing this. Suppose I applied for the research analyst position and didn't get it. Which of the following would then be more likely to eventually land me a job at OPP, and how much more likely (assuming I would perform well in both)?

a) becoming research analyst at GiveWell

b) doing research in one of OPP's focus areas (biosecurity/AI safety).

2Holden Karnofsky5y
I would rate those about equally, though I'd add that GiveWell would prefer not to hire people whose main goal is to go to OP.

What's daily work and office culture look like in the OPP? Do you all host group activities, happy hours, or anything like that outside of working hours?

4Holden Karnofsky5y
We currently have a happy hour every 3 weeks and host group activities as well, including occasional parties and a multiple-day staff retreat this year. We want to make it easy for staff to socialize and be friends, without making it a requirement or an overly hard nudge (if people would rather stick to their work, that's fine by us).

Somebody asked the following question:

I understand that OP’s preference is to have promising candidates attend an in-person work trial rather than do an additional remote work test; would this preference still stand if the candidate in question has to obtain a US work visa sponsorship in order to attend the in-person trial?

Our reply is: Yes, that preference stands regardless of current work authorization status, though of course in some cases there won't be any way for us to help an applicant get US work authorization, depending on their situation.

Thanks for the answer, Luke. I removed the question from here after I reread the "Update on Open Philanthropy research analyst recruiting for 2018" file sent earlier and found the answer there!

1a. Has Open Phil set any aggressive org-wide goals or timelines for 2018?
1b. The published plan for 2018 says that OPP expects to give "well over $100 million" [1]. What is this expectation based on? Or is it a goal?

2a. Other than current funders, who considers the research coming from Open Phil RAs to be reputable, credible and useful? (ie. government?)
2b. Does it matter that RAs aren't PhDs or that Open Phil isn't directly affiliated with any educational institution?

[1] https://www.openphilanthropy.org/blog/our-progress-2017-and-plans-2018

Quick replies to each: 1a. Our goals for 2018 are laid out in the post you linked to. 1b. The expectation is based mostly on the fact that we gave well over $100 million last year, and we're devoting similar time and effort to grantmaking in 2018. 2a. Open Phil is still a fairly new organization, and I don't think many know much about us yet. Probably we are best known in the effective altruism community, where we seem to have a strong reputation. 2b. Does it matter for our reputation, do you mean? I'm not sure. I'm not aware of us having received critiques about that.
The purpose of these questions was to better estimate if an RAs impact can be expected to increase, decrease, or remain the same in the coming years. An aggressive measurable goal (ie. increase estimated QALYs gained by a factor of x) would indicate to me that an RAs expected impact would increase. (It's possible that a measurable goal might be trivial to set because the error bars might be too large. I don't know enough to know.) If other funders (esp. big funders such as government) considered Open Phil research credible enough to base their decisions on, that would also indicate more expected impact. ie. already published research would be reused in the future by other large donors to effectively allocate more funds. Either way, it seems that an RAs expected impact is higher than many other career alternatives, even if it decreases a bit in the next few years.
  • If someone was looking to work for OPP would an honours* or masters program be more beneficial than an undergraduate degree?

  • Are there particular questions or areas that could be worked on for a research project in honours/masters that are particularly helpful directly or develop the right kinds of skills for OPP? (especially in economics, philosophy or cognitive science)

  • ("Honours" in Australia is a 1 year research/coursework program)

Completion of an honours or masters program provides us with a bit more evidence about an applicant's capabilities than an undergraduate degree does, but both are less informative to us than the applicant's performance on the various work samples that are part of our application process. Because our roles are so "generalist," there are few domains that are especially relevant, though microeconomics and statistics are two unusually broadly relevant fields. In general, we find that those with a STEM background do especially well at the kind of work we do, but a STEM background is not required. A couple other things that are likely helpful for getting and excelling in an Open Phil research analyst role are calibration training [https://www.openphilanthropy.org/blog/efforts-improve-accuracy-our-judgments-and-forecasts#Calibration_training] and practice making Fermi estimates [https://www.lesswrong.com/posts/PsEppdvgRisz5xAHG/fermi-estimates].

What are the stages of the application process? What proportion of applicants do you expect to filter out at each stage?

Because applicants are working through our application process at different speeds, we're still learning what portion of applicants are invited forward at each stage. We are also adjusting the process as we go along. As of today, our application process looks like this: 1. Initial submission of application + résumé. 2. 2-question timed test. 3. A "conversation notes" work test. (compensated via honorarium) 4. A brief call, to explain the rest of our process and answer the applicant's questions. 5. An "internal grant write-up" work test. (compensated via honorarium) To some degree we are still determining next steps after #5, and they depend somewhat on the applicant's availability and preferences, and that's the main thing we discuss with applicants at step #4, on an individual basis.

What advice would you give to a potential OpenPhil hire re how to think about the value of joining OpenPhil full time vs starting an organization that OpenPhil would be likely to fund vs joining an organization that has already received funding from OpenPhil?

(I work for Open Phil.) This is tough to answer in general; see my reply to a similar question here [http://effective-altruism.com/ea/1mf/hi_im_holden_karnofsky_ama_about_jobs_at_open/drr] . Probably best to discuss this on an individual basis if/when an applicant reaches the interview stage of our application process.

Is there a standard contract length you will offer RA's if there is no trial period?

(I work for Open Phil.) For salaried positions there is no "contract length"; employment at Open Philanthropy is at-will [https://en.wikipedia.org/wiki/At-will_employment], just as it is with most California employers.

I'm curious whether Open Phil has done any estimates of the relative impact of someone who goes to work for you vs someone who does direct work in one of your focus areas? All else equal (i.e., assume personal fit is the same, etc.), which option might be better?

(I work for Open Phil.) This is tough to answer in general. It would depend hugely on the focus area, their role in that focus area, their fit for Open Phil research analyst work, etc. I can easily think of people who I think are doing more good by working in a focus area, and I can easily think of other people who would be valuable in a focus area but even more valuable as a generalist research analyst at Open Phil.

What are the most important ways in which Open Phil's culture is different from GiveWell's culture?

The core values and norms are very similar, so specific differences are relatively hard to articulate/pinpoint (which isn't to say they aren't there).
Are you saying that Open Phil & GiveWell cultures are different in hard-to-articulate but important ways? Or that the cultures are basically the same in any way that matters?
It's hard to say. To me the cultures seem very similar, but my personal access to GiveWell culture is limited since I work for Open Phil. Perhaps to other people the cultures seems importantly different, but I can't predict what those perceived differences would be.


What would be the attitude towards someone who wanted to work with you after undergrad for a year or two, but then go on to graduate school (likely for philosophy in my case), with an eye towards then continuing to work with you or other EA orgs after grad school?

(I work for Open Phil.) We'd encourage someone like that to apply, and flag in their application that they currently would plan to leave after a couple years to go to graduate school. Depending on their fit for the work, we might be excited to consider their application anyway, or we might not.

Do you recommend any textbooks (or other resources) that might help an applicant become more familiar with the expected value frameworks used at OpenPhil?

(I work for Open Phil.) Spending time with GiveWell's CEA [https://www.givewell.org/how-we-work/our-criteria/cost-effectiveness/cost-effectiveness-models] is probably more helpful than reading a textbook. You could also just practice doing Fermi estimates [http://lesswrong.com/lw/h5e/fermi_estimates/] in general.

Hi Holden, nice initiative.

I have a question about the Research Analyst role. How generalist will they be? I can imagine them being somewhat focused on one or two focus areas besides more general issues such as how to implement moral uncertainty practically.

1Holden Karnofsky5y
This varies by the individual. We have some Research Analysts who are always working on a variety of things, and some who have become quite specialized. It varies largely by the interests/preferences of the employee.

I really appreciate that your application form is so short, but what's the logic behind focusing on applicant's undergraduate degrees? What are you assessing in the application?

3Holden Karnofsky5y
All else equal, we consider applicants stronger when they have degrees in challenging fields from strong institutions. It’s not the only thing we’re looking at, even at that early stage. And the early stage is for filtering; ultimately, things like work trial assignments will be far more important to hiring decisions.
What does a "challenging field" mean here? Do you have a ranking of fields by degree of challenge or rely mainly on intuition?
Not sure if interpreting Khorton correctly, but interested anyway: Why focus on undergrad and not on postgrad (or highest level achieved/pursuing)?
6Holden Karnofsky5y
Hm, I'm not sure why our form asks for more detail on undergrad relative to grad - we copied the form from GiveWell and may not have thought about it. It's possible this is because the form was being used in an earlier GiveWell search where few applicants had been to grad schools. I'll ask around about this.

How do you enjoy living in/near and working San Francisco? How's the commute, the expensive housing, and all that affected your lives?

I am a Program Officer at Open Philanthropy who joined as a Research Analyst about 3 years ago. The prior two places I lived were New Brunswick, NJ and Oxford, UK. I live in a house with a few friends. It is 25-30m commute door-to-door via BART. My rent and monthly expenses are comparable to what I had in Oxford but noticeably larger than what I had in New Brunswick. I got pay increases when I moved to Open Phil, and additional raises over time. I’m comfortable on my current salary and could afford to get a single-bedroom apartment if I wanted, but I’m happy where I am. Overall, I would say that it was an easy adjustment.
Surely rent is much higher than Oxford on average? It's possible to get a great place in Oxford for under £700 per month, while comparable in SF would be $1300+. Food also seems about 30% more expensive, and in Oxford you don't have to pay for a commute. My overall guess is that $80k p.a. in SF is equivalent to about £40k p.a. in Oxford.
Cost-of-living comparison between San Francisco, CA and Oxford, UK: https://www.numbeo.com/cost-of-living/compare_cities.jsp?country1=United+Kingdom&country2=United+States&city1=Oxford&city2=San+Francisco%2C+CA&tracking=getDispatchComparison [https://www.numbeo.com/cost-of-living/compare_cities.jsp?country1=United+Kingdom&country2=United+States&city1=Oxford&city2=San+Francisco%2C+CA&tracking=getDispatchComparison]
I don't mean to make a claim re: averages, just relaying personal experience.
To chime in as someone who has very recently spent a lot of time in both London and SF, a 1.8:1 ratio (as in $1.8y is about the same as £y) is very roughly what I would have said for living costs between that pair, though living circumstances will vary significantly. Pound to dollar exchange rates have moved a ton in the last few years, whereas I don't think local salaries or costs of living have moved nearly as much, so I expect that 1.8:1 heuristic to be more stable/useful than trying to do the same comparison including a currency conversion (depending what point in the last few years you picked/moved, that ratio would imply anywhere between a 1.05x increase and a 1.55x increase).
2Holden Karnofsky5y
Perhaps other staff will chime in here, but my take: our pay is competitive and takes cost of living into account, and we are near public transportation, so I don't think the rents or commutes are a major issue. As a former NYC resident, I think the Bay Area is a great place to live (weather, food, etc.) and has a very strong effective altruist community. I don't see a lot of drawbacks to living here if you can make it work.

Is there any room in the application process for applicants to submit samples of original research or academic letters of recommendation?

Thank you!

Yes, you may submit a writing sample by sending it to jobs@openphilanthropy.org [jobs@openphilanthropy.org], as FirstName.LastName.Sample (e.g. John.Smith.Sample.doc or John.Smith.Sample.pdf). If you'd like to submit a letter of recommendation, please include it as a page of your résumé. Please keep in mind that writing samples and letters of recommendation are entirely optional, so if you don't already have them handy, I don't recommend spending time pulling them together. Our application process puts much more weight on work test performance anyway.

How does Open Philanthropy weigh conformity against talent?

What kind of conformity are you asking about? Certainly, some degree of alignment with our mission and values is important to us, and so is talent and "fit" for the work. Our team members are encouraged to focus on optimizing for Open Phil's mission, even when it means pushing back on their manager.
Thanks for the response, yes I was wondering about conformity in the sense of prevailing thinking within a particular cause area. Is there an expectation for talent to conform to prevailing thinking to a certain degree and would this then reinforce that idea of being talented, or could talent be more related to a set of core values or principles? I think some cause areas seem to have fairly high expectations of conformity toward in-group / out-group identity, so if this is the case then talented people may conform or not (given the assumption that not all talented people would necessarily be in-group thinkers), but it seems to confer various advantages on those that do.

GiveWell is also hiring for several roles right now. How should a person decide whether to apply to GiveWell or Open Phil? Are there significant differences in the work culture? Do Research Analysts at each org take on similar types of tasks?

1Holden Karnofsky5y
They're different organizations and I don't know nearly as much about the GiveWell role. One big difference is the causes we work on. If you're interested in both, I'd recommend applying to both, and if you are offered both roles, there will be lots of opportunities to learn more about each at that point in order to inform the decision.

Hi there! Thank you for doing this. Is there a place for people who have just or recently finished their undergrad to help in your AI governance and policy position?

2Holden Karnofsky5y
I'd recommend that recent grads looking to help with AI governance and policy apply for the Research Analyst [https://www.openphilanthropy.org/get-involved/jobs/research-analyst-2018] position. With Research Analysts, we'll first focus on mentorship & training, then try to figure out where everyone can do the most good based on their interests and skills. Someone who has a high aptitude & interest for AI strategy would likely end up putting substantial time into that within a year or so (maybe less). You can also check out roles at the Future of Humanity Institute [https://www.fhi.ox.ac.uk/vacancies/].
Thank you for your reply Holden! I submitted my application yesterday after hearing you all were talent constrained, so I hope to hear from you all sometime soon. What does the training process look like for people who are hired on as research analysts?
1Holden Karnofsky5y
Broadly speaking, we're going to try to give people assignments that are relevant to our work and that we think include a lot of the core needed skills - things like evaluating a potential grant (or renewal) and writing up the case for or against. We'll evaluate these assignments, give substantial feedback, and iterate so that people improve. We'll also be providing resources for gaining background knowledge, such as "flex time," recommended reading lists and optional Q&A sessions. We've seen people improve a lot in the past and become core contributors, and think this basic approach is likely to lead to more of that.

Meta: It might be good to announce AMA's in advance so that more people know to be online at that time.

This one was announced here [http://effective-altruism.com/ea/1me/upcoming_ama_with_holden_karnofsky_on_job/] .