All of Tom Barnes's Comments + Replies

I thought it might be helpful for me to add my own thoughts, as a fund manger at EAIF (Note I'm speaking in a personal capacity, not on behalf of EA Funds or EV).

1. Firstly, I'd like to apologise for my role in these mistakes. I was the Primary Investigator (PI) for Igor's application, and thus I share some responsibility here. Specifically as the PI, I should have (a) evaluated the application sooner, (b) reached a final recommendation sooner, and (c) been more responsive to communications after making a decision
   1. I did not make an initial decision until Novem

My boring answer would be to see details on our website. In terms of submission style, we say:

• We recommend that applicants take about 1–2 hours to write their applications. This does not include the time spent developing the plan and strategy for the project – we recommend thinking about those carefully prior to applying.
• Please keep your answers brief and ensure the total length of your responses does not exceed 10,000 characters. We recommend a total length of 2,000–5,000 characters.
• We recommend focusing on the substantive arguments in favour of your proj

Currently we don't have a process for retroactively evaluating EAIF grants. However, there are a couple of informal channels which can help to improve decision-making:

• We request that grantees fill out a short form detailing the impact of their grant after six months. These reports are both directly helpful for evaluating a future application from the grantee, and indirectly helpful at calibrating the "bang-for-your-buck" we should expect from different grant sizes for different projects
• When evaluting the renewal of a grant, we can compare the initial appli

Hey - I think it's important to clarify that EAIF is optimising for something fairly different from GiveWell (although we share the same broad aim):

• Specifically, GiveWell is optimising for lives saved in the next few years, under the constraint of health projects in LMICs, with a high probability of impact and fairly immediate / verifable results. 
• Meanwhile, EAIF is focused on a hits-based, low-certainty area, where the evidence base is weaker, grants have longer paths to impact, and the overarching goal is often unclear.

As such, a direct/equivalent c... (read more)

I think the premise of your question is roughly correct: I do think it's pretty hard to "help EA notice what it is important to work on", for a bunch of reasons:

• It could lead to new, unexpected directions which might be counterintuive / controversial.
• it requires the community to have the psychological, financial and intellectual safety to identify / work on causes which may not be promising
• It needs a non-trivial number of people to engage with the result of exploration, and act upon it (including people who can direct substantial resources)
• It has a very lo

Good Question! We have discussed running RFP(s) to more directly support projects we'd like to see. First, though, I think we want to do some more strategic thinking about the direction we want EAIF to go in, and hence at this stage I think we are fairly unsure about which project types we'd like to see more of.

Caveats aside, I personally^[1] would be pretty interested in:

• Macrostrategy / cause prioritization research. I think a substantional amount of intellectual progress was made in the 2000s / early 2010s from a constellation of different places (e.

Hey, good question!

Here's a crude rationale: 

• Suppose that by donating $1k to an EAIF project, they get 1 new person to consider donating more effectively. 
• This 1 new person pledges to give 1% of their salary to GiveWell's top charities, and they do this for the next 10 years. 
• If they make (say) $50k / year, then over 10 years they will donate $5k to GiveWell charities. 
• The net result is that a $1k donation to EAIF led to $5k donated to top GiveWell charities - or a dollar donated to EAIF goes 5x further than a GiveWell Top charity

Of cou... (read more)

I agree there's no single unified resource. Having said that, I found Richard Ngo's "five alignment clusters" pretty helpful for bucketing different groups & arguments together. Reposting below:

1. MIRI cluster. Think that P(doom) is very high, based on intuitions about instrumental convergence, deceptive alignment, etc. Does work that's very different from mainstream ML. Central members: Eliezer Yudkowsky, Nate Soares.
2. Structural risk cluster. Think that doom is more likely than not, but not for the same reasons as the MIRI cluster. Instead, this cluster f

A couple of weeks ago I blocked all mentions of "Effective Altruism", "AI Safety", "OpenAI", etc from my twitter feed. Since then I've noticed it become much less of a time sink, and much better for mental health. Would strongly recommend!

I wrote the following on a draft of this post. For context, I currently do (very) part-time work at EAIF

Overall, I‘m pretty excited to see EAIF orient to a principles-first EA. Despite recent challenges, I continue to believe that the EA community is doing something special and important, and is fundamentally worth fighting for. With this reorientation of EAIF, I hope we can get the EA community back to a strong position. I share many of the uncertainties listed - about whether this is a viable project, how EAIF will practically evaluate grants under this worldview, or if it’s even philosophically coherent. Nonetheless, I’m excited to see what can be done.

Yeah that's fair. I wrote this somewhat off the cuff, but because it got more engagement than I thought I'd make it a full post if I wrote again

Is your claim "Impartial altruists with ~no credence on longtermism would have more impact donating to AI/GCRs over animals / global health"?

To my mind, this is the crux, because:

1. If Yes, then I agree that it totally makes sense for non-longtermist EAs to donate to AI/GCRs
2. If No, then I'm confused why one wouldn't donate to animals / global health instead?

[I use "donate" rather than "work on" because donations aren't sensitive to individual circumstances, e.g. personal fit. I'm also assuming impartiality because this seems core to EA to me, but of course one could donate / work on a topic for non-impartial/ non-EA reasons]

Mildly against the Longtermism --> GCR shift
Epistemic status: Pretty uncertain, somewhat rambly

TL;DR replacing longtermism with GCRs might get more resources to longtermist causes, but at the expense of non-GCR longtermist interventions and broader community epistemics

Over the last ~6 months I've noticed a general shift amongst EA orgs to focus less on reducing risks from AI, Bio, nukes, etc based on the logic of longtermism, and more based on Global Catastrophic Risks (GCRs) directly. Some data points on this:

• Open Phil renaming it's EA Community Gro

I haven't yet decided, but it's likely that a majority of my donations will go to this year's donor lottery. I'm fairly convinced by the arguments in favour of donor lotteries [1, 2], and would encourage others to consider them if they're unsure where to give. 

Having said that, lotteries have less fuzzies than donating directly, so I may separately give to some effective charities which I'm personally excited about.

Thanks - I just saw RP put out this post, which makes much the same point. Good to be cautious about interpreting these results!

YouGov Poll on SBF and EA

I recently came across this article from YouGov (published last week), summarizing a survey of US citizens for their opinions on Sam Bankman-Fried, Cryptocurrency and Effective Altruism.

I half-expected the survey responses to be pretty negative about EA, given press coverage and potential priming effects associating SBF to EA. So I was positively surprised that:

(it's worth noting that there were only ~1000 participants, and the survey was online only)

(FYI to others - I've just seen Ajeya's very helpful writeup, which has already partially answer this question!)

What's the reason for the change from Longtermism to GCRs? How has/will this change strategy going forward?

It seems that OP's AI safety & gov teams have both been historically capacity-constrained. Why the decision to hire for these roles now (rather than earlier)?

Ten Project Ideas for AI X-Risk Prioritization

I made a list of 10 ideas I'd be excited for someone to tackle, within the broad problem of "how to prioritize resources within AI X-risk?" I won’t claim these projects are more / less valuable than other things people could be doing. However, I'd be excited if someone gave a stab at them

10 Ideas:

1. Threat Model Prioritization
2. Country Prioritization
3. An Inside-view timelines model
4. Modelling AI Lab deployment decisions
5. How are resources currently allocated between risks / interventions / countries
6. How to allocate

Thanks Ajeya, this is very helpful and clarifying!

I am the only person who is primarily focused on funding technical research projects ... I began making grants in November 2022

Does this mean that prior to November 2022 there were ~no full-time technical AI safety grantmakers at Open Philanthropy? 

OP (prev. GiveWell labs) has been evaluating grants in the AI safety space for over 10 years. In that time the AI safety field and Open Philanthropy have both grown, with OP granting over $300m on AI risk. Open Phil has also done a lot of research on the pro... (read more)

This looks very exciting, thanks for posting!

I'll quickly mention a couple of things that stuck out to me that might make the CEA potentially overoptimistic:

1. IQ points lost per μg/dl of lead - this is likely a logarithmic relationship (as suggested by Bernard and Schukraft). For a BLL of 2.4 - 10 μg/dl, IQ loss from 1 μg/dl increase may be close to 0.5, but above 10, it's closer to 0.2 per 1 μg/dl increase, and above 20, closer to 0.1. Average BLL in Bangladesh seem to be around 6.8 μg/dl, though amongst residents living near turmeric sources of lead, it co

My bad, thanks so much!

It would be great to have some way to filter for multiple topics.

Example: Suppose I want to find posts related to the cost-effectiveness of AI safety. Instead of just filtering for "AI safety", or for just "Forecasting and estimation", I might want to find posts only at the intersection of those two. I attempted to do this by customizing my frontpage feed, but this doesn't really work (since it heavily biases to new/upvoted posts)

it relies primarily on heuristics like organiser track record and higher-level reasoning about plans. 

I think this is mostly correct, with the caveat that we don't exclusively rely on qualitative factors and subjective judgement alone. The way I'd frame it is more as a spectrum between

[Heuristics] <------> [GiveWell-style cost-effectiveness modelling]

I think I'd place FP's longtermist evaluation methodology somewhere between those two poles, with flexibility based on what's feasible in each cause

I'll +1 everything Johannes has already said, and add that several people (including myself) have been chewing over the "how to rate longtermist projects" question for quite some time. I'm unsure when we will post something publicly, but I hope it won't be too far in the future.

If anyone is curious for details feel free to reach out!

Quick take: renaming shortforms to Quick takes is a mistake

This looks super interesting, thanks for posting! I especially appreciate the "How to apply" section

One thing I'm interested in is seeing how this actually looks in practice - specifying real exogenous uncertainties (e.g. about timelines, takeoff speeds, etc), policy levers (e.g. these ideas, different AI safety research agendas, etc), relations (e.g. between AI labs, governments, etc) and performance metrics (e.g "p(doom)", plus many of the sub-goals you outline). What are the conclusions? What would this imply about prioritization decisions? etc

I appreci... (read more)

Should recent ai progress change the plans of people working on global health who are focused on economic outcomes?

I think so, see here or here for a bit more discussion on this

If you think that AI will go pretty well by default (which I think many neartermists do)

My guess/impression is that this just hasn't been discussed by neartermists very much (which I think is one sad side-effect from bucketing all AI stuff in a "longtermist" worldview)

I think this could be an interesting avenue to explore. One very basic way to (very roughly) do this is to model p(doom) effectively as a discount rate. This could be an additional user input on GiveWell's spreadsheets.

So for example, if your p(doom) is 20% in 20 years, then you could increase the discount rate by roughly 1% per year

[Techinically this will be somewhat off since (I'm guessing) most people's p(doom) doesn't increase at a constant rate, in the way a fixed discount rate does.]

Rob Besinger of MIRI tweets:

...I'm happy to say that MIRI leadership thinks "humanity never builds AGI" would be the worst catastrophe in history, would cost nearly all of the future's value, and is basically just unacceptably bad as an option.

Just to add that the Research Institute for Future Design (RIFD) is a Founders Pledge recommendation for longtermist institutional reform

(disclaimer: I am a researcher at Founders Pledge)

OpenPhil might be in a position to expand EA’s expected impact if it added a cause area that allowed for more speculative investments in Global Health & Development.

My impression is that Open Philanthropy's Global Health and Development team already does this? For example, OP has focus areas on Global aid policy, Scientific research and South Asian air quality, areas which are inherently risky/uncertain.

They have also take a hit based approach philosophically, and this is what distinguishes them from GiveWell - see e.g.

Hits. We are explicitly purs

OpenPhil might be in a position to expand EA’s expected impact if it added a cause area that allowed for more speculative investments in Global Health & Development.

My impression is that Open Philanthropy's Global Health and Development team already does this? For example, OP has focus areas on Global aid policy, Scientific research and South Asian air quality, areas which are inherently risky/uncertain.

They have also take a hit based approach philosophically, and this is what distinguishes them from GiveWell - see e.g.

Hits. We are explicitly purs

GiveWell have looked into Global Health regulation - see more here: https://www.givewell.org/research/public-health-regulation-update-August-2021

Thanks for writing this Lizka! I agree with many of the points in this [I was also a visiting fellow on the longtermist team this summer]. I'll throw my two cents in about my own reflections (I broadly share Lizka's experience, so here I just highlight the upsides/downsides things that especially resonated with me, or things unique to my own situation):

Vague background:

• Finished BSc in PPE this June
• No EA research experience and very little academic research experience
• Introduced to EA in 2019

Upsides:

• Work in areas that are intellectually stimulating

This is super helpful, thank you!

Which departments/roles do you think are most important to work in from an EA perspective? The Cabinet Office, HM Treasury and FCDO seem particularly impactful, but are also the most crowded and competitive. Are there lesser known departments doing neglected but important work? (e.g. my impression is DEFRA would be this for animal welfare policy - are there similar opportunities in other cause areas?). Thanks!