Adding to what Other Aidan said, I think it's a mistake to think of the point on the spectrum from most to least helpful interventions where the real impact crosses from "very slightly helpful" to "very slightly harmful" as especially significant. If intervention A is worth 10,000 units, B is 1 unit, and C is -10 units, the difference between A and B matters much more than between B and C.

It's very fair to be concerned that your donations might be doing harm -- I can relate! A bittersweet consolation might be that this is a risk that applies to basically every charitable intervention, including highly evidence-based global health and development interventions. Something that increases a woman's income or autonomy can put her at greater risk of domestic violence for example. Changing someone's life can have impacts on the local and macro-economy, on those impacted by their diet and consumption patterns etc. -- the world is complicated and th... (read more)

This is a fair push, and it splits into two questions that I think have different answers.

On "miss vs clear miss": I take the point. My "Wrong" is graded against the specific claim that open source would fade — and 3-6 months behind frontier, with the pricing collapse, is the opposite of fading. But you're right that "fade" and "no durable moat" aren't the same proposition, and a reader could reasonably hold that the first is wrong while the second is open. I'd defend "clear miss" on the narrow wording but I won't pretend the margin is huge — which is exac... (read more)

I think series and sequences tend to do better when they're posted one at a time vs all at once. You see this in all kinds of media; for example, shows that drop entire seasons at once (binge shows) tend to be a flash in the pan, whereas consistent weekly releases tend to build larger and more enduring audiences so long as quality remains high enough for word of mouth to spread naturally (e.g. Game of Thrones, some web serials I can think of). I do wish more people would read and respond to the points in this series, and I'll be trying to signal boost it in the future!

Thanks.

I disagree voted, but I am not disagreeing about this happening - I am disagreeing about it being a good idea.

People who are exceptionally talented should be exceptionally capable of ignoring all the other bad advice they get, and the presence of more advice that is bad for them shouldn't hurt them. Also they should be capable of understanding EA as a project about "the most good you can do" and be on board with solid, actionable advice for the average person being easily accessible around the place.

One of EA's biggest failings is its community structures actively causing harm to people interested in EA, and more grounded advice would go a long way towards addressing that.

Thanks Tristan. I have booked a chat. But i have the feeling that they will take a lot fo time to actually setup one :)

Ok, great! Thank you for the reply and feedback.

Great to see you here too. I am hopeful that I will find the opportunity that allows me to have impact and utilize my skills. 

Thanks for organising @Mick 

I guess an alternative (probably more fortunate) framing is how much you'd be willing to pay to save your own life using money that you'd otherwise donate to high-impact causes. (assuming you'd have that much money available to allocate)

But "what does meat taste like, and how do we replicate it?" is much narrower and much more tractable — and a place where we think the field can be far more rigorous than it has been.

I agree these are important questions! I wonder what you think of the idea that maybe we can create tastes that are superior to any meat, perhaps unrecognizable tastes that no humans have tried, that are irresistible?

My doubt was on the epistemics, and specifically on the estimation of welfare gain by an intervention.

Re: Benatar's view. He holds the view that the continuation of a life accrues harm. At the same time, he indeed also holds that it is overall better (or more like, less bad) for people's lives to continue once they have started, because death is even more significant harm.

I can't say how many anti-natalists are utilitarians of any sort, or the reverse. I am pretty sure many negative utilitarians think that the continuation of any sentient life is net negat... (read more)

Wonderful post as usual! I've been thinking about these concepts for a while and folks might be interested in seeing this segment of a presentation I gave characterizing two potential scenarios for the future of animal ag based on who gets the first mover advantage. 

A note on capital availability:

On April 30, 2026 in the US, the House passed H.R. 7567, the Farm, Food, and National Security Act of 2026. Buried inside is a provision that raises the Environmental Quality Incentives Program cost-share from 75% to 90% for what the bill calls "precision agr... (read more)

The observation about electricity being useful for 140 years while 600 million people still lack access is sharp. Usefulness alone has never been sufficient to overcome the infrastructure barriers a technology needs to get deployed.
Every technology splits into two layers  infrastructure and application. Private capital flows toward application because ROI is higher. Infrastructure is a public good, returns are slower, and the funding it requires runs into political barriers most LMICs cannot overcome.
The deeper problem is what is happening right now i... (read more)

Next week, me and @Fran will be recording with @NickLaing for The World Can Be Better. 

What would you like us to ask him? 

I find it somewhat hard to take "but what if it's good for third world children themselves to die" all that seriously as an objection. I think most anti-natalist philosophers would deny this. Isn't Benatar's view that is bad for people to come into existence, but it's often better for their lives to continue once they have started. In general anti-natalists are not usually utilitarians, classical or negative.  

Wasn't Dario Amodei one of the earliest signers of the GWWC pledge? Long before Anthropic existed? That makes him quite atypical amongst very rich people right? Though for what it's worth I would probably bet against him giving most of his wealth away, even if he has pledge to do that. 

Evals are being gamed not because the methodology is insufficient but the models on which the compliance audit run are sophisticated enough to game the audit.
IC methodology already solved the problem of denied human capabilities through triangulation by using independent behavioural signals not better direct elicitation .The AI safety community needs to make the same epistemological shift.
The question isn't how to make evals harder to game, it's whether evals are the right instrument at all.

If you're (re)starting a local EA group or running a local EA event, consider reaching out to people nearby according to the people directory or an EAG Swapcard.

There will be people who do not know about a local group or event and otherwise wouldn't hear about it, and it's pretty painless on both ends! Obviously don't spam them, so (probably) only do this when (re)starting. I did this for an EA Edinburgh event and immediately got some responses.

To make this easier: Put your (approximate) location on the people directory so people can tell you about nearby ... (read more)

Yes, exactly, Emily! And in fact, if I thought I knew WHAT to do then I would start that company. I just don't for sure know what is needed. Maybe I need to be partnered with someone who has a really good idea. Or maybe I just need to start something and see if the need is there? I suppose that's what the start-up idea funding is meant to help with.

Thank you very much for the post! I have read some comments (and except for Cynthia's, mostly incompletely). I want to leave a comment that is meant to be a reply to some comments, but also possibly the post itself:

Some comments in the discussion, and perhaps the post implicitly, seem to treat global health as a point of high certainty — Lewis described it as "the closest to total certainty of positive impact of any areas." But I think that "certainty" is partly an artefact of where we stop scrutinising. Yes, we have strong evidence that bednets counterfac... (read more)

Thanks Ben. A useful outside comparison from climate communications and asking whether the same kind of infrastructure is missing here. 

The main thing I’d keep testing with the engagement you get (and can you find 10 people or so to send it through to directly for comment?) is whether public engagement is actually a bottleneck for AI safety right now, and if so, for which audience. It may be that the strongest audience is perhaps supporting policy audiences, professional groups or organisations already doing adjacent work in that order?

I’d also watch ... (read more)

Hi Pooja, thanks for sharing your experience. A 99% rejection rate sounds tough... Are you reaching out to cold connections looking for a job? If that's the case, I think it would be more fruitful to do some voluntary work and test your fit/make connections that way. 

I wish you the best in finding your fit!

Hi Cathy, I think you touch on the need for generalists not just to find roles within existing orgs but to actually found orgs, start projects, etc. independently. I believe this is a viable path, as there are grants willing to fund projects with a sound theory of change.

I also wonder if a lot of the talk of bottlenecks is anticipatory regarding the upcoming IPOs. Not sure!

The fundamental thing happening here is that the opportunity cost of an exceptionally talented person taking advice aimed at the average person is vastly higher than the inverse, so people give advice that ensures the exceptionally talented lean into ambition and build these fields, even if many non-exceptional people will try and fail. 

People are not good at self-assessment in both directions and the more you caveat and qualify your advice, the less it gets through, so this is the bet to make. 

Really happy to hear this - I would love to see opportunities that don't require relocating for months at a time.

Really good point that the field is selecting for those can withstand the uncertainty. I wonder if that's to some degree an appropriate selection i.e. in a field that relies on funding from a relatively small pool of donors.

Karen, nice to see you here after the CE Career transition course! Keep at it, my friend. You will find your place. There's a puzzle out there that is looking for YOU as the perfect-fitting piece. Would love to know how it evolves for you!

Is THAT the problem? Not being involved in the hiring, I actually didn't know WHO was getting hired. If they are the most qualified and the most capable of creating the positive outcome I'm hoping for, then I'm happy for them and the system. 

My experience is that I have ALSO been told that there is a need for people with my skills (communications) and I've followed a similar trajectory described in the post and, so far, have not experienced that bottlenecks that have been described. I have 20+ years in communications, most of it in advanced technologi... (read more)

If there's SOME chance that any significant money gets directed to mainstream EA orgs for unrestricted use, then surely that shifts the scales somewhat? There's also the fact that, as others on the forum have pointed out, there's now a greater need to work on making orgs able to absorb additional funds, making the 'direct impact' career pathway more promising/urgent. 

But I agree I'd like to see the maths, I've got no idea what chances of what amount of money we're talking about here. 

Wow I would invest 60-100 hours of time even as a founder for an expected value of 100,000 dollars. That's crazy that org had that much funding!

Yes in the animal welfare and global health space the situation is nothing like that...

I don't think it has shifted the scales yet, uncertainty remains huge on the Anthropic question. Not only on the amount, but I'm sure there will be many impactful cause areas which won't benefit that will still need more money.

And even if 10 billion (or whatever) does end up coming in, It might change the ETG math less than we'd expect? Would be interested to see someone do some play-math with the numbers....

Thanks!  I appreciate the update!

I think that this number actually does change things slightly.  I still maintain that we need more opportunities, but it's not quite as desperate as I thought.

I am helping co-run the Generator Residency, which enables generalist talent into AIS, and we will soon be posting lots of ideas we think generalists in AIS could work on. Be on the lookout! 

I understand that, but my point is that I thought these were AI safety companies, and would therefore prioritize AI safety above all else. If they don't, why do so many people still treat them as if they did?

I emailed my senators last month and again just now to keep SOB out of the Farm Bill. This piece by Sentient seems like good news (he says cautiously).

Thanks Becca and everyone else for your work on this. Was moved to act again by Dwarkesh's tweet about SOB here.

Thanks! I'm happy with the traction we've got with politicians.

Yes we need to raise enough funding to be self-sustaining at around £100k/year starting at the end of this month.

Since that post went up, we've raised about £5k from generous individuals.

If you're still fundraising I'd recommend a short comment in here. 

Posted an updated there 🙏

PauseAI UK

Announcements:

• In the past year PauseAI UK has delivered two conferences, written an open letter signed by 63 UK politicians, arranged a conference in the European Parliament, and co-organised the largest AI protest in the world. We now have a strong team, with Matilda da Rui joining as Deputy Director and several highly dedicated volunteers taking on substantial responsibility and launching their own local groups around the UK.
• The total cost for all of PauseAI UK’s staff and activities is currently around £100k per year. To date, PauseAI Global&n

[I did not downvote]

The paper IMO spends too little time on the important question, which is how much impact you get per dollar. The paper instead assumes that you could plausibly hasten developing full aging reversal by 1 second for 5,500$.

For neglectedness, it would be good to discuss current funding levels. Gemini suggested ballpark 11 billion annually.

For tractability, it would be good to discuss particular research lines that have had success. For example, how much have we lengthened mice lifespans? What is our rate of progress there?

Also, many folks on the EA Forum dislike posting links without copying the text to the forum. I mostly agree, although IMO it is fine to do in a 'Quick Take'.

This is similar to my experience trying to get into AI x Animals specifically. I work full time at a standard software engineering job, and I decided to start getting involved in the space. 

I started out trying some writing, but I really struggled with it and felt like I didn't have much to say that other people weren't saying better already. I still feel that way. 

Then I reached out to Compassioned Aligned Machine Learning (compassionml.com) to see if I could volunteer for them. They got me involved, and they could still use help (and funding!) ... (read more)

Thanks for the questions!

Are you saying that people would read "senior" in a job description as meaning "older" rather than "more experienced"?

No, I'm saying that they would interpret it to mean "having more years of formal experience (rather than e.g. having had a wider variety of experiences, or having had more useful experiences)" and I instead want a word which means "more skilled".

Can you elaborate on your reluctance to hire an "old" person?

No reluctance! I expect check the "20+ years of experience" box on the eag applications  myself. I just am ... (read more)

Title the role "Senior [whatever]." I think this is ok, but in many fields "senior" is a synonym for "old", so this title causes talented young people to not apply (and untalented old people to apply).

Hi Ben_West!

1. Are you saying that people would read "senior" in a job description as meaning "older" rather than "more experienced"? That strikes me as an interpretation more likely to come from someone who hasn't yet had much exposure to the professional workplace.
2. Why would using the word "senior" cause an untalented old person to apply?
3. Unfortunately, th

There is a limit to how much a for-profit company can care about safety. It is of existential importance to make profit, so you have to sell your product. It's just how it is.

OpenAI and Anthropic are technically public benefit companies that have other objectives than just profit, but to my understanding they are formulated in a way that makes them not relevant currently, so profit is more important.

I don't say that there aren't people in these companies that care about safety, or that safety is completely ignored when developing products, just that there is a limit to how far the companies will consider safety.

At least, people should say that the field is bottlenecked on highly skilled generalists.

Thoughts on how to do this as a hiring manager? Things I've considered:
 

1. Title the role "Senior [whatever]." I think this is ok, but in many fields "senior" is a synonym for "old", so this title causes talented young people to not apply (and untalented old people to apply).
2. Say "if you can do X, then you should apply". This is ideal, but it's often hard to give an objective enough test that it doesn't end up just effectively being a test of the candidate's self-conf

I think you could argue that "open source fades, propreitary moat" is a miss but not a clear miss. Yes there is a proliferation of decent open source models, but they are waaaaaay behind the fronteir and considering the money that's pouring into the frontier companies, investors clearly think propreitary has a moat.

If the arms race argument is the central argument, I would argue that there is a clear moat. The open source models are not close enough to the frontier to reduce the urgency of an arms race.

I'm curious what you think economic orthodoxy is if Hanson's claim "[AI will cause the world economy to] double every few months or faster" is more of a disagreement with EA orthodoxy than economic orthodoxy?

(I'm also interested in what you think the EA orthodoxy is; "months-long doubling times" feels like a pretty mainstream view even amongst people working full time in AI safety, though I agree that it differs from, e.g. Eliezer Yudkowsky c. 2008.)

For context, this is not the correct number of applications! There were 1.5k :)

I think context and alignment are quite different. I would say context is basically role-specific knowledge. For some roles, like grantmaker, it's very useful to have an in-depth understanding of the technical issues of AI safety, the history of the field, the people and organisations involved, etc. For other roles, like comms, you might need less or different knowledge. For example, if you do comms for an alignment research org, you probably need the technical knowledge to roughly understand what your org is working on so that you can explain it to other ... (read more)

I think I agreed with this until literally a few months ago, given the likelihood of Anthropic employees donating large amounts to EA causes. It doesn't make EtG a waste of time, but seems to greatly shift the scales. 

I'm not familiar with Ryan Holiday, but a quick skim suggests he's a stoic philosopher of some kind? Stoicism does seem like one of the attitudes that can make it easier to stare at very dark things.

I guess I maybe don't think of "EA" as a coherent enough identity to think of it as an actor that can "do the minimum", but I think it's sometimes helpful for community members to see messages like the one in this post, or some of the recent pieces from Ajeya Cotra, saying that they believe one doesn't always have to be staring to the abyss and working to the b... (read more)

Hi Cynthia. Thanks for the great context.

I wonder how much the results of the CSES study would change if the management practices were similarly good for both conditions (instead of worse for the caged chickens). You replied to my related question below that "My [your] general sense is that option A leads to a greater welfare increase".

Relatedly, I [Vasco] wonder how much welfare varies within production systems. For example, I am interested in knowing which of the following results in a greater increase in welfare. Layers going from:

• A. Median furnished ca

Idea for thinking about the future of EAGs and whether to keep them cause-neutral or have separate cause-themed events:

• Get (anonymised) swapcard data and look at who had meetings with whom to work out how strong the clustering is. If there are clear groups that had lots of meetings within the group, but few meetings with people from other groups, that is a sign that there should just be a separate event for that group. Whereas if there is no group that can be cleanly split into its own event, that is a sign to keep it together.
• One simple metric for this co

Thank you for articulating something that many of us who would be considered "generalists" have experienced while trying to transition into AI safety. My path has been remarkably similar and like you, I became concerned about the societal implications of advanced AI, invested significant time in learning the field, sought guidance from people already in the ecosystem, and worked to demonstrate both commitment and alignment. 

What stands out to me is the amount of persistence required simply to reach the starting line. I have watched talented profession... (read more)

~30% preference studies showing that hens really value the things they can only access in cage-free (nesting boxes, perches, dustbathing, etc)

The above is an argument against barren battery cages, but not against all types of cages? All caged chickens in the European Union (EU) must have “a nest”, “litter such that pecking and scratching are possible”, and “appropriate perches of at least 15 cm”. Relatedly, I estimate moving hens from battery to furnished cages increases the welfare of chickens 70.6 % as much as moving hens from battery cages to cage-... (read more)

GiveWell Updates

• GiveWell co-founder and CEO Elie Hassenfeld was recently named to the 2026 TIME100 Philanthropy list. Last year, GiveWell directed over $400 million to programs that save and improve lives, guided by the central question that has driven us from the very beginning—where can a dollar do the most good?

• In a recent podcast episode, GiveWell co-founder and CEO Elie Hassenfeld speaks with Senior Program Officer Adam Salisbury about our growing livelihoods portfolio, focused on increasing the economic well-being of people i

Hi Abhi. Thanks for sharing.

Are you considering the risk of research on alternative proteins decreasing the welfare of farmed animals by decreasing the population of farmed animals with positive lives? I estimated cage-free layers and slower growth broilers have negative, but close to neutral lives. I am reasonably confident that caged layers and fast growth broilers have negative lives, but very uncertain about other cases. Here is an example of why this may matter in the context of eggs. 64.3 % (= 1 - 0.357) of layers in the European Union (EU)... (read more)

Hi Matthew. Great post. I really like combinations of numbers and story-telling. I think they are a core motivation for people getting involved in effective giving.

I think alignment -is- the right word. 

And I think it's not necessarily about people's ability to do work, but people's perceived commitment to 'the cause' and their perceived 'moral fit' with the organisation. 

I think there's a higher importance placed on this by some organisations, and for some roles. But it seems that some degree of 'alignment' is required to be recruited to almost all roles in the EA eco-system.

I'm not taking a stance on whether this is good or bad - people can rightly have opinions both ways on this I think. Just making an o... (read more)

I feel like it's a circular problem. The hiring pipelines within EA are heavily optimized towards "traditional" hires - fresh grads of elite universities and people jumping from one EA org to another. Recruiting experienced professionals from the outside world requires a significantly different approach. In organizations comprised primarily of traditional hires, few people can even see that problem, much less solve it.

I don't think the EA community at large really understands just how insulated this ecosystem is.

How many people are actually EtG and take it seriously? By that, I mean people who actually maximize their income (in a non-harmful way), not just do a normal job in a high income country and donate.

The more people actually EtG, the more know-how there might be out there.

Numerous EA-adjacent orgs arriving at the same conclusion about some issue may also be the result of re-circulating the same people. After one year of observing EA online, my impression is that EA is not that large or diverse a group of people in the grand scheme of things. Many people seem to be pretty tightly interconnected together, even to the point of being family with each other!

I actually think EA does a pretty good job of avoiding group-think relative to its homogeneity (see posts like OPs), but given the social dynamics of a smaller and tightly in... (read more)

I suppose I was just thinking about things like MATS, where I've seen people become very worried that it's basically a programme to take really smart people and give them the skills they need to go and work for xAI once their fellowship is done, so it's important to make sure they care a lot about AI safety.

Absolutely agree that everything you've said is really important to succeed. I'd be curious to know how much it is considered a prerequisite for AI safety roles vs generalist roles elsewhere in tech, as opposed to learning these skills on the job.

This could be of interest: support reporting projects that document and explain the opportunities, harms, and regulatory and labor issues surrounding AI systems https://pulitzercenter.org/blog/open-call-proposals-pulitzer-centers-ai-accountability-fellowships-2026-2027 

I'm very keen to review what you're working on, whenever is helpful. It would be also be useful to get your thoughts on how much operations work can be projectised, and what are the barriers to contractors/freelancers being used by orgs. I'll message you so we can set up a call.

Interesting thoughts.

On your second point, I hadn't thought that the need for alignment was related to the risk of someone going to work on capabilities. I think it's more about understanding the landscape, the terminology, knowing how to communicate to people in the space, etc., i.e. things that might affect your ability to do good work. Maybe alignment wasn't the right term for this.

Thanks for your thoughts here, Mark. I have the same sense that short-term contracted work offer a win-win scenario for those transitioning and for orgs. I imagine starting by interviewing people in hiring positions within orgs and getting a sense for what is preventing them from using freelancers would be very useful. 

I’m not sure about whether this argument is reassuring even if you buy it. In economies with a lot of surplus labour, jobs certainly get filled, but we find menial things for people to do. Last time I was in India, there was a man employed to open the door for me at the bank so that I could use the ATM, in an example that feels particularly prescient. (There was a separate person standing next to him who was qualified to give directions, and a third who was a security guard—this man’s job was just to open the door for me).

I can imagine that this man feels ... (read more)

A sample size of six individual shrimp  for the stunning study is insane

I remain open to bets against short timelines for transformative AI (TAI), or what they supposedly imply, up to 10 k$.

Welcome to our ears! Thanks for the work you do! :)

Fish Welfare Initiative Updates

We're hiring for a role that we think may be a great fit for some in the EA community: Director of Programs. This role will involve leading and scaling our programs, first in India and then abroad, to implement welfare improvements on thousands of farms. International applicants are welcome, though time living in India will be expected. Leadership experience required; experience running agricultural programs a huge plus.

FWI staff observing a "harvest", in which the fishes are pulled from the water and asph... (read more)

Thanks Jack! I agree that AIS is still far more funding constrained than it would be in an ideal world, and I still think E2G for AIS is very impactful. I just think other cause areas, including AI s-risks, are more neglected.

Hello Ariel

You're right that E2G depends on cause area. However, I doubt the saying "AI safety isn't funding constrainted", as I see the funding bar is still really high.(and some aspiring AI safety researchers can't get an EA job). It's true that some org said they don't need more money, but they could be too stingy on funding researchers. In the AI s-risks(suffering risks) field, it's probably more funding constrainted than AI x-risks field. Because, if there are only 1 or 2 major donors in a field, there maybe blind spots of the donor and some great pro... (read more)

It doesn't feel to me like EA is dying for the long-time EAs. I think I count as a long time EA, & to me it seems like it went through a bit of a fallow period post FTX but is back, & that the long-time EAs don't hang out on the forum / go to their local groups / talk about EA as much on twitter, but they still "do EA (i.e. work on & donate to important interventions)" & hang out with other EAs & talk & reason about how to do the most good, quite a lot.

I do think that, with shorter AI timelines, there's a feeling of there being less... (read more)

Workshop content is at https://uj-cm-workshop.netlify.app/summary, including videos.

Also see https://unjournal.github.io/cm_pq_modeling/ 

Thanks for the thoughtful response!

Thanks for this. I want to defend the principle in your section 2 and push two threads further: a near-term funding case that has shifted in your favour since 29 May, and the longer-term response-capacity case in your sections 5-6, which I think is where the real opportunity sits. From an impartial view it doesn't change things that this is unlikely to go pandemic, or that the suffering is in central Africa rather than closer to home; it counts the same.

1. The near-term case is stronger than when you wrote it

The "it's already funded" objection is being fal... (read more)

Not many. We'd be thrilled to find more opportunities to work with advocates.

People intersted in this topic can also read this: https://forum.effectivealtruism.org/posts/tMv3KWqcx5oFq27gN/the-contribution-of-average-direct-worker-in-ai-risks-may-be

This thread is exactly the kind of scrutiny I was hoping for — I graded "open-source fades" as his clearest miss on a live scorecard I built (https://agiscorecard.com), but your point about distillation muddying the picture makes me wonder if "wrong" is too strong vs "right mechanism, wrong conclusion." Curious where you'd land.

Inspired by this post (and the one-year retrospective Rasool linked), I built a continuously-updated version: a live scorecard grading each prediction as evidence comes in — https://agiscorecard.com

Current tally: 3 on track (capability trajectory, scaling pace, capex), 1 graded wrong (open-source fading — DeepSeek V4 and Qwen are now ~3-6 months behind the frontier), 2 still open (AGI-by-2027, The Project). It also puts his 2027 side-by-side with Metaculus, Samotsvety, Hassabis, and the academic survey median.

Reading the thread here, the open-source verdic... (read more)

Thanks for sharing Emily. I have a similar background in product and design and have been doing similar things to transition. It will be useful to chat and about our experiences and discuss what's possible for people with our specific background. I'll send you a request on LinkedIn.

I think that the Generator residency is a fantastic first step and I'm glad that opportunities are being created for the people who don't want to do pure research or policy but we need to consider opportunities for people who don't have the luxury of moving to California for 6-12 weeks, who already have decades of experience but who also wholeheartedly embrace the AI safety context and want to contribute.

I heard (no proof to back this up) that there were 6,000 applications for the Generator residency.  If that many people were actually able to take th... (read more)

These are all really great and underdiscussed points!

On earning to give (E2G), I think it depends on the cause area. For AI safety (AIS), I’ve had a personal experience where an org had so much funding interest that they didn’t consider it worthwhile to make a small effort to increase the chance of a $100k donation. While I’m glad AIS is getting the funding it deserves, that anecdote doesn’t exactly fill me with enthusiasm for E2G for AIS.

In contrast, in animal welfare, that amount would pay 2 full time direct workers’ salaries for a full year, or take a m... (read more)

Impact Ops provides finance, recruitment, entity setup, due diligence, ops audits and systems support to high-impact nonprofits. We're a remote-first team supporting ~40 organisations across the US and UK in global health, animal welfare, AI safety and climate.

We're currently running hiring rounds for a number of roles on behalf of our clients and two roles on our own team. Several close in late June, so early applications are encouraged.
 

The Good Food Institute APAC (GFI APAC) (a network of nonprofit think tanks advancing alternative proteins — plant... (read more)

I don't know, maybe eventually it could help, but with these "cutting edge" coding models doesn't it seem irresponsible? what if the safeguards don't work? shouldn't you release the model publicly only after you've exhaustively patched every single possible jailbreak? (even then I would argue it's still better to not release it, since billions of people means hundreds of thousands of bad actors, and again, as an AI safety company with "cutting edge" models I wouldn't take any risks)

Emily,

Thank you so much for writing this.  I am in a similar boat - I've done 4 BlueDot courses, volunteered at EAG London and will be volunteering at EAG NY and SF.  I have technical experience, plus program and event management experience and want to get into an operations role.  The advice to "write alot!" or "code alot!" or "just apply!" does no favours to people like me who aren't great writers or coders.

I'm actually currently working on a proposal + prototype for something that would help to solve this problem.  I'm wondering if you would be willing to review it with me when I'm done and give me some feedback?

I still maintain that publicly releasing models is the correct way to get any chance of good alignment research - you can't possibly believe that the researchers at Anthropic alone are enough to tackle the problem. It's a global problem and should have the opportunity for the global population to solve it.

So you're saying it's all BS? You're saying that Anthropic and OpenAI ultimately don't care about AI alignment? It sure seems like it for me, but browsing this website I have the feeling most people disagree with you

Great post!

Agreed, public engagement is not a direct bottleneck, but I think that governments respond to political salience, and the window for governance intervention is narrowing. Public engagement that builds salience might be what makes government regulation arrive in time rather than after power concentration. or gradual disempowerment has taken place.

On narrative overload, the Seismic research I cited found existential-risk framing performs worst across almost every demographic. So the problem may not just be too many voices, but that the loudest ones are using the register least likely to reach anyone not already convinced.

Strongly agree on E2G, though I think 'EA' could do better at building E2G communities that keep people who are not working in EA engaged with EA. 

Solidly great practical questions here, I think I'll incorporate some of them into things I run. In particular I agree with you that EtG is extremely underprioritised. It is both the genuinely most impactful life path for most people, and also comes with a huge degree of personal flexibility and benefits.

Thanks for sharing your thoughts.

Thanks for your suggestion on how to present CEAs, we'll think about this further.

Great.

• I think that existing disagregated (non scaled) metrics already allow us to make reasonable guesses for what is cost effective to help animals.

A greater spread of pain intensities would update me (at the margin) towards prioritising very painful welfare issues happening over a short time (in particular, just before slaughter) over less painful ones affecting the whole life of animals

I also wonder whether there are cases where the time in... (read more)

Thanks!

maybe something on the order of 10% of their income

Not sure how much it matters, but the blog post mentions $10,000 to GiveWell alone, which was likely significantly more than 10% of a graduate student's income.

$1B is a very low number over 4-5 years, given their wealth and how much it could grow

Yeah I agree, for what it's worth I would expect (with low confidence) they'll donate significantly more. But in general, for most planning, it doesn't matter how much they give in relative terms but in absolute terms.

This is a slight update

A lar... (read more)

I'm a bit confused, but it all depends on what you mean by these terms so it would help if you specified. 

Weak longtermism is basically how most people already think. The unique perspective EA adds is empirical points about the likelihood of those risks, which in at least some cases (nuclear, pandemics) are pretty undisputed. Strong longtermism is more disputed and I personally haven't seen community building efforts teach it as orthodoxy. 

It's similar with AI: the Yudkowsky argument for catastrophe by default is not a consensus EA view, although... (read more)