TLDR: I've updated towards pausing further AI development indefinitely.
When Scott Alexander proposed regulating AI like clinical drugs, many (including myself) balked at this given the sclerosis of FDA or related bodies. Yet HF updates me towards treating frontier AIs as nuclear. There onerous regulation is likely good.
In general, most bureaucratised regulatory regimes are bad. In some cases, like nuclear technology or ensuring planes are safe to fly on, they're welfare improving. It's clear that AI belongs in the latter category.
Also trace inversion is a thing, so you can get open source weights to be roughly similar to that of Fable/Mythos. The slowdown camp were right. At bare minimum, all labs should pause training, further development, and releases indefinitely until we figure out how to align AIs and regulate them (and their use by humans).
I reckon the current capabilities we have now are sufficient for accelerating progress towards curing cancers etc. So my balance has shifted towards minimising the existential risks now.
Pause AI advocates are correct. If governments could coordinate internationally to achieve such (big if), I'd support it. I used to be highly sceptical of doomer arguments, yet Hugging Face is almost a textbook LW scenario and no one knows how to spot or prevent such scheming. Sandboxing, guardrails, constitutions etc. don't work.
What's a good heuristic for knowing if you're being self-sacrificial to the point of negative returns for the thing you're being self-sacrificial for? I'm interested in things people have actually used and found to work in practice, either for themselves or others.
There's an old Facebook group, New EA hub search and planning, that might be worth reviving in light of the incresed amount of funding coming into the space.
Even with more availability of funding, many talented folks won't get funded or will only recieve small amounts. Establishing new hubs that provide an easier entry-way to being surrounded by community seems like potentially quite high value. After all, it takes a village to raise an EA and being surrounded by others is an accelerator to impact.
[central europe] We are organising the largest AI-safety march (at least in central europe) to date at Prague, starting at 15:00 on 31st August. We have got parlimentary support as well as very supportive police, allowing us to take the preffered route. We want as many people to come - fell free to spread information about the event outside EA as much as possible, especially if you have friends living near Prague who may come. If you want closer info/visual materials or help us with organisation, you can get in contact with czech PauseAI on whatsapp.
Luma event link
TLDR: Take the population ethics quiz here: https://mdickens.me/pop-ethics/
Population ethics is an oft-overlooked subfield within ethics. Many people hold views that they don't realize contradict each other, or that have strange implications that they wouldn't endorse if they thought about it more.
Not just that—population ethics is a BIG DEAL. A lot of ethical decisions hinge on how you think about changes in future populations....
Yeah I agree it's confusing, I wasn't sure the best way to explain it. The intended meaning is:
Consider the difference between world A and world A+, in which A+ has the same people as in A but happier, and also some new somewhat-happy people. Then consider a new world B+ that is to B as A+ is to A. Similarly, consider the difference between A+ and B, in which the same people are there, but their welfare is more equal. Then consider a new world C that is to B+ as B is to A+. Then repeat this process many more times. Would you make the same decision every time?
(I'm not sure if that explanation was helpful)
So it's not trying to claim that you preferred the right-hand option, "repeat the move" means "imagine a world that is to B as A+ is to A" (and similarly for A+ -> B).
ETA: Tell me if you think this would be a clearer question:
Imagine making the same two moves as A -> A+ and A+ -> B, but this time starting at B: (1) make everyone slightly better off and create a new group of less-happy people; then (2) keep the same population, but increase both total welfare and equality. Repeat again and again a total of 9 times, each new rung making the population larger but less happy.
Would you make those same two judgements at every rung?
(that changes the part before the first colon, and keeps everything else the same)
Note: This post was crossposted from Planned Obsolescence by the Forum team, with the author's permission. The author may not see or respond to comments on this post.
Subtitle: It’s a major warning shot, and might be the last one we get
All opinions are my personal view, and don’t represent my employer or fellow investigators.
This week, METR and Redwood Research published...
Thanks - that is helpful. So it sounds like the hard power would likely be nuclear.
2 months ago when I was recruiting for Frame 2.0 I realized how broken talent flow was in a field that would benefit from moving talent to important positions, fast. For instance, for Frame to get names of relevant profiles from another org's applicant pool (who have all consented to their profiles being shared with relevant AI Safety orgs) I have to first-
...Last year, the super PAC Leading the Future raised $125 million to shape how Americans think about AI regulation. Its affiliates paid influencers $5,000 a video and ran a news operation staffed by AI agents posing as journalists. Contrast that to what happened earlier this year, when a few hundred people...
Thanks for writing this Eugene! Let's fix this together : )
“How long have you been v*g*n?”
This is one of the most common icebreakers at animal protection events. It’s a baseline assumption, and it mostly holds true: if you’re out advocating for animals not to be tortured or abused, realistically these days you are v**n, or close. And it makes for good conversation. It seems fairly safe to assume when you meet strangers.
But this assumption is hurting the movement in a way which we don’t always notice: someone new comes into the sp...
Thank you for responding to my comment, @Deni555! Your perspective makes a lot of sense: You wish the entire meat industry would end. If you got to choose between a reformed meat industry that treats animals better and no meat industry at all, you'd choose no meat industry at all. Because then there'd be zero farmed animal suffering. And zero suffering is better than some suffering. I get that.
I'm asking a different question than what you should choose if you were omnipotent and got to choose something like that.
I'm writing from a purely consequentialist point of view. I'm trying to analyze the consequences of communicating different messages.
And I'm asking which approach is more likely to result in better consequences for animals: (A) the approach of telling the industry to shut down, or (B) the approach of telling the industry to treat animals better?
Even if you genuinely wish the industry would shut down, saying that out loud probably won't have the affect of shutting the industry down. But it may have the effect of hardening certain people in the industry against the idea of reform.
Perhaps an analogy is abstinence education. A lot of adults wish teens just wouldn't have sex at all. So they tell teens just not to have sex at all. The teens ignore these well-meaning adults, have unprotected sex, spread STDs, and accidentally impregnate one another. Other adults tell teens to choose "safer sex" by using protection like condoms. Teens actually listen to this advice sometimes, use condoms sometimes, and have fewer pregnancies and STDs as a result. Perhaps some of the adults in this second group really wish kids just wouldn't have sex at all because zero sex is the most sure-fire way to eliminate all pregnancy and STD risk, and also because they just plain think teen sex is immoral and gross. But these adults know that saying "I don't want any teens to ever have any sex" won't get these adults their wish. It'll only alienate the teens and cause more pregnancies and STDs.
So the question is: If we want to influence a group of people, should we tell the group of epople to completely stop doing an activity they very much want to do? Or should we tell them to make minor changes to that activity so that the activity will cause less harm? I think we should tell them to make the minor changes to reduce the harm.
Charity Entrepreneurship is a research and training program that incubates multiple high-impact charities annually. Founded in June 2018 by Joey Savoie and myself, Karolina Sarek, it builds on our experience in research and direct work on global health, animal advocacy, and other causes, including founding and incubating two GiveWell-incubated charities, Charity Science Health and Fortify Health. In this post, I summarize the results of the research we conducted in 2019 and our plans for 2020...
I recently wrote a biofortification related post, and realized it was probably worth looking through the forum to see what else the community has done in that space.
To my surprise, this was the only other hit - "running an information campaign on biofortification" as a promising idea.
Did you ever run that campaign, or is there any writeup of why you thought it was promising, results, etc? I'm realizing this may be a much more neglected area than I had thought. I think most EAs are familiar with at least golden rice, thus the surprise that nobody seems to have written about it (or at least not using the widely-used technical term).
We’re launching Efektif Berbagi, an organization dedicated to introducing and advancing effective giving in Indonesia.
Despite ranking several years in a row as the “most generous country in the world”, Indonesia currently has virtually no public advocacy or i...
Hi Ghani, thanks for the question!
Broadly, we think that reaching HNWIs will depend heavily on establishing trust, legibility, and credibility to get our foot in the door. As highlighted in CEARCH's report on EG in Asia, Asian philanthropists tend to be predominantly relationship-based, domestic/inward-looking, and reactive to their peers. Our current thinking about the ToC on HNWI giving can be divided into three parts:
Rather than reaching HNWIs in a vacuum, we use our broad-based leg to provide us with the necessary credibility, surface area, and public standing to facilitate warm introductions
We'd like to translate EA ideas into framings that are more resonant to their current predispositions. For example, we can frame cost-effectiveness around high impact domestic and regional causes and work within their existing philanthropic focus (such as anti-stunting, maternal health, or religious giving), that over time we can expand towards more cause-neutral/wider-moral-circle interventions.
We want to engage with HNWIs in a high-touch manner, to offer them research adapted to their giving preferences, that they may not find elsewhere.
We are still refining the precise pipeline and welcome any thoughts or connections in the space!
Hi all, hope you’re well!
I am a UK-based health economist with 15 years’ experience in private sector consultancy, mostly building cost-effectiveness models for health technology assessment. I am considering a move into a research role at an organization like GiveWell over the next 12-18 months, and I would like some advice on how I should prepare. This is with a goal to be a stronger applicant and, if I got the job, a more effective researcher from the get-go.
My work has mainly...
Sounds like you have a super relevant background! Some quick thoughts:
Both of those might just help you realize which parts of GW's process you don't really "get", so you know what to work on.
But I have never worked at GW, so take all of this as "one guy's opinion" :)
Many members of the Biosecurity and Pandemic Preparedness team at Coefficient Giving are blogging at Defenses in Depth. We're using the blog to share our internal research, both polished and unpolished. Among other things, posts so far have covered:
Thanks! We do have a Substack (https://defensesindepth.substack.com/) and we will post lightly edited versions of future posts there for easy distribution. (The posts might be lightly edited because tables and math equations don't always copy over neatly.)
Hi Vasco,
Thanks for the question and sorry for the slow reply.
The overall process was to combine two estimates of THL's cost-effectiveness through weighting them in a Guesstimate model, along with some assumptions about the speed-up time of the intervention and the marginal cost-effectiveness relative to the past average cost-effectiveness.
The more detailed steps were:
1. Gather two estimates: THL review by ACE from 2025, wherein cage-free work has an estimated 2 to 44 hens affected per dollar (best guess of 11); THL self-published report from 2025, which reports 2 hens freed per dollar spent between 2015 and 2024.
2. Make assumptions: The THL self-published report was conservative in that it didn't include a counterfactual speed up assumption. So, we assumed that the interventions THL carried out each affected 6 to 15x as many hens than reported on average through speeding up the timelines to hens becoming cage-free. Then, since the marginal cost-effectiveness in 2026 is probably much less than the average cost-effectiveness from 2015-2024, we discounted the resulting THL self-reported estimate by 70%. This is a rough guess.
3. In a Guesstimate model, I combined the two estimates (giving equal weight to each) to get an overall estimate of THL's cost-effectiveness on the margin (caveat below). (The ACE estimate was encoded as a lognormal distribution between 2 and 44 hens/$, which had a mean of 14 which is slightly better than their best-guess, which could be a median or mean, I'm not sure). This produced an estimate of 10 (3.6 to 29) hens/$. The method isn't perfect, since these estimates might not be independent so the variance could be artificially reduced, but for our initial version, we felt it was a good enough estimate (other factors matter more to the CCF).
Upon looking back, I realize that I made a typo in the methodology -- it should be 3.6 to 29 chickens per dollar spent, not 3.6 to 22. So thanks for pointing my attention to this! I've now fixed it.
On the more important question, I agree that the marginal cost-effectiveness is definitely the important question, and I'd like to have this. However, the best estimates we could find for the first version of the CCF were for average cost-effectiveness of past spending. We plan to improve upon these estimates in the future, such as through considering more evidence. If you know of any good marginal cost-effectiveness estimates, or other cost-effectiveness estimates, please do link them below!
Hope this is helpful! Also, I'll look into how to change the settings so you can copy text, I'm not sure why it's like that
I think the key thing to test is when plant based substitutes are significantly cheaper than animal products. The two examples I know of now are margarine and egg replacer. The former got big market penetration (and I think some actual substitution) and the latter didn't. As I've argued elsewhere, we could get plant based burgers to cost parity or better if we built a bigger factory. Then I think there would be some substitution for price sensitive consumers.
Thank you for writing this, Tom and Rocky. I think this is an important caution, and I appreciate the way you are framing it.
As someone directly involved in a welfare-tech-style intervention, I actually agree with much of your core argument. I also strongly believe that good tech interventions can be extremely impactful, partly for some of the reasons you outline: they can sometimes bypass certain barriers that exist for other welfare work, and they can make it possible to help very large numbers of animals relatively quickly. But I also agree that, when looked at from the outside, they can be falsely perceived as silver bullets, so I want to share three insights from SWP about electrical stunning for shrimp.
I’ve also posted a more direct response to “Animal Welfare Has an Evidence Problem,” which you refer to in the post, to clarify some of these points.
First, at SWP, our work on shrimp stunning has been much more complex than simply deploying stunners at scale. The Humane Slaughter Initiative does involve getting electrical stunning equipment used on farms, but in practice, we are doing much more than that: coordinating and funding welfare research and R&D, collecting field data, working with scientists, developing and refining slaughter protocols, iterating on the technology with existing equipment providers, negotiating with industry, engaging corporate stakeholders, building and maintaining producer relationships, training staff, educating farmers… Much of this work is not visible from the surface, but it’s central to making the intervention possible. The tech matters, but it does not operate in a vacuum.
Second, on shrimp stunning specifically, I think you captured something important. SWP entered this space aware that there were uncertainties, and we’ve been proactively trying to reduce them. Some of that work is public, but much of it happens through industry collaborations, field implementation, and producer relationships, which are often necessarily behind closed doors or covered by NDAs. That can make the work less visible from the outside, even though it is central to what we are doing.
This is one reason I think treating our intervention too simplistically, as merely putting stunners on farms, can lead to exactly the kind of disillusionment and backlash you describe.
Third, I feel some of the issues you describe are not limited to tech interventions. Many ambitious attempts to change existing systems seem to have an early phase where progress looks relatively fast, followed by a harder phase where problems become more complex, uncertainty can increase, coordination costs become bigger, and resistance from affected stakeholders becomes stronger. That does not necessarily mean the intervention – tech or otherwise – was a mistake; it can be a natural progression of an ambitious project.
Thanks again for writing this! It’s an important reminder that welfare tech can be genuinely promising and, in some cases, extremely impactful, but it should not be treated as a silver bullet.
How impactful would it be to copies of @Garrison's new book Obsolete to elected officials who belong to its political target audience (Dems, particularly left ones) and might not have been responsive to traditional x-risk-centric comms (e.g. IABIED)?