TLDR: I've updated towards pausing further AI development indefinitely.
When Scott Alexander proposed regulating AI like clinical drugs, many (including myself) balked at this given the sclerosis of FDA or related bodies. Yet HF updates me towards treating frontier AIs as nuclear. There onerous regulation is likely good.
In general, most bureaucratised regulatory regimes are bad. In some cases, like nuclear technology or ensuring planes are safe to fly on, they're welfare improving. It's clear that AI belongs in the latter category.
Also trace inversion is a thing, so you can get open source weights to be roughly similar to that of Fable/Mythos. The slowdown camp were right. At bare minimum, all labs should pause training, further development, and releases indefinitely until we figure out how to align AIs and regulate them (and their use by humans).
I reckon the current capabilities we have now are sufficient for accelerating progress towards curing cancers etc. So my balance has shifted towards minimising the existential risks now.
Pause AI advocates are correct. If governments could coordinate internationally to achieve such (big if), I'd support it. I used to be highly sceptical of doomer arguments, yet Hugging Face is almost a textbook LW scenario and no one knows how to spot or prevent such scheming. Sandboxing, guardrails, constitutions etc. don't work.
As someone who hires people (I am hiring right now, see bottom of the post), I often get emails asking whether someone should apply for a role I've listed.
If you're considering contacting a hiring manager to ask if you're a good fit for the role, you should just apply.
Why?
Notable exceptions, where you should email:
To be clear, the urge to send an email to check if you should apply is totally normal (I have done it before), and anyone who has done this has nothing to worry about. However, I would strongly recommend applying despite your reservations. You can just do things!
Also, if you want to apply to become Access to Medicines Initiative's Head of Monitoring, Evaluation, and Data, JOB DESCRIPTION AND APPLICATION HERE: https://docs.google.com/document/d/1nqOz2a5bTQs5W361G5gpJ3brLEf6xuG0
What's a good heuristic for knowing if you're being self-sacrificial to the point of negative returns for the thing you're being self-sacrificial for? I'm interested in things people have actually used and found to work in practice, either for themselves or others.
There's an old Facebook group, New EA hub search and planning, that might be worth reviving in light of the incresed amount of funding coming into the space.
Even with more availability of funding, many talented folks won't get funded or will only recieve small amounts. Establishing new hubs that provide an easier entry-way to being surrounded by community seems like potentially quite high value. After all, it takes a village to raise an EA and being surrounded by others is an accelerator to impact.
Hi Vasco,
Thanks for the question and sorry for the slow reply.
The overall process was to combine two estimates of THL's cost-effectiveness through weighting them in a Guesstimate model, along with some assumptions about the speed-up time of the intervention and the marginal cost-effectiveness relative to the past average cost-effectiveness.
The more detailed steps were:
1. Gather two estimates: THL review by ACE from 2025, wherein cage-free work has an estimated 2 to 44 hens affected per dollar (best guess of 11); THL self-published report from 2025, which reports 2 hens freed per dollar spent between 2015 and 2024.
2. Make assumptions: The THL self-published report was conservative in that it didn't include a counterfactual speed up assumption. So, we assumed that the interventions THL carried out each affected 6 to 15x as many hens than reported on average through speeding up the timelines to hens becoming cage-free. Then, since the marginal cost-effectiveness in 2026 is probably much less than the average cost-effectiveness from 2015-2024, we discounted the resulting THL self-reported estimate by 70%. This is a rough guess.
3. In a Guesstimate model, I combined the two estimates (giving equal weight to each) to get an overall estimate of THL's cost-effectiveness on the margin (caveat below). (The ACE estimate was encoded as a lognormal distribution between 2 and 44 hens/$, which had a mean of 14 which is slightly better than their best-guess, which could be a median or mean, I'm not sure). This produced an estimate of 10 (3.6 to 29) hens/$. The method isn't perfect, since these estimates might not be independent so the variance could be artificially reduced, but for our initial version, we felt it was a good enough estimate (other factors matter more to the CCF).
Upon looking back, I realize that I made a typo in the methodology -- it should be 3.6 to 29 chickens per dollar spent, not 3.6 to 22. So thanks for pointing my attention to this! I've now fixed it.
On the more important question, I agree that the marginal cost-effectiveness is definitely the important question, and I'd like to have this. However, the best estimates we could find for the first version of the CCF were for average cost-effectiveness of past spending. We plan to improve upon these estimates in the future, such as through considering more evidence. If you know of any good marginal cost-effectiveness estimates, or other cost-effectiveness estimates, please do link them below!
Hope this is helpful! Also, I'll look into how to change the settings so you can copy text, I'm not sure why it's like that
Thank you for writing this, Tom and Rocky. I think this is an important caution, and I appreciate the way you are framing it.
As someone directly involved in a welfare-tech-style intervention, I actually agree with much of your core argument. I also strongly believe that good tech interventions can be extremely impactful, partly for some of the reasons you outline: they can sometimes bypass certain barriers that exist for other welfare work, and they can make it possible to help very large numbers of animals relatively quickly. But I also agree that, when looked at from the outside, they can be falsely perceived as silver bullets, so I want to share three insights from SWP about electrical stunning for shrimp.
I’ve also posted a more direct response to “Animal Welfare Has an Evidence Problem,” which you refer to in the post, to clarify some of these points.
First, at SWP, our work on shrimp stunning has been much more complex than simply deploying stunners at scale. The Humane Slaughter Initiative does involve getting electrical stunning equipment used on farms, but in practice, we are doing much more than that: coordinating and funding welfare research and R&D, collecting field data, working with scientists, developing and refining slaughter protocols, iterating on the technology with existing equipment providers, negotiating with industry, engaging corporate stakeholders, building and maintaining producer relationships, training staff, educating farmers… Much of this work is not visible from the surface, but it’s central to making the intervention possible. The tech matters, but it does not operate in a vacuum.
Second, on shrimp stunning specifically, I think you captured something important. SWP entered this space aware that there were uncertainties, and we’ve been proactively trying to reduce them. Some of that work is public, but much of it happens through industry collaborations, field implementation, and producer relationships, which are often necessarily behind closed doors or covered by NDAs. That can make the work less visible from the outside, even though it is central to what we are doing.
This is one reason I think treating our intervention too simplistically, as merely putting stunners on farms, can lead to exactly the kind of disillusionment and backlash you describe.
Third, I feel some of the issues you describe are not limited to tech interventions. Many ambitious attempts to change existing systems seem to have an early phase where progress looks relatively fast, followed by a harder phase where problems become more complex, uncertainty can increase, coordination costs become bigger, and resistance from affected stakeholders becomes stronger. That does not necessarily mean the intervention – tech or otherwise – was a mistake; it can be a natural progression of an ambitious project.
Thanks again for writing this! It’s an important reminder that welfare tech can be genuinely promising and, in some cases, extremely impactful, but it should not be treated as a silver bullet.
I think that this diagnosis is basically on target; it points to something that seems relatively under-resourced, despite some focus by think tanks, and something that the major AIxBio safety groups are not as focused on. I also agree that we don't know if AGI will subsume this, and that it's plausible but uncertain if other bottlenecks matter more, but that's an uncertainty we can't resolve without simply waiting for the outcomes, and so this seems very high value in expectation.
I'm less certain about the object level questions, and don't have strong intuitions - so I think that conditional on not hearing from someone more informed about this that there are additional questions or concerns, or literature you should look at, the best way to figure out whether this is needed, and what the risk is, is to start the work - good luck, and I'd be happy to chat about this more!
TLDR: I've updated towards pausing further AI development indefinitely.
When Scott Alexander proposed regulating AI like clinical drugs, many (including myself) balked at this given the sclerosis of FDA or related bodies. Yet HF updates me towards treating frontier AIs as nuclear. There onerous regulation is likely good.
In general, most bureaucratised regulatory regimes are bad. In some cases, like nuclear technology or ensuring planes are safe to fly on, they're welfare improving. It's clear that AI belongs in the latter category.
Also trace inversion is a thing, so you can get open source weights to be roughly similar to that of Fable/Mythos. The slowdown camp were right. At bare minimum, all labs should pause training, further development, and releases indefinitely until we figure out how to align AIs and regulate them (and their use by humans).
I reckon the current capabilities we have now are sufficient for accelerating progress towards curing cancers etc. So my balance has shifted towards minimising the existential risks now.
Pause AI advocates are correct. If governments could coordinate internationally to achieve such (big if), I'd support it. I used to be highly sceptical of doomer arguments, yet Hugging Face is almost a textbook LW scenario and no one knows how to spot or prevent such scheming. Sandboxing, guardrails, constitutions etc. don't work.
The illusion is irresistible. Behind every face there is a self. We see the signal of consciousness in a gleaming eye and imagine some ethereal space beneath the vault of the skull lit by shifting patterns of feeling and thought, charged with intention. An essence. But what do we find in that space behind the face, when we look? [Nothing but] flesh and blood and bone and brain. I know, I've seen. You look down into an open head, watching the brain pulsate, wat...
Thanks Vasco, much appreciated. As it happens I recently got to attend a presentation by Kammerer and I was quite impressed. There are obviously many different ways to expand and elaborate on illusionism as an approach to consciousness, and I thought his ideas just seemed unusually detailed and thoughtful. So I look forward to reading his new book (or at least that chapter).
The Saturation View holds that a world of varied joys is better than a world of equal but identical joys. Elliott Thornley objected that no one benefits from this variety, and Will MacAskill answered that...
Thanks for putting your thoughts in detail about all this! For me it comes down to the question: Is there any value in something that doesn't exist? I think there isn't. A variety which is between lives, which is experienced by no one, doesn't exist. On the other hand when I calculate welfare and I sum them up, I am doing an operation on the welfare but I am only playing with things that really exist. I'm not adding anything fictional. I think our language in many ways promote caring for fictional not existing things. imo this is a philosophical mistake. We should only care about reality.
This is crossposted from my Substack
TL;DR:
-Most people cannot reduce jealousy much or at all
- It fundamentally causes way more drama because of strong emotions, jealousy, no default norms to fall back to, and there being exponentially more surface area for conflict
- For a small minority of people, it makes them happier, and those are the people who tend to stick with it and write the books on i...
This post summarizes a new preprint on alternative proteins from the Humane and Sustainable Food Lab: New alternatives, same orders. We investigated 19...
I think the key thing to test is when plant based substitutes are significantly cheaper than animal products. The two examples I know of now are margarine and egg replacer. The former got big market penetration (and I think some actual substitution) and the latter didn't. As I've argued elsewhere, we could get plant based burgers to cost parity or better if we built a bigger factory. Then I think there would be some substitution for price sensitive consumers.
This is a cross post from my blog. It's meant a general introduction to effective charity, and it's my own rendition of Famine, Affluence, and Morality.
You’re going on a gentle stroll through the woods when you stumble upon a child drowning in a pond. You can easily wade into the pond and save the child with no harm to yours...
Humans didn't evolve to understand communication at a distance. It's a very recent phenomenon (started in the 1830s, about the last 200 years). Humans are also very capable of deception. So how can anyone who is told that a child across the world is drowning trust that:
Giving locally gives me a chance to verify all of that.
I spent 45 minutes on a quick take and then accidentally deleted it.[1] So here’s the short version:
I’m worried about things like plan A because it kind of looks like they might give an autocracy power forever. If we co-develop AGI with China and then allow China to do...
There's an old Facebook group, New EA hub search and planning, that might be worth reviving in light of the incresed amount of funding coming into the space.
Even with more availability of funding, many talented folks won't get funded or will only recieve small amounts. Establishing new hubs that provide an easier entry-way to being surrounded by community seems like potentially quite high value. After all, it takes a village to raise an EA and being surrounded by others is an accelerator to impact.
How impactful would it be to copies of @Garrison's new book Obsolete to elected officials who belong to its political target audience (Dems, particularly left ones) and might not have been responsive to traditional x-risk-centric comms (e.g. IABIED)?