This probably had my favorite "humor style" of this day's posts. Particularly liked the family planning mention.

A marginal $500,000 should go to:

The backfire effects of general alignment work early on in AI safety may have outweighed the benefits; I worry that the same could be true for animal-specific alignment.

If I really believe that, I should probably want to avoid money going into animal-specific alignment at this stage, while extra 500,000$ to general alignment, while not necessarily positive, is less likely to cause major backfire events?

Solving WAS intuitively seems too niche for people to deliberately change their mind on that, but I could be wrong. After all, the Bible says that the Lion will lie down with the lamb and eat straw like the ox, so it could be that human preferences tend to come back to the idea that animal suffering can be bad even when it doesn't depend on human actions.

I hesitated on how to frame the deep ecology thing, because I think it's entirely possible that it ends up locked in. I think my thought was something like the following. If AGI gets the values of its builders and then never modifies it, in the current race, it's unlikely that AGI would lock in deep ecology values: these don't seem massive in Chinese labs (could be wrong), and people in AI labs in the West are not hardcore ecologists, for the most part, because of political divisions.^[1]

I do agree that AI systems could populate other worlds with animals for other reasons. Logically, we can't cover all of the reasons why systems that we don't know anything about would do something. The same applies to future humans.

(More broadly, I deliberately under-hedged all of the above. I don't think we have any action-guidance on AGIxAnimals)

1. ^{^}

   Maybe it's good that environmentalists hate AI so much, because it ensures that people in labs are less likely to be friendly to pro-ecology views?

Agree that these are important and unresolved crucial considerations. 

I guess a "meta" consideration here is to what extent things that hold in our world hold in a "human-friendly" post-AGI world. I'm pessimistic on resolving this, because given that there is absolutely no track record, we should be very uncertain on our answer to that question. We'll have to wait and see (if we can see), or just have different factions taking different bets: there could be alt protein groups focusing on preventing further bans, and alt protein group building their ToC on the assumption that bans will not matter in a post-AGI world.

Fair pushback! 

For P1, I assumed that AGI going well for humans was basically even ASI going well for humans (just: "it happens we're in a good scenario for the fleshy humans"). I don't know if ASI is much less likely to go right than AGI - something as capable as AGI could already very easily be misaligned and I'm not sure that scales with increase in capabilities.

For P2, I'm scared that we don't necessarily need the top of the distribution of ASI to do this. I could imagine non-AGI worlds where human-brain-driven technological progress gets us there, though it seems very complicated resource-wise at this stage. 

I agree that these two arguments together could undermine my vague "one order of magnitude difference" claim, but I'm not sure how much I believe them. I do come down to believing that most of my considerations will face counter-considerations which I am currently unaware of.

This is definitely a hard debate to disentangle, because I would personally reject the question of alignment as a crux. For now, I strongly believe that the total welfare of animals has been entirely uncorrelated with our moral intentions toward animals. Total welfare has mostly changed because of land use, due to human interests. 

I agree that in AGI-transformed futures that go well for humans, human desires may start playing a larger role. However, I expect that whether we mean well for animals (or don't care much about them) will not be cleanly correlated with outcomes for them. 

There are worlds where we mean well for a large part of animals, stop intentionally killing them, and help certain wild animals. But that world could very well end up having a large population of animals living bad lives.

On the other hand, out of apathy and even negative feeling toward wild animals, we may decide to limit their spread and use resources in a way that optimizes for human flourishing, over animal abundance. That world could end up being much better for animal welfare.

Maybe some extreme scenarios tip the scales, for example if we bred incredibly happy genetically modified animals due to positive feelings toward them. But I'm not confident on putting any weight on such utilitarian-leaning scenarios when assessing post-AGI futures. Because part of the reason human moral intentions are not correlated with total animal welfare is that humans are not scope-sensitive utilitarians.

(Copied from my Symposium position statement)

If I accept conventional assumptions in EA Animal welfare^[1], AGI will be negative for animals in expectation. On the other hand, AGI being good for humans makes it worse for animals in expectation. However, both rogue AGI and human-friendly AGI seem positive for animals in most scenarios: it just happens that the "bad" scenarios seem much worse than the "good" scenario.

Why is that? AGI, whether rogue or human-aligned, may not decide to keep other planets free of biological animals (though it seems like a bigger risk for human-aligned AGI). And EA Animal Welfare advocates generally believe that the likelihood that wild animal welfare is negative makes such spreading of biological animals too risky.

A small chance of this decision being made outweighs the positives. This seems very unlikely with rogue AGI (0.1%, perhaps much less), but it could still dominate the scales in my view. An AGI that is more human-friendly seems at least one order of magnitude more likely to terraform other planets.^[2]

That said, this doesn't flip the sign of AI safety work. This judgment is lightly held; digital minds (human-like or animal-like) are a larger portion of welfare patients in expectation; and I have no idea of what the counterfactuals are. Thus, I don't treat this as an action-guiding beliefs.

To caveat, I think terraforming is still relatively unlikely in human-friendly scenarios because biodiversity becomes less instrumentally valuable post-AGI, so memes that would favor the existence of wild animal populations would lose in popularity. Even in human lock-in scenarios, the values that control AGI won't favor deep ecology.

How about farmed animals? Even in precision Livestock Farming's best and worst cases, suffering in factory farms shifts by a few orders of magnitude at most.^[3] AGI makes the end of factory farming through developing alternatives more likely, though I'm more convinced by "biological food systems become unnecessary or unrecognizable" than "clean meat wins". In the vast majority of scenarios, wild animals would be the most numerous moral patients.^[4]

However, again, alien counterfactuals probably messes all of this up. If biological beings from other planets can colonize planets in our sphere of influence, then I have to put myself at 0%.

1. ^{^}

   Farmed animal welfare is negative, wild animal welfare is negative, "good" and "bad" relate to expected total welfare

2. ^{^}

   Though what that looks like is still underdefined.

3. ^{^}

   However, precision livestock farming offers massive near-term risks and opportunities for farmed animals, and interest in this area appears justified.

^{^}

Human-friendly AGI could decide to only keep animals under human control, but that would probably not lead to massive animal populations.

Show all footnotes

My position statement (20% disagree with the statement "If AGI goes well for humans, it'll go well for animals")^[1]

If I accept conventional assumptions in EA Animal welfare^[2], AGI will be negative for animals in expectation. On the other hand, AGI being good for humans makes it worse for animals in expectation. However, both rogue AGI and human-friendly AGI seem positive for animals in most scenarios: it just happens that the "bad" scenarios seem much worse than the "good" scenario.

Why is that? AGI, whether rogue or human-aligned, may not decide to keep other planets free of biological animals (though it seems like a bigger risk for human-aligned AGI). And EA Animal Welfare advocates generally believe that the likelihood that wild animal welfare is negative makes such spreading of biological animals too risky.

A small chance of this decision being made outweighs the positives. This seems very unlikely with rogue AGI (0.1%, perhaps much less), but it could still dominate the scales in my view. An AGI that is more human-friendly seems at least one order of magnitude more likely to terraform other planets.^[3]

That said, this doesn't flip the sign of AI safety work. This judgment is lightly held; digital minds (human-like or animal-like) are a larger portion of welfare patients in expectation; and I have no idea of what the counterfactuals are. Thus, I don't treat this as an action-guiding beliefs.

To caveat, I think terraforming is still relatively unlikely in human-friendly scenarios because biodiversity becomes less instrumentally valuable post-AGI, so memes that would favor the existence of wild animal populations would lose in popularity. Even in human lock-in scenarios, the values that control AGI won't favor deep ecology.

How about farmed animals? Even in precision Livestock Farming's best and worst cases, suffering in factory farms shifts by a few orders of magnitude at most.^[4] AGI makes the end of factory farming through developing alternatives more likely, though I'm more convinced by "biological food systems become unnecessary or unrecognizable" than "clean meat wins". In the vast majority of scenarios, wild animals would be the most numerous moral patients.^[5]

1. ^{^}

   Probably 0% on reflection because aliens could count as animals, but it's less indicative

2. ^{^}

   Farmed animal welfare is negative, wild animal welfare is negative, "good" and "bad" relate to expected total welfare

3. ^{^}

   Though what that looks like is still underdefined.

^{^}

However, precision livestock farming offers massive near-term risks and opportunities for farmed animals, and interest in this area appears justified.

^{^}

Human-friendly AGI could decide to only keep animals under human control, but that would probably not lead to massive animal populations.

Show all footnotes

Far from a shovel-ready proposal, but you may appreciate this post from three years ago: Corporate campaigns work: a key learning for AI Safety