I'm earning to give as a Quant Researcher at the Quantic group at Walleye Capital, a hedge fund. In my free time, I enjoy reading, discussing moral philosophy, and dancing bachata and salsa.
On animal consciousness, the fact that smart people like Eliezer and possibly Jeff Kauffman believe chickens aren't sentient is maybe enough to be ~99% sure chickens are conscious rather than 100%. Speaking for myself (and I expect most EAs would agree), the prior on large animals being conscious due to them having similar brain structures to ours seems really strong. Eliezer seems to disagree due to being extremely confident in a higher order self-modeling theory of consciousness. I find his theory dubious. Even if it were correct, I don't think his theory necessarily implies what he thinks it implies. And even if it really implied chickens are unlikely to be conscious, that wouldn't substantially alter the case for animal welfare.[1]
On the epistemic front, I agree that it's worth being explicit about whatever level of uncertainty we have. But for those who think animals are unlikely to be conscious, I rarely see the same level of humility or prudence on their side -- if there's only a 10% chance chickens are conscious, then EA's cage-free campaign work should be lauded as a valuable long shot, and animal products are still enormously bad in expectation, so they seem worth avoiding.
Edit: On smaller animals like fish, shrimp, and insects, EAs do exhibit more thoughtful discussion about consciousness probability. And Rethink Priorities' Welfare Ranges are conditioned on explicit consciousness probability even for larger animals like chickens.
Bentham's Bulldog has written a few posts against Eliezer's view of animal consciousness. It's a polemic style, so I won't pretend the author exhibits intellectual humility either, but I agree with the author's arguments.
That's a cool thought and I certainly hope you're right. I frankly don't know how to think about it. When I read about the teletransportation paradox, the fact that there's a copy of me doesn't relieve my anxiety much about my body being destroyed. But when I think about the fact that my brain's constituent atoms are replaced over time, I do feel a sense of continuity, but maybe that's an illusion? It's hard to know.
Hi, thanks Vasco! The Tomasik post is interesting, but he assumes a theory of consciousness which rules out lots of theories many people (including myself) find plausible like psychophysical laws, type B physicalism, and more.
On animals, my main view is that the vast majority of the effect of any near-term intervention is on animals rather than humans. I've come around to the idea that there are many more cost-effective animal interventions than THL, having donated to SWP and the Insect Institute. GFI is a strong choice as well. I agree soil invertebrates matter in expectation and that the animal welfare community should put some level of effort into incorporating them into cost-effectiveness estimates.
Thanks Tristan, that is a cool framing! And yes I agree that if it's true, it nicely captures some of the lessons from my post in a psychologically easier way.
There are theoretical assumptions underlying this, like EDT over CDT as you point out. It also seems like you're assuming a large multiverse, which SIA likes a lot more than SSA. Do you endorse SIA as well?
Thanks for engaging. I'd push back strongly on these points. In any case, my post is just saying the AGI sim hypothesis is plausible enough to merit discussion of its implications, so the bar for my post is just that it's not silly enough to dismiss.
You frame Carroll's argument as "enough to end all discussion", but not even Carroll seems to believe that--he demonstrates thoughtful uncertainty and says "likely there are standard responses", and indeed there are. The standard response is that deeper sims don't need to be incapable of running sims, they can just get shut off before they do. Bostrom anticipated this argument in his original paper, writing that nested sims are a reason why we'd expect ours to be "terminated when we are about to become posthuman". This scenario is one of the main implications of the AGI sim hypothesis I discuss in my post above. There are other responses to Carroll's argument too.
On the IVF analogy, the simulation argument says that if we get evidence that most observers with experiences like ours are simulated, unless we have evidence to the contrary, we should think we are also simulated. The idea that our descendants or AGI would run sims of AGI origins, which would create lots of observers with experiences like ours, is evidence that most observers with experiences like ours are simulated. In contrast, planning to use IVF is not evidence that you yourself are an IVF baby.
On the ethical and opportunity cost arguments, your arguments are evidence for one of my post's main scenarios -- the simulation being run by a misaligned AGI -- rather than evidence against the simulation hypothesis altogether. They have some force, which is why I personally think the misaligned AGI simulator is the more likely one, but I don't think they push that hard. Humans tolerate an immense amount of preventable suffering today (global poverty, factory farming, wild animal suffering). I wouldn't be that surprised if a "human-aligned" AGI would run simulations like ours (if we are indeed a sim).
On theism, I agree there are some resemblances, specifically the claim that the universe was made for a reason. But the evidence for the hypotheses differs tremendously. The simulation hypothesis is built from an expectation of future sim-running capabilities we can extrapolate from known physics. If we learned that there was a way to create pocket universes whose inhabitants worshipped us, that rightly should increase our credence in theism. That's the situation I think we're in wrt the simulation hypothesis. There's also a lot of specific things the AGI sim hypothesis predicts that theism doesn't: us being in the AI transition, an impersonal simulator rather than a personal god, etc.
This is good advice. Additionally (and perhaps more controversially), if you’re a person who considers digital minds and/or animal welfare close in priority to your top AI cause area, it might be worth considering moving some funding to those cause areas, since they’ll still be proportionally more neglected relative to AI safety.
These cause areas also have their own political opportunities (like Save Our Bacon) which disproportionally benefit from individual donors.
Trump will be President until 2029. He will not sign AI safety legislation passed on Democratic party lines. Overriding his veto (or an impeachment conviction) would require 1/3 of GOP senators to cross party lines, which will never happen. Making AI safety partisan would ensure we have a 0% chance of federal AI safety legislation until at least 2029. Trump’s “paced the frontier” before by limiting Fable access when it was in his interest, and he may do it again if he perceives it to be in his interest. I understand your frustration with Trump, but there is nothing to gain and a lot to lose from partisanship here.
It’s completely fine to criticize him, but I think it’s wrong to shame him. To your point, Austin’s critics have their sincere convictions, and I  would find it equally wrong for Austin to shame them.
I think it’s virtually always a bad idea to shame people for standing up for their convictions, even if you disagree with those convictions. Shaming people alienates them and entrenches them in their positions. In contrast, civil debate improves community epistemics and is far more likely to change the person’s mind.Â
For an example, due to rejecting person-affecting views (PAV), I don’t think there’s much of a moral difference between the work of population-reducing EA charities and, well, killing people. A few years ago, I publicly critiqued these charities for this reason, and I ended up having some civil conversations with the charity founders, who endorse PAV and sincerely care about helping people. We’re on good terms and are all still members of the EA community. I don’t think anything positive would have come out of me shaming these founders, and I’m glad I didn’t.
Similarly, my presumption is that Austin is hiring Caroline because he thinks she’s very unlikely to reoffend and he believes empathy to her as an individual is more important than the reputational damage that will result from this. I think the object-level debate on these points in the comments section is great, but shaming him for holding this perspective would be neither kind nor helpful.
On animal consciousness, the fact that smart people like Eliezer and possibly Jeff Kauffman believe chickens aren't sentient is maybe enough to be ~99% sure chickens are conscious rather than 100%. Speaking for myself (and I expect most EAs would agree), the prior on large animals being conscious due to them having similar brain structures to ours seems really strong. Eliezer seems to disagree due to being extremely confident in a higher order self-modeling theory of consciousness. I find his theory dubious. Even if it were correct, I don't think his theory necessarily implies what he thinks it implies. And even if it really implied chickens are unlikely to be conscious, that wouldn't substantially alter the case for animal welfare.[1]
On the epistemic front, I agree that it's worth being explicit about whatever level of uncertainty we have. But for those who think animals are unlikely to be conscious, I rarely see the same level of humility or prudence on their side -- if there's only a 10% chance chickens are conscious, then EA's cage-free campaign work should be lauded as a valuable long shot, and animal products are still enormously bad in expectation, so they seem worth avoiding.
Edit: On smaller animals like fish, shrimp, and insects, EAs do exhibit more thoughtful discussion about consciousness probability. And Rethink Priorities' Welfare Ranges are conditioned on explicit consciousness probability even for larger animals like chickens.
Bentham's Bulldog has written a few posts against Eliezer's view of animal consciousness. It's a polemic style, so I won't pretend the author exhibits intellectual humility either, but I agree with the author's arguments.
That's a cool thought and I certainly hope you're right. I frankly don't know how to think about it. When I read about the teletransportation paradox, the fact that there's a copy of me doesn't relieve my anxiety much about my body being destroyed. But when I think about the fact that my brain's constituent atoms are replaced over time, I do feel a sense of continuity, but maybe that's an illusion? It's hard to know.
Hi, thanks Vasco! The Tomasik post is interesting, but he assumes a theory of consciousness which rules out lots of theories many people (including myself) find plausible like psychophysical laws, type B physicalism, and more.
On animals, my main view is that the vast majority of the effect of any near-term intervention is on animals rather than humans. I've come around to the idea that there are many more cost-effective animal interventions than THL, having donated to SWP and the Insect Institute. GFI is a strong choice as well. I agree soil invertebrates matter in expectation and that the animal welfare community should put some level of effort into incorporating them into cost-effectiveness estimates.
Thanks Tristan, that is a cool framing! And yes I agree that if it's true, it nicely captures some of the lessons from my post in a psychologically easier way.
There are theoretical assumptions underlying this, like EDT over CDT as you point out. It also seems like you're assuming a large multiverse, which SIA likes a lot more than SSA. Do you endorse SIA as well?
Hi Ben, thanks for the thoughtful comment. I'll DM to discuss further.
Thanks for engaging. I'd push back strongly on these points. In any case, my post is just saying the AGI sim hypothesis is plausible enough to merit discussion of its implications, so the bar for my post is just that it's not silly enough to dismiss.
You frame Carroll's argument as "enough to end all discussion", but not even Carroll seems to believe that--he demonstrates thoughtful uncertainty and says "likely there are standard responses", and indeed there are. The standard response is that deeper sims don't need to be incapable of running sims, they can just get shut off before they do. Bostrom anticipated this argument in his original paper, writing that nested sims are a reason why we'd expect ours to be "terminated when we are about to become posthuman". This scenario is one of the main implications of the AGI sim hypothesis I discuss in my post above. There are other responses to Carroll's argument too.
On the IVF analogy, the simulation argument says that if we get evidence that most observers with experiences like ours are simulated, unless we have evidence to the contrary, we should think we are also simulated. The idea that our descendants or AGI would run sims of AGI origins, which would create lots of observers with experiences like ours, is evidence that most observers with experiences like ours are simulated. In contrast, planning to use IVF is not evidence that you yourself are an IVF baby.
On the ethical and opportunity cost arguments, your arguments are evidence for one of my post's main scenarios -- the simulation being run by a misaligned AGI -- rather than evidence against the simulation hypothesis altogether. They have some force, which is why I personally think the misaligned AGI simulator is the more likely one, but I don't think they push that hard. Humans tolerate an immense amount of preventable suffering today (global poverty, factory farming, wild animal suffering). I wouldn't be that surprised if a "human-aligned" AGI would run simulations like ours (if we are indeed a sim).
On theism, I agree there are some resemblances, specifically the claim that the universe was made for a reason. But the evidence for the hypotheses differs tremendously. The simulation hypothesis is built from an expectation of future sim-running capabilities we can extrapolate from known physics. If we learned that there was a way to create pocket universes whose inhabitants worshipped us, that rightly should increase our credence in theism. That's the situation I think we're in wrt the simulation hypothesis. There's also a lot of specific things the AGI sim hypothesis predicts that theism doesn't: us being in the AI transition, an impersonal simulator rather than a personal god, etc.
This is good advice. Additionally (and perhaps more controversially), if you’re a person who considers digital minds and/or animal welfare close in priority to your top AI cause area, it might be worth considering moving some funding to those cause areas, since they’ll still be proportionally more neglected relative to AI safety.
These cause areas also have their own political opportunities (like Save Our Bacon) which disproportionally benefit from individual donors.
For reasons given here, I don’t think this case is deserving of shaming the way murder would be.
Trump will be President until 2029. He will not sign AI safety legislation passed on Democratic party lines. Overriding his veto (or an impeachment conviction) would require 1/3 of GOP senators to cross party lines, which will never happen. Making AI safety partisan would ensure we have a 0% chance of federal AI safety legislation until at least 2029. Trump’s “paced the frontier” before by limiting Fable access when it was in his interest, and he may do it again if he perceives it to be in his interest. I understand your frustration with Trump, but there is nothing to gain and a lot to lose from partisanship here.
It’s completely fine to criticize him, but I think it’s wrong to shame him. To your point, Austin’s critics have their sincere convictions, and I  would find it equally wrong for Austin to shame them.
I think it’s virtually always a bad idea to shame people for standing up for their convictions, even if you disagree with those convictions. Shaming people alienates them and entrenches them in their positions. In contrast, civil debate improves community epistemics and is far more likely to change the person’s mind.Â
For an example, due to rejecting person-affecting views (PAV), I don’t think there’s much of a moral difference between the work of population-reducing EA charities and, well, killing people. A few years ago, I publicly critiqued these charities for this reason, and I ended up having some civil conversations with the charity founders, who endorse PAV and sincerely care about helping people. We’re on good terms and are all still members of the EA community. I don’t think anything positive would have come out of me shaming these founders, and I’m glad I didn’t.
Similarly, my presumption is that Austin is hiring Caroline because he thinks she’s very unlikely to reoffend and he believes empathy to her as an individual is more important than the reputational damage that will result from this. I think the object-level debate on these points in the comments section is great, but shaming him for holding this perspective would be neither kind nor helpful.