Hi, I'm Luke Muehlhauser. AMA about Open Philanthropy's new report on consciousness and moral patienthood

We have!

Prior to beginning this investigation, we were on the fence about whether we should recommend grants related to fish welfare. As a result of initial findings from this report (long before publication), we decided to go ahead with investigating fish welfare grants:

A third key output from this investigation is that we decided (months ago) to begin investigating possible grants targeting fish welfare. This is largely due to my failure to find any compelling reason to “draw lines” in phylogeny (see previous section). As such, I could find little justification for suggesting that there is a knowably large difference between the probability of chicken consciousness and the probability of fish consciousness. Furthermore, humans harm and kill many, many more fishes than chickens, and some fish welfare interventions appear to be relatively cheap.

Since then, Lewis has recommended >$1 million in grants related to fish welfare: 1, 2, 3.

That said, keep in mind this caveat from the report:

Of course, this decision to investigate possible fish welfare grants could later be shown to have been unwise, even if the Open Philanthropy Project assumes my personal probabilities of consciousness in different taxa, and even if those probabilities don’t change. For example, I have yet to examine other potential criteria for moral patienthood besides consciousness, and I have not yet examined the question of moral weight (see above). The question of moral weight, especially, could eventually undermine the case for fish welfare grants, even if the case for chicken welfare grants remains robust. Nevertheless, and consistent with our strategies of hits-based giving and worldview diversification, we decided to seek opportunities to benefit fishes in case they should be considered moral patients with non-negligible weight.

Re: (1), I'll focus on the common fruit fly for concreteness. Before I began this investigation, I probably would've given fruit fly consciousness very low probability (perhaps <5%), and virtually all of that probability mass would've been coming from a perspective of "I really don't see how fruit flies could be conscious, but smart people who have studied the issue far more than I have seem to think it's plausible, so I guess I should also think it's at least a little plausible." Now, having studied consciousness a fair bit, I have more specific ideas about how it might turn out to be the case that fruit flies are conscious, even if I think they're relatively low probabilitiy, and of course I retain some degree of "and maybe my ideas about consciousness are wrong, and fruit flies are conscious via mechanisms that I don't currently find at all plausible." As reported in section 4.2, my current probability that fruit flies are conscious (as loosely defined in section 2.3.1 is 10%.

Re: (2). This question raises issues related to Pascal's Mugging. I don't pretend to have a solution, but some especially relevant discussions are Pascal's Muggle: Infinitesimal Priors and Strong Evidence, Why we can’t take expected value estimates literally (even when they’re unbiased), Modeling Extreme Model Uncertainty, and Sequence thinking vs. cluster thinking. As mentioned here, the Open Philanthropy Project remains eager to get more clarity on how we should think about making decisions under different kinds of uncertainty, and we expect to write more about this issue in the future.

In the meantime, the possibility of electron consciousness does not currently inform my actions.

Re: (3). You're right that Chalmers is "an interesting choice" for me to get feedback from. For example, Chalmers is a property dualist and panpsychist, which in some ways is about as different a position on consciousness from mine as you could find. On the other hand, Chalmers is (in my opinion) an unusually sharp thinker about consciousness, and also he has a deserved reputation for his ability to give critical feedback to others from the perspective of their own view. And, as hoped, Chalmers' feedback on an earlier draft of my report was very helpful, and I invested dozens of hours improving the draft in response to his feedback alone.

There were a few other consciousness researchers (whom I won't name) that I had hoped to get feedback from, but they weren't interested to give it. That's not surprising, since my report is so different from the type of work that consciousness researchers typically engage with.

My report makes so many claims (or at least, "guesses") that I have no doubt that if other consciousness experts gave extensive feedback on it, they would find plenty with which they disagree. In some cases, I know from their writing some specific things they would disagree with. But in many cases, I'm not sure where they would disagree, both because I haven't read all their works on consciousness, and because most consciosuness experts have only written about a tiny portion of the issues covered (at least briefly) in my report.

Re: (4). This question is beyond the scope of the intended purpose of this AMA, but I'll make a couple brief comments. It would take a lot of work for me to write a similar document that usefully complements Carl's, but I may do so one day. An old post of mine on this general topic is Scholarship: How to Do It Efficiently, but it's pretty narrow in scope.

Quick note: I agree with Benito that it's preferable to split different questions into separate comments, but in this case don't worry about doing so. I'll reply to all your questions as soon as I can, though I'm going to answer some other, quicker-to-answer questions first.

(Meta: It might be more helpful to submit individual questions as separate comments, so that people can up vote them separately and people's favourite questions (and associated answers) can rise to the top.)

I'm not sure what you mean by "objective definition" or "objectively correct answer," but I don't think I think of consciousness as being "objective" in your sense of the term.

The final question, for me, is "What should I care about?" I elaborate my "idealized" process for answering this question in section 6.1.2. Right now, my leading guess for what I'd conclude upon going through some approximation of that idealized process is that I'd care about beings with valenced conscious experience, albeit with different moral weights depending on a variety of other factors (early speculations in Appendix Z7).

But of course, I don't know quite what sense of "valenced conscious experience" I'd end up caring about upon undergoing my idealized process for making moral judgments, and the best I can do at this point is something like the definition by example (at least for the "consciousness" part) that I begin to elaborate in section 2.3.1.

Re: Type A physicalism, aka Type A materialism. As mentioned in section 2.3.2, I do think my current view is best thought of as "'type A materialism,' or perhaps toward the varieties of 'type Q' or 'type C' materialism that threaten to collapse into 'type A' materialism anyway…" (see the footnote after this phrase for explanations). One longer article that might help clarify how I think about "type A materialism" w.r.t. consciousness or other things is Mixed Reference: The Great Reductionist Project and its dependencies.

That said, I do think the "triviality" objection is a serious one (Ctrl+F the report for "triviality objection to functionalism"), and I haven't studied the issue enough to have a preferred answer for it, nor am I confident there will ever be a satisfying answer to it — at least, for the purposes of figuring out what I should care about. Brian wrote a helpful explainer on some of these issues: How to Interpret a Physical System as a Mind. I endorse many of the points he argues for there, though he and I end up with somewhat different intuitions about what we morally care about, as discussed in the notes from our conversation.

Among the PCIFs mentioned in section 3.2.2, I consider it to be one of the more interesting ones, principally because it may be indicative of certain kinds of "self-modeling," and the theoretical approaches to phenomenal consciousness I intuitively find most promising are those which involve certain kinds of self-modeling.

That said, I still give it very little weight, since it's very unclear what, exactly, different mirror-related behaviors imply about the self-modeling capacities of the animal, and how those relate to the self-modeling capacities that may be necessary for conscious experience. For example see the quotes and sources in footnote 109.

Unfortunately, the literature on consciousness is in even greater terminological disarray than many other fields, for understandable reasons: (1) to an unusual degree, we still don't know what we're talking about w.r.t. consciousness, and as a result, (2) consciousness studies is an extremely interdisciplinary "field," with contributions from scholars of nearly non-overlapping professional backgrounds. So, I'm not sure what a useful source on "useful terminology" would look like. Possibly a broad and shallow survey like my report or Weisberg (2014) is the best we can do at this stage.

I do suspect it’s possible for long-term meditation practitioners to have observations about the “internal” explananda of consciousness that non-practitioners (such as myself) can’t have. Susan Blackmore may be one such person. Of course, before taking any particular report seriously I’d want to check whether the reporting practitioner seems to be a reliable and clear reporter of their internal experience, whether their reports can be replicated by others in a certain way, etc.

As for determining the “true nature” of conscious experience: that’s more a matter of normal science, and especially of proposing cognitive algorithms that would instantiate the observed explananda of consciousness, and the meditation practitioners who are helpful for increasing our understanding of the explananda of consciousness might or might not happen to also be useful for doing the scientific work of coming up with strong explanations of those explananda.

These questions are beyond the intended scope of this AMA, but I’ll make a few quick remarks about the second question.

The calendar time spent is pretty uninformative, since large chunks of it passed by while I was working on other Open Phil projects that were (temporarily) more urgent, or waiting for feedback from someone, or for other reasons. As for total hours, my work logs suggest something like >800 hours on this report (by far my largest project thus far), though that includes lots of “getting up to speed” background reading on the various fields with which the report engages.

I'm not sure. My best guess would be something in the direction of the GDAK-inspired cognitive architecture discussed in section 6.2.4, but I don't yet have a clear picture of all the cognitive features that would need to be working together in a certain way for me to be even moderately confident the program is "conscious" in the sense defined in the report. I'm pretty sure none of the programs I described in the report are conscious (in the sense defined in the report).

For others' benefit, what I said in the report was:

I think I can make a weakly plausible case for (e.g.) Gazami crab consciousness, and I think I can make a weakly plausible case for chimpanzee non-consciousness.

By "weakly plausible" I meant that I think I can argue for a ~10% chance of Gazami crab consciousness, and separately for a ~10% chance of chimpanzee non-consciousness.

Such arguments would draw from considerations that are mentioned at least briefly somewhere in the report, but it would bundle them together in a certain way and elaborate certain points.

My argument for ~10% chance of chimpanzee non-consciousness would look something like an updated version of Macphail (1998), plus many of the considerations from Dennett (2017). Or, to elaborate that a bit: given the current state of evidence on animal cognition and behavior, and given what is achievable using relatively simple deep learning architectures (including deep reinforcement learning), it seems plausible (though far from guaranteed) that the vast majority of animal behaviors, including fairly sophisticated ones, are the product of fairly simple (unconscious) learning algorithms operating in environments with particular reward and punishment gradients, plus various biases in the learning algorithms "organized in advance of experience" via evolution. Furthermore, it seems plausible (though not likely, I would say) that phenomenal consciousness depends on a relatively sophisticated suite of reasoning and self-modeling capacities that humans possess and chimpanzees do not (and which may also explain why chimpanzees can't seem to learn human-like syntactically advanced language). I am pretty confident this conjunction of hypotheses isn't true, but I think something like this is "weakly plausible." There are other stories by which it could turn out that chimpanzees aren't conscious, but the story outlined above is (very loosely speaking) the "story" I find most plausible (among stories by which chimpanzees might not be conscious).

My case for a ~10% chance of Gazami crab consciousness would involve pulling together a variety of weak considerations in favor of the "weak plausibility" of Gazami crab consciousness. For example: (1) given the considerations from Appendix H, perhaps phenomenal consciousness can be realized by fairly simple cognitive algorithms, (2) even assuming fairly "sophisticated" cognition is required for consciousness (e.g. a certain kind of self-model), perhaps 100,000 neurons are sufficient for that, and (3) perhaps I'm confused about something fairly fundamental, and I should be deferring some probability mass to the apparently large number of consciousness scholars who are physicalist functionalists and yet think it's quite plausible that arthropods are conscious.

I'll use the terms "nociception" and "pain" as defined in Appendix D:

nociception is the encoding and processing of noxious stimuli, where a noxious stimulus is an actually or potentially body-damaging event (either external or internal, e.g. cutaneous or visceral)… Pain, in contrast to mere nociception, is an unpleasant conscious experience associated with actual or potential body damage (or akin to unpleasant experiences associated with noxious stimuli).

…Nociception can occur without pain, and pain can occur without nociception. Loeser & Treede (2008) provide examples: “after local anesthesia of the mandibular nerve for dental procedures, there is peripheral nociception without pain, whereas in a patient with thalamic pain [a kind of neuropathic pain resulting from stroke], there is pain without peripheral nociception.”

Often, it's assumed that if animals are conscious, then it's likely that they experience pain in conjunction with the types of nociceptive processing that, in humans, would be accompanied by conscious pain — or at least, this seems likely for the animals that are fairly similar to us in their neural architecture (e.g. mammals, or perhaps all vertebrates). And likewise, it seems that if we limit the nociception that occurs in their bodies, then this will also limit the conscious pain they experience if they are conscious at all.

Unfortunately, humans also experience pain without nociception, e.g. (as my report says) "neuropathic pain, and perhaps also… some cases of psychologically-created experiences of pain, e.g. when a subject is hallucinating or dreaming a painful experience." Assuming whichever animals are conscious are also capable of non-nociceptive pain, this means that conscious animals may be capable of suffering in ways that are difficult for us to detect.

Even still, I think we can study the mechanisms of nociception (and other triggers for conscious pain) independently of studying consciousness, and this could lead to interventions that (probably) address the usual causes of conscious pain even if we don't know whether a given animal is conscious or not.

However, I'm not sure this is quite what you meant to be asking; let me know if there was a somewhat different question you were hoping I'd answer.

It's a very difficult question. I wrote up some initial speculations on how I might address the "moral weight" question in Appendix Z7. Each of those "candidate dimensions of moral concern" can also be studied on their own, often in pretty similar ways to how "consciousness in general" can be studied.

I'm not sure whether the thing you're trying to say is compatible with what I'd say or not. The way I'd say it is this:

The 'weak illusionist' says that while many features of conscious experience can be 'explained away' as illusions, the 'something it's like'-ness of conscious experience is not (and perhaps cannot be) an illusion, and thus must be "explained" rather than "explained away." In contrast, the "strong illusionist" says that even the 'something it's like'-ness of conscious experience is an illusion.

But what might it be for the 'something it's like'-ness to be an illusion? Basically, that it seems to us that there is more to conscious experience than 'zero qualia', but in fact there are only zero qualia. E.g. it seems to us that there are 'diet qualia' that are more than 'zero qualia', but in fact 'diet qualia' have no distinctive features beyond 'zero qualia.'

Now in fact, I think there probably is more to 'zero qualia' than Frankish's "properties of experiences that dispose us to judge that experiences have introspectable qualitative properties that are intrinsic, ineffable, and subjective," but I don't think those additional properties will be difficult for the strong illusionst to adopt, and I don't think they'll vindicate a position according to which there is a distinctive 'diet qualia' option. Speaking of diet qualia vs. zero qualia is very rough: the true form of the answer will be classes of cognitive algorithms (on my view).

As mentioned in another comment, I'm perhaps most excited about the potential informativeness of (1) computational modeling of the sort I describe in section 6.2.4, (2) certain kinds of studies of human consciousness (Ctrl+F in the report for "perhaps the most promising path forward"), and (3) improvements to tools and techniques of human neuroscience that could help with (2) (Ctrl+F in the report for "we need fundamental breakthroughs").

But if I spent ~10 hours on each of the suggested research ideas from section 5, I'd probably have a better sense of each suggestion's cost and likely benefits, and my intuitions about prioritization would probably change.

What's the explanatory gap argument?

The report aims to be a "direct inquiry into moral status," but because it does so from an anti-realist perspective, a certain notion of idealized preferences comes into play. In other words: if you don't think "objective values" are "written into the fabric of the universe," then (according to one meta-ethical perspective) all that exists are particular creatures that value things, and facts about what those creatures would value if they had more time to think about their values and knew more true facts and so on. I won't make the case for this meta-ethical approach here, but I link some relevant sources in the report, in particular in footnote 239.

This is one reason I say at the top of the report that:

This report is unusually personal in nature, as it necessarily draws heavily from the empirical and moral intuitions of the investigator. Thus, the rest of this report does not necessarily reflect the intuitions and judgments of the Open Philanthropy Project in general. I explain my views in this report merely so they can serve as one input among many as the Open Philanthropy Project considers how to clarify its values and make its grantmaking choices.

And in fact, as I understand it, the people involved in making Open Phil grantmaking decisions about farm animal welfare do have substantial disagreements with my own moral intuitions, and are not making their grantmaking decisions solely on the basis of my own moral intuitions, or even solely on the basis of guesses about what my "idealized" moral intuitions would be.

I think of this as more of a definitional issue than a probability issue. Given functionalism about consciousness, if the population of China managed to implement the necessary function of consciousness, then this system would be conscious. The problem is that, on my view, the function of consciousness is likely so complicated and specific that for the population of China to implement it, the "population of China" would be basically unrecognizable as "the population of China." Hence, I think the Chinese nation thought experiment is a misleading intuition pump.

I probably have thoughts on this, but first: Can you say more about what would count as "two systems in conflict"? E.g. would a mere competition among neural signals count? Or would it have to be something more "sophisticated," in a certain way? Also, is the "secondary layer" you're talking about also meant to be "hidden", or are you talking about a "phenomenally conscious" second layer?

I think I always found Dennett's general approach quite plausible (albeit a bit too "hasty" in its proposed reductions; see footnote 222), though I hadn't read other illusionist accounts prior to beginning this investigation. For me, reading Frankish made a bigger difference to my confidence in illusionism than any particular neuroscience papers or books.

Personally, I find discussions of p-zombies somewhat unhelpful, but for those steeped in that literature, it might be a useful set of concepts for explaining illusionism. My first recommendation would still be Frankish's papers, though.

Section 2.2.2 and its many footnotes survey the suggested options briefly but fairly thoroughly, I think. My sense is that even theories of moral patienthood (or "moral status", or "full moral status", or other related but not always identical concepts) which don't emphasize phenomenal consciousness as a necessary condition do, upon closer inspection, include phenomenal consciousness as a necessary condition. For example, in one case a theory (I can't remember which one, now) emphasized "self-awareness" but didn't mention consciousness, but upon reading further, it turned out that their notion of "self-awareness" required phenomenal consciousness, and that they would not think that a system that was "self-aware" without being phenomenally conscious was a moral patient.

That said, it is common for theorists to add additional requirements beyond phenomenal consciousness, especially "valenced experience" / "sentience", but also many other properties such as self-awareness, desires, and the other things listed in the bullet list of section 2.2.2. For elaborations, see the sources cited in the footnotes in that bullet list.

Of course, one might also have a meta-ethical approach that doesn't make use of ideas of "moral patienthood" at all. For example, see the sources cited in the footnote after the phrase "As with many framing choices in this report, this is far from the only way to approach the question…"

I'm not sure I'd say the literature on the evolution of consciousness is especially "wanting" — it's just a really hard problem, and so as with theories of how consciousness works, I didn't find any theories of how consciousness evolved that were even moderately persuasive.

In our current state of uncertainty, little bits of "partial progress" can be made from many angles, and the evolution of consciousness is one of those angles. I don't think I'd highlight it as especially promising, though (in terms of reduction of uncertainty per unit effort). On the present margin, I'm probably more excited about the potential informativeness of (1) computational modeling of the sort I describe in section 6.2.4, (2) certain kinds of studies of human consciousness (Ctrl+F in the report for "perhaps the most promising path forward"), and (3) improvements to tools and techniques of human neuroscience that could help with (2) (Ctrl+F in the report for "we need fundamental breakthroughs").

The "GDAK-inspired cognitive architecture approach" tackles the problem from a different angle. It asks: "What are all the consciousness explananda we can identify and describe in some detail, and which cognitive algorithms might we combine with each other to build a computer program that would exhibit as many of those explananda as possible (including "internally," not just in its "external" behavior), with as much precision as possible?" My current hunch is that further theorizing about the evolution of consciousness wouldn't contribute to that project in an especially direct way, though it might help guide the search process for consciousness explananda, or contribute in some other somewhat indirect way.

I think the strategy you outline is, very roughly, one of the most common strategies for coming up with cognitive theories of consciousness, or at least for coming up with cognitive theories of particular features of consciousness.

However, upon reading a given paper or book of this sort, I'm often left unsure whether the author thinks they've "explained consciousness," or whether they think what they've done so far is more like "gesturing in the right direction." Indeed, Dennett called his 1991 book Consciousness Explained, but even as late as 2011 is saying that some of his favorite theories are "merely the beginning, rather than the end, of the study of consciousness" (Cohen & Dennett 2011).

And of course, many theories of consciousness — including some of the most popular ones — don't appeal to cognitive mechanisms at all.

As for your last paragraph: I'm not sure what you mean by "using said program to make predictions, and, instead of largely introspecting yourselves (c) gathering the mass introspections of many people)?" Could you elaborate on what you're asking, there?

As I mention at one point in the report, “We have begun to collaborate with a programmer on such a project.” That project is only a quick experiment, but once the experiment has hit certain milestones, I’ll have a better guess about which deliverables might be achievable with how much resources. We hope to write about this experiment in the future, at least briefly.

As for catalyzing a paradigm shift in consciousness studies. I’m not sure about that, but I think we could make a lot of progress if we could find 5 great researchers to fund in the paradigm from Appendix B. However, that would require a substantial investment of staff time, and might turn out not to be tractable, and might also compete for scarce talent with other projects that are even more urgent and important (in Open Phil’s estimation, anyway). We're still thinking about whether we want to take a shot at this, though.