Hide table of contents

Comment Permalink

(EDIT: Split this up into two comments, the other here.)

I think that there's probably a minimum level of substrate independence we should accept, e.g. that it doesn't matter exactly what matter a "brain" is made out of, as long as the causal structure is similar enough on a fine enough level. The mere fact that neurons are largely made out of carbon doesn't seem essential. Furthermore, human and (apparently) conscious animal brains are noisy and vary substantially from one another, so exact duplication of the causal structure doesn't seem necessary, as long as the errors don't accumulate so much that the result isn't similar to a plausible state for a plausible conscious biological brain.^[1] So, I'm inclined to say that we could replace biological neurons with artificial neurons and retain consciousness, at least in principle, but it could depend on the artificial neurons.

It's worth pointing out that the China brain^[2] and a digital mind (or digital simulation of a mind, on computers like today's) aren't really causally isomorphic to biological brains even if you ignore a lot of the details of biological brains. Obviously, you also have to ignore a lot of the details of the China brain and digital minds. But I could imagine that the extra details in the China brain and digital minds make a difference.

In a simulated neuron, both in the China brain and digital minds, there are details to ignore. In the China brain, that's all the stuff happening inside each person simulating a neuron. For a digital mind, there's probably lots of extra hardware stuff going on.
In a digital mind on a computer like today's computers or even distributed across hundreds of computers or processing units (CPU cores, GPUs), it seems you must ignore the fact that the digital state transitions are orchestrated centrally "from the outside", today through some kind of loop (e.g. a for-loop or while-loop, or some number of these with some asynchronous distribution). Individual biological neurons act relatively autonomously/asynchronously just in response to local neural activity (including electrical and chemical signals), without this kind of external central orchestration. Actually, if you were to ignore the centralized orchestration in a digital mind, depending on how you cash that out, the digital mind might never change states, so maybe the digital mind isn't actually isomorphic to a biological brain at the right level(s) of causal structure for each at all.

These extra details make me less sure that we should attribute consciousness to the China brain and digital minds, but they don’t seem decisive.

^{^}

From footnote 4 from Godfrey-Smith, 2023 (based on the talk he gave):

At the NYU talk, Chalmers raised a passage from The Conscious Mind (p. 331) where he claims, in relation to replacement scenarios, that "when it comes to duplicating our cognitive capacities, a close approximation is as good as the real thing." His argument is that in biological systems, random "noise" processes play a role (greater than the role of any analogous processes in a computer). When the biological system performs some operation, the outcome is never entirely reliable and will instead fall within a band of possibilities. An artificial duplicate of the biological system only has to give a result somewhere in that band. The duplicate's output might depart from what the biological system actually does, on some occasion, but the biological system could just as well have produced the same output as the duplicate, if noise had played a different role. When a duplicate gives a result within the band, it is doing "as well as the system itself can reliably do."
In response, it is true that this role for noise is an important micro-functional feature of living systems. In addition, neurons change what they do as a result of their normal operation, they don't respond to the "same" stimulus twice in the same way (see "Mind, Matter, and Metabolism" for references). The "rules" or the "program" being followed are always changing as a result of the activity of the system itself and its embedding in other biological processes. Over time, the effects of these factors will accumulate and compound – a comparison of what a living system and a duplicate might do in a single operation doesn't capture their importance. I see all this not as a "lowering of the bar" that enables us to keep talking in a rough way about functional identity, but another functional difference between living and artificial systems.

^{^}

From the Wikipedia page:

the China brain thought experiment (also known as the Chinese Nation or Chinese Gym) considers what would happen if each member of the Chinese nation were asked to simulate the action of one neuron in the brain, using telephones or walkie-talkies to simulate the axons and dendrites that connect neurons. Would this arrangement have a mind or consciousness in the same way that brains do?

(China's population, at 1.4 billion, isn't large enough for each person to only simulate one neuron and so simulate a whole human brain with >80 billion neurons, but we could imagine a larger population, or a smaller animal brain being simulated, e.g. various mammals or birds.)

See in context

How to Resist the Fading Qualia Argument (Andreas Mogensen)

by Global Priorities Institute, Andreas_Mogensen

Mar 26 20242 min read 8

30

PhilosophyConsciousness researchGlobal Priorities InstituteGlobal priorities researchResearchSentience

Frontpage

How to Resist the Fading Qualia Argument (Andreas Mogensen)

Abstract

Introduction

Read the rest of the paper

8 comments

This is a linkpost for https://globalprioritiesinstitute.org/how-to-resist-the-fading-qualia-argument-mogensen/

This paper was published as a GPI working paper in March 2024.

Abstract

The Fading Qualia Argument is perhaps the strongest argument supporting the view that in order for a system to be conscious, it does not need to be made of anything in particular, so long as its internal parts have the right causal relations to each other and to the system’s inputs and outputs. I show how the argument can be resisted given two key assumptions: that consciousness is associated with vagueness at its boundaries and that conscious neural activity has a particular kind of holistic structure. I take this to show that what is arguably our strongest argument supporting the view that consciousness is substrate independent has important weaknesses, as a result of which we should decrease our confidence that consciousness can be realized in systems whose physical composition is very different from our own.

Introduction

Many believe that in order for a system to be conscious, it does not need to be made of anything in particular, so long as its internal parts have the right causal relations to each other and to the system’s inputs and outputs. As a result, many also believe that the right software could in principle allow there to be something it is like to inhabit a digital computer, controlled by an integrated circuit etched in silicon. A recent expert report concludes that if consciousness requires only the right causal relations among a system’s inputs, internal states, and outputs, then “conscious AI systems could realistically be built in the near term.” (Butlin et al. 2023: 6) If that were to happen, it could be of enormous moral importance, since digital minds could have superhuman capacities for well-being and ill-being (Shulman and Bostrom 2021).

But is it really plausible that any system with the right functional organization will be conscious - even if it is made of beer-cans and string (Searle 1980) or consists of a large assembly of people with walky-talkies (Block 1978)? My goal in this paper is to raise doubts about what I take to be our strongest argument supporting the view that consciousness is substrate independent in something like this sense.^[1] The argument I have in mind is Chalmers’ Fading Qualia Argument (Chalmers 1996: 253–263). I show how it is possible to resist the argument by appeal to two key assumptions: that consciousness is associated with vagueness at its boundaries and that conscious neural activity has a particular kind of holistic structure. Since these assumptions are controversial, I claim only to have exposed important weaknesses in the Fading Qualia Argument.

I’ll begin in section 2 by explaining what the Fading Qualia Argument is supposed to show and the broader dialectical context it inhabits. In section 3, I give a detailed presentation of the argument. In section 4, I show how the argument can be answered given the right assumptions about vagueness and the structure of conscious neural activity. At this point, I rely on the assumption that vagueness gives rise to truth-value gaps. In section 5, I explain how the argument can be answered even if we reject that assumption. In section 6, I say more about the particular assumption about the holistic structure of conscious neural activity needed to resist the Fading Qualia Argument in the way I outline. I take the need to rely on this assumption to be the greatest weakness of the proposed response.

Read the rest of the paper

^{^}

See the third paragraph in section 2 for discussion of two ways in which the conclusion supported by this argument is weaker than some may expect a principle of substrate independence to be.

30 Reactions

More posts like this

Will AI Avoid Exploitation? (Adam Bales)

Global Priorities Institute

Summary: Against the singularity hypothesis

Global Priorities Institute+2 more

Summary: The scope of longtermism

Global Priorities Institute+1 more

Comments8

Sorted by

New & upvoted

Click to highlight new comments since: Today at 1:25 AM

MichaelStJules1y4

(EDIT: Split this up into two comments, the other here.)

In a simulated neuron, both in the China brain and digital minds, there are details to ignore. In the China brain, that's all the stuff happening inside each person simulating a neuron. For a digital mind, there's probably lots of extra hardware stuff going on.
In a digital mind on a computer like today's computers or even distributed across hundreds of computers or processing units (CPU cores, GPUs), it seems you must ignore the fact that the digital state transitions are orchestrated centrally "from the outside", today through some kind of loop (e.g. a for-loop or while-loop, or some number of these with some asynchronous distribution). Individual biological neurons act relatively autonomously/asynchronously just in response to local neural activity (including electrical and chemical signals), without this kind of external central orchestration. Actually, if you were to ignore the centralized orchestration in a digital mind, depending on how you cash that out, the digital mind might never change states, so maybe the digital mind isn't actually isomorphic to a biological brain at the right level(s) of causal structure for each at all.

These extra details make me less sure that we should attribute consciousness to the China brain and digital minds, but they don’t seem decisive.

^{^}

From footnote 4 from Godfrey-Smith, 2023 (based on the talk he gave):

At the NYU talk, Chalmers raised a passage from The Conscious Mind (p. 331) where he claims, in relation to replacement scenarios, that "when it comes to duplicating our cognitive capacities, a close approximation is as good as the real thing." His argument is that in biological systems, random "noise" processes play a role (greater than the role of any analogous processes in a computer). When the biological system performs some operation, the outcome is never entirely reliable and will instead fall within a band of possibilities. An artificial duplicate of the biological system only has to give a result somewhere in that band. The duplicate's output might depart from what the biological system actually does, on some occasion, but the biological system could just as well have produced the same output as the duplicate, if noise had played a different role. When a duplicate gives a result within the band, it is doing "as well as the system itself can reliably do."
In response, it is true that this role for noise is an important micro-functional feature of living systems. In addition, neurons change what they do as a result of their normal operation, they don't respond to the "same" stimulus twice in the same way (see "Mind, Matter, and Metabolism" for references). The "rules" or the "program" being followed are always changing as a result of the activity of the system itself and its embedding in other biological processes. Over time, the effects of these factors will accumulate and compound – a comparison of what a living system and a duplicate might do in a single operation doesn't capture their importance. I see all this not as a "lowering of the bar" that enables us to keep talking in a rough way about functional identity, but another functional difference between living and artificial systems.

^{^}

From the Wikipedia page:

the China brain thought experiment (also known as the Chinese Nation or Chinese Gym) considers what would happen if each member of the Chinese nation were asked to simulate the action of one neuron in the brain, using telephones or walkie-talkies to simulate the axons and dendrites that connect neurons. Would this arrangement have a mind or consciousness in the same way that brains do?

MichaelStJules1y3

Some other arguments that push in favour of functionalism, the consciousness of simulated brains, including the China brain and digital minds, and brains with other artificial neurons:

We may ourselves be simulated in a similar way without knowing it, if our entire reality is also simulated. We wouldn't necessarily have access to what the simulation is run on.
In a simulated brain and the conscious biological brain it simulates, introspection would give the brains same beliefs about phenomenal properties and qualia, because it's only sensitive to the causal/functional structure at a given level of detail, and those details are by design/assumption preserved under simulation. If the biological brain is phenomenally conscious, but the simulated brain is not, then it's a surprising coincidence that the resulting beliefs about phenomenal consciousness are accurate in the biological brain but not in the simulated brain. Introspection doesn't seem to give the biological brain any more reason to believe in its own phenomenal consciousness than it should for the simulated brain in its own, because introspection is only sensitive to causal/functional details common to both.^[1]
It's hard for me to imagine a compelling explanation of our consciousness that doesn't extend to simulated brains, including the China brain and digital minds. Theories out there now don't seem on track to address the hard problem, and this and other reasons (like above) incline me to dissolve it and accept illusionism about phenomenal properties/consciousness. Illusionism is generally functionalist, and I don't see how an illusionist theory would deny the consciousness of the China brain and digital simulations of brains.

^{^}

This is essentially the coincidence argument for illusionism in Chalmers, 2018.

Arepo1y3

We may ourselves be simulated in a similar way without knowing it, if our entire reality is also simulated. We wouldn't necessarily have access to what the simulation is run on.

It seems weird to meaningfully update in favour of some concrete view on the basis that something might be true but that

we have no evidence for it, and
if it is true then everything we know about the universe is equally undermined

MichaelStJules1y2

I agree there is something a bit weird about it, but I'm not sure I endorse that reaction. This doesn’t seem so different from p-zombies, and probably some moral thought experiments.

I don't think it's true that everything we know about the universe would be equally undermined. Most things wouldn't be undermined at all or at worst would need to be slightly reinterpreted. Our understanding of physics in our universe could still be about as reliable (depending on the simulation), and so would anything that follows from it. There's just more stuff outside our universe.

I guess you can imagine short simulations where all our understanding of physics is actually just implanted memories and fabricated records. But in doing so, you're throwing away too much of the causal structure that apparently explains our beliefs and makes them reliable. Longer simulations can preserve that causal structure.

Arepo1y2

This doesn’t seem so different from p-zombies, and probably some moral thought experiments.

I'm not sure what you mean here. That the simulation argument doesn't seem different from those? Or that the argument that 'we have no evidence of their existence and therefore shouldn't update on speculation about them' is comparable to what I'm saying about the simulation hypothesis?

If the latter, fwiw, I feel the same way about p-zombies and (other) thought experiments. They are a terrible methodology for reasoning about anything, very occasionally the only option we can think of, but one from which philosophers don't feel nearly enough urgency to find alternatives to which to move.

Our understanding of physics in our universe could still be about as reliable (depending on the simulation), and so would anything that follows from it. There's just more stuff outside our universe.

I don't see how this would allow us to update on anything based on speculation about the 'more stuff'. Yeah, we might choose to presume our pocket simulation will continue to behave as it has, but we don't get to then say 'there's some class of matter other than our own simulated matter which generates consciousness therefore consciousnessness is substrate independence.

As you say in your other comment, there's probably some minimal level of substrate independence that non-solipsists have to accept, but that turns it into an empirical question (as it should be) - so an imagined metaverse gives us no reason to change our view on how substrate independent consciousness is.

in doing so, you're throwing away too much of the causal structure that apparently explains our beliefs and makes them reliable

This seems like an argument from sadness. What we would lose by imagining some outcomes shouldn't affect our overall epistemics.

Arepo1y2

Is there are online version of the case for the fading qualia argument? This feels a bit abstract without it...

David Mathers🔸1y3

The best argument for functionalism* in my opinion is that there aren't really any good alternatives. If mental state kinds aren't functional kinds, they'd presumably have to be neuroscientific kinds. But if that's right, then we can already know now aliens without neurons aren't conscious. Which seems wild to me: how can we possibly know if aliens are conscious till we meet them, and observe their behavior and how it depends on what goes on inside them? And surely once we do meet them, no one is going to say "oh consciousness is this sort of neurobiological property, we looked in their head with our scanner and they have no neurons, problem solved, we know they aren't conscious. People seem to want there to be some intermediate view that says "oh of course there might be conscious aliens with different biology, we just mean to rule out weird functional duplicates of humans like a robot controlled by radio signals running the same program as human**", but it's really unclear how to do that in a principled way. (And I suspect the root of the desire to do so is a sort of primitive sense that living matter can have feelings but dead matter can't, which I think people would consciously disavow if they understood it was driving their views)

*There's an incredibly technical complication here about the fact that "functionalism" is usually defined in opposition to mind-body dualism, but in the current context it makes more sense to classify certain forms of dualism as functionalist, since they agree with functionalism about what guarantees something is conscious in the actual world. But I'm going to ignore it because I don't think I can explain it to non-philosophers quickly and easily.

**https://en.wikipedia.org/wiki/China_brain

MichaelStJules1y2

Related recent talk:

and notes.