My current takes on AI Welfare

A crux that I have here is that research that takes a while to explain is not going to inspire a popular movement. 

Okay, what comes to mind for me here is quantum mechanics and how we've come up with some pretty good analogies to explain parts of it. 

Do we really need to communicate the full intricacies of AI sentience to say that an AI is conscious? I guess that this isn't the case.

The world where EA research and advocacy for AI welfare is most crucial is one where the reasons to think that AI systems are conscious are non-obvious, such that we require research to discover them, and require advocacy to convince the broader public of them.
But I think that world where this is true, and the advocacy succeeds, is a pretty unlikely one. 

I think this is creating a potential false dichotomy? 
Here's what I believe might happen in AI Sentience without any intervention as an example:

1. Consciousness is IIT (Integrated Information Theory) or GWT (Global Workspace Theory) based in some way or another. In other words, we have some sort of underlying field of sentience like the electromagnetic field and when parts of the field interact in specific ways then "consciousness" appears as a point load in that field.
2. Consciousness is then only verifiable if this field has consequences on the other fields of reality; otherwise, it is non-popperian, like the Multiverse theory. 
3. Number 2 is really hard to prove and so we're left with very correlational evidence. It is also tightly connected to what we think of as metaphysics, meaning that we're going to be quite confused about it. 
4. Therefore, general legislators and researchers leave this up to chance and do not compute any complete metrics, as it is too difficult a problem. They hope that AIs don't have sentience. 

In this world, adding some AI sentience research from the EA Direction could have the consequences of:

1. Making AI labs have consciousness researchers on board so that they don't torture billions of iterations of the same AI.
2. Make governments create consciousness legislation and think tanks for the rights of AI.
3. Create technical benchmarks and theories about what is deemed to be conscious (See this initial, really good report for example)

You don't have to convince the general public; you have to convince the major stakeholders of tests that check for AI consciousness. It honestly seems kind of similar to what we have done for the safety of AI models but instead for the consciousness of them?

I'm quite excited for this week as it is a topic I'm very interested in but something that I also feel that I can't really talk about that much or take seriously as it is a bit fringe so thank you for having it!

Could you expand on what you regard as a key difference in our epistemic position with respect to animals vs. even in theory with respect to AI systems? Could this difference be put in terms of a claim you accept when applied to animals but not even in theory when applied to AI systems?

In connection with evaluating animal/AI consciousness, you mention behavior, history, incentives, purpose, and mechanism.  Do you regard any of these factors as most directly relevant to consciousness? Are any of these only relevant as proxies for, say, mechanisms?

(My hunch is that more information on these points would make it easier for me or other readers to try to change your mind!)

Re “A crux here is that philosophy of mind doesn't really make much progress”: for what it’s worth, from the inside of the field, it feels to me like philosophy of mind makes a lot of progress, but (i) the signal-to-noise ratio in the field is bad, (ii) the field is large, sprawling, and uncoordinated, (iii) an impact-focused mindset is rare within the field, and (iv) only a small percentage of the effort in the field has been devoted to producing research that is directly relevant to AI welfare. This suggest to me that even if there isn’t a lot of relevant, discernible-from-the-outside progress in philosophy of mind, relevant progress may be fairly tractable.

Thanks for sharing your thoughts!

I’m unsure whether we can in principle ascertain whether a digital mind is conscious. 

I used to believe this too, but I've actually become quite optimistic that this is a tractable question. If, as David Pearce argues, the laws of physics don't break down inside the brain, and we define the constraints that a satisfactory theory of consciousness should fulfill, then I believe we can make very good progress. For instance, one such constraint is that the theory should solve the binding problem (see also here). So if we cannot observe a binding mechanism in digital computers, or explain how digital computers could in principle give rise to binding (as I and other at QRI argue), then that should be very strong evidence against digital computers being conscious. More in: Digital Sentience: Can Digital Computers Ever "Wake Up"?

(Note that this doesn't rule out other forms of artificial, non-digital consciousness!)

This could lead to behaviour analogous to conscious behaviour in humans, but with a very different mechanism or purpose, which does not actually produce qualia. I do not know how we would be able to tell the difference, even in theory.

How about: Remove all text of humans discussing their conscious experiences (or even the existence of consciousness) from the AI's training set. See if it still claims to have internal experiences.

I don't think this is a perfect method:

• If it still talks about internal experiences, maybe it was able to extrapolate the ability to discuss internal experiences from text that wasn't removed.

• If it doesn't talk about internal experiences, maybe it has them and just lacks the ability to talk about them. Some animals are probably like this.

Finally, in principle I can imagine that ingesting text related to internal experiences is actually what causes an AI to learn to have them.