Research Fellow at the Future of Humanity Institute


Topic Contributions


Some potential lessons from Carrick’s Congressional bid

Just wanted to say that I really appreciated this post. As someone who followed the campaign with interest, but not super closely, I found it very informative about the campaign. And it covered all of the key questions I have been vaguely wondering about re: EAs running for office.

List of lists of EA-related open philosophy research questions

opinionated (per its title) and non-comprehensive, but "Key questions about artificial sentience: an opinionated introduction" by me:


My Job: EA Office Manager

I work at Trajan House and I wanted to comment on this:

But a great office gives people the freedom to not worry about what they need for work, a warm environment in which they feel welcome and more productive, and supports them in ways they did not think were necessary.

By these metrics, Trajan House is a really great office! I'm so grateful for the work that Jonathan and the other operations staff do. It definitely makes me happier and more productive.

Trajan House in 2022 is a thriving hub of work, conversation, and fun.

Updates from Leverage Research: history, mistakes and new focus

Leverage just released a working paper, "On Intention Research". From the post:

Starting in 2017, some of Leverage’s psychology researchers stumbled across unusual effects relating to the importance and power of subtle nonverbal communication. Initially, researchers began by attempting to understand and replicate some surprising effects caused by practitioners in traditions like bodywork and energy healing. Over time researchers investigated a wide range of phenomena in subtle nonverbal communication and developed an explanation for these phenomena according to which one’s expectations about what will happen (one’s intentions) in part determine what information is communicated and received nonverbally. This area of research is known as “intention research.”

Those involved in intention research report encountering phenomena that they found quite surprising and challenging to explain. Their findings led many of Leverage’s psychology researchers to conclude that nonverbal communication is at least as expressive and psychologically central as verbal communication. Unfortunately, it also led to some negative psychological and psychosomatic effects and contributed to a significant increase in social tension at Leverage prior to its dissolution in 2019.

This research report describes what intention research was, why researchers pursued it, what they discovered, and the historical antecedents for these discoveries. The piece concludes with a discussion of the risks and challenges associated with further research.

Key questions about artificial sentience: an opinionated guide

Thanks for the comment! I agree with the thrust of this comment.

Learning more and thinking more clearly about implementation of computation in general and neural computation in particular, is perennially on my intellectual to-do list list.

We don't want to allow just any arbitrary gerrymandered states to count as an adequate implementation of consciousness's functional roles

maybe the neurons printed on each page aren't doing enough causal work in generating the next edition

I agree with the way you've formulated the problem, and the possible solution - I'm guessing that an adequate theory of implementation deals with both of them. Some condition about there being the right kind of "reliable, counterfactual-supporting connection between the states" (that quote is from Chalmers' take on these issues).

But I have not yet figured out how to think about these things to my satisfaction.

How many EAs failed in high risk, high reward projects?
Answer by rgbApr 26, 202262

Some past example that come to mind. Kudos to all of the people mentioned for trying ambitious things, and writing up the retrospectives:

  1. Not strictly speaking "EA", but an early effort from folks in the rationality community started an evidence-based medicine organization called MetaMed

Zvi Mowshowitz's post-mortem: https://thezvi.wordpress.com/2015/06/30/the-thing-and-the-symbolic-representation-of-the-thing/

Sarah Constantin's post-mortem: https://docs.google.com/document/d/1HzZd3jsG9YMU4DqHc62mMqKWtRer_KqFpiaeN-Q1rlI/edit

  1. Michael Plant has a post-mortem of his mental health app, Hippo

  2. Looking at around, I also found this list

Some other posts are the Good Technology Projects' postmortema postmortem of a mental health app by Michael Plant, organisations discuss their learnings in retrospectives like Fish Welfare Initiative or in posts announcing decisions to shut down like Students for High Impact Charities. In the Rationalist community, there was the Arbital Postmortem. You can see more examples on the Forum postmortems and retrospectives tag, and examples from the LessWrong community in their analogous tag.

The Bearable Brightness of Wellbeing: The life of an HLI researcher

Thanks for writing this! Your work sounds super interesting. You write, “ But you could be rewarded by the euphoric sense of revelation. Some of that sense may even be authentic; most of it will be fool’s gold.” What are some times you got that euphoric sense in your research for HLI?

The pretty hard problem of consciousness

[Replying separately with comments on progress on the pretty hard problem; the hard problem; and the meta-problem of consciousness]

The meta-problem of consciousness is distinct from both a) the hard problem: roughly, the fundamental relationship between the physical and the phenomenal b) the pretty hard problem, roughly, knowing which systems are phenomenally consciousness

The meta-problem is c) explaining "why we think consciousness poses a hard problem, or in other terms, the problem of explaining why we think consciousness is hard to explain" (6)

The meta-problem has a very interesting relationship to the hard problem. To see what this relationship is, we need a distinction between what the "hard problem" of explaining consciousness, and what Chalmers calls the 'easy' problems of explaining "various objective behavioural or cognitive functions such as learning, memory, perceptual integration, and verbal report".

(Much like 'pretty hard', the 'easy' is tongue in cheek - the easy problems are tremendously difficult and thousands of brilliant people with expensive fancy machines are constantly hard at work on them).

Ease of the easy problems: "the easy problems are easy because we have a standard paradigm for explaining them. To explain a function, we just need to find an appropriate neural or computational mechanism that performs that function. We know how to do this at least in principle."

Hardness of the hard problem: "Even after we have explained all the objective functions that we like, there may still remain a further question: why is all this functioning accompanied by conscious experience?...the standard methods in the cognitive sciences have difficulty in gaining purchase on the hard problem."

The meta problem is interesting because it is deeply related to the hard problem, but it is strictly speaking an 'easy' problem: it is about explaining certain cognitive and behavioral functions. For example: thinking "I am currently seeing purple and it seems strange to me that this experience could simply be explained in terms of physics" or "It sure seems like Mary in the black and white room lacks knowledge of what it's like to see red"; or sitting down and writing "boy consciousness sure is puzzling, I bet I can funding to work on this."

Chalmers hopes that cognitive science can make traction on the meta-problem, by explaining how these cognitive functions and behaviors come about in 'topic neutral' terms that don't commit to any particular metaphysical theory of consciousness. And then if we have a solution to the meta problem, this might shed light on the hard problem.

One particular intriguing connection is that it seems like a) a solution to the meta problem should at least be possible and b) if it is, then it gives us a really good reason not to trust our beliefs about consciousness!

  1. A solution to the meta problem is possible, so there is a correct explanation of our beliefs about consciousness that is independent of consciousness.
  2. If there is a correct explanation of our beliefs about consciousness that is independent of consciousness, those beliefs are not justified.
  3. Our beliefs about consciousness are not justified.

Part of the aforementioned growing interest in illusionism is that I think this argument is pretty good. Chalmers came up with it and elaborated it - even though he is not an illusionist - and I like his elaboration of it more than his replies!

The pretty hard problem of consciousness

[Replying separately with comments on progress on the pretty hard problem; the hard problem; and the meta-problem of consciousness]

Progress on the hard problem

I am much less sure of how to think about this than about the pretty hard problem. This is in part because in general, I'm pretty confused about how philosophical methodology works, what it can achieve, and the extent to which there is progress in philosophy. This uncertainty is not in spite of, but probably because of doing a PhD in philosophy! I have considerable uncertainty about these background issues.

One claim that I would hang my hat on is that the elaboration of (plausible) philosophical positions in greater detail, and more detailed scrutiny of them, is a kind of progress. And in this regard, I think the last 25 years have seen a lot of progress on the hard problem. The possible solution space has been sketched more clearly, and arguments elaborated. One particularly interesting trend is the elaboration of the more 'extreme' solutions to the hard problem: panpsychism and illusionism. Panpsychism solves the hard problem by making consciousness fundamental and widespread; illusionism dissolves the hard problem by denying the existence of consciousness.

Funnily enough, panpsychists and illusionists actually agree on a lot - they are both skeptical of programs that seek to identify consciousness with some physical, computational, or neural property; they both think that if consciousness exists it then it has some strange-sounding relation to the physical. For illusionists, this (putative) anomalousness of consciousness is part of why they conclude it must not exist. For panpsychists, this (putative) anomalousness of consciousness is part of why they are led to embrace a position that strikes many as radical. You can think of this situation by analogy: theologically conservative religious believers and hardcore atheists are often united in their criticisms of theologically liberal religious believers. Panpsychists and illusionists are both united in their criticisms of 'moderate' solutions to the hard problem.

I think the elaboration of these positions is progress. And this situation also forces non-panpsychist consciousness realists, who reject the 'extremism' of both illusionism and panpsychism, to respond and elaborate their views in a stronger way.

For my part, reading the recent literature on illusionism has made me far more sympathetic to it as a position than I was before. (At first glance, illusionism can just sound like an immediate non-starter. Cartoon sketch of an objection: How could consciousness be an 'illusion' - illusions are mismatches between appearance and reality, and with consciousness the appearance is the reality. Anyway, illusionists can respond to this objection - that's a subject for another day). If I continue to be sympathetic to illusionism, then I can say: the growing elaboration of and appeal of illusionism in the last decade represents progress.

But I think there is at least a 40% chance that my mind will have changed significantly regarding illusionism within the next three months.

The pretty hard problem of consciousness

That's a great question. I'll reply separately with my takes on progress on a) the pretty hard problem, b) the hard problem, and c) something called the meta-problem of consciousness [1].

[1] With apologies for introducing yet another 'problem' to distinguish between, when I've already introduced two! (Perhaps you can put these three problems into Anki?)

Progress on the pretty hard problem

This is my attempt to explain Jonathan Birch's recent proposal for studying invertebrate consciousness. Let me know if it makes rough sense!

The problem with studying animal consciousness is that it is hard to know how much we can extrapolate from what we know about what suffices for human consciousness. Let's grant that we know from experiments on humans that you will be conscious of a visual perception if you have a neural system for broadcasting information to multiple sub-systems in the brain. (This is the Global Workspace Theory mentioned above), and that visual perception is broadcast. Great, now we know that this sophisticated human Global Workspace suffices for consciousness. But how much of that is necessary? How much simpler could the Global Workspace be and still result in consciousness?

When we try to take a theory of consciousness "off the shelf" and apply it to animals, we face a choice of how strict to be. We could say that the Global Workspace must be as complicated as the human case. Then no animals count as conscious. We could say that the Global Workspace can be very simple. Then maybe even simple programs count as conscious. To know how strict or liberal to be in applying the theory, we need to know what animals are conscious. Which is the very question!

Some people try to get around this by proposing tests for consciousness that avoid the need for theory--the Turing Test would be an example of this in the AI case. But these usually end up sneaking theory in the backdoor.

Here's Birch's proposal for getting around this impass.

  1. Make a minimal theoretical assumption about consciousness.

The 'facilitation hypothesis':

Phenomenally conscious perception, relative to unconscious perception, facilitates a "cluster" of cognitive abilities.

It's a cluster because it seems like “the abilities will come and go together, co-varying in a way that depends on whether or not a stimulus is consciously perceived” (8). Empirically we have evidence that some abilities in the cluster include: trace conditioning, rapid reversal learning, cross-modal learning.

  1. Look for these clusters of abilities of animals.

  2. See if things which are able to make perceptions unconscious in humans--flashing them quickly and so forth--seems to 'knock out' that cluster in animals. If we can make the clusters come and go like this, it's a pretty reasonable inference that the cause of this is consciousness coming and going.

As I understand it, Birch (a philosopher) is currently working with scientists to flash stuff at bees and so forth. I think Birch's research proposal is a great conceptual advance and I find the empirical research itself very exciting and am curious to see what comes out of it.

Load More