Hide table of contents

Back in November 2022 I wrote a post with some mini summaries of Global Priorities Institute (GPI) papers which received a positive reception. This post contains some summaries for recent GPI papers (Sept 2023 - March 2024).

These summaries have been reviewed and signed off by GPI. Note that I am only summarising papers that have a GPI-affiliated author, but there are additional papers on the GPI website. I am focusing on theoretical papers where understanding the argumentation can be useful. For two empirically-focused papers I simply copy over the abstract which I think is sufficient. For one paper the author did not feel a short summary would be appropriate.

A non-identity dilemma for person-affecting views (Elliott Thornley)

The bottom line:

  • Every possible person-affecting view runs into serious issues. The conclusion is that such views are false, and therefore that we should do more to reduce the risk of human extinction this century.

My brief summary:

  • Person-affecting views (PAVs) in population ethics state that (in cases where all else is equal) we’re permitted but not required to create people who would enjoy good lives.
  • Narrow PAVs suggest it's permissible to prefer creating a life that's barely good over one that's excellent, while wide PAVs argue for the necessity of choosing the superior life.
  • Narrow PAVs imply a trilemma. Consider a choice case called ‘Expanded Non-Identity’ which has three outcomes: (1) Amy living a minimally good life, (2) Bob experiencing an excellent life, and (3) both leading mediocre lives.
  • If a narrow PAV says that (1) is permissible, it is permissible to choose dominated options. If it says (3) is permissible, it is permissible to trade off a vast amount of an individual’s wellbeing to create a different mediocre life. And if it says (2) is permissible, adding an impermissible option can (weirdly) make a previously permissible option become impermissible. Each of these implications is seriously implausible and we seem to have a problem for narrow PAVs.
  • Wide PAVs run into a problem with sequential choice. Consider a choice case called ‘Two-Shot Non-Identity’ in which one first decides whether or not to create Amy with a barely good life, and then decides whether or not to create Bob with a wonderful life.
  • Considering this choice case shows that wide PAVs make the permissibility of choices depend on factors that seem morally irrelevant, and can force decisions that seem to fundamentally undermine PAVs.
  • The conclusion is that all PAVs face serious issues. In cases where all else is equal, we’re required to create people who would enjoy good lives. This implies that we should do more to reduce the risk of human extinction this century.
  • See paper and this longer summary.

How to resist the Fading Qualia Argument (Andreas Mogensen)

The bottom line:

  • Given reasonable empirical assumptions, the Fading Qualia Argument supports the view that conscious AI systems could realistically be built in the near term, but the assumptions of vagueness of consciousness at its boundaries and the holistic nature of consciousness decrease confidence in the argument.

My brief summary:

  • The Fading Qualia Argument (FQA) suggests that consciousness is not dependent on the physical material of a system but on the causal interactions within it. If the argument holds, it is plausible that conscious AI systems could realistically be built in the near term, something of huge moral importance.
  • FQA imagines gradually replacing neurons in someone's brain one-by-one with functionally identical silicon chips. It argues there can be neither suddenly disappearing consciousness nor fading consciousness in this process without extremely implausible consequences.
  • This argument can be resisted by interpreting the neural replacement spectrum as involving vagueness, suggesting that it's indeterminate whether systems with partially replaced neurons remain conscious and challenging the idea that consciousness persists unchanged throughout the neuron replacement process.
  • We could try to get around this reply by imagining that key parts of the brain, like the visual areas, are completely replaced with silicon before the rest, which we might think should determinately result in loss of visual experience while the person is otherwise determinately conscious. However, this version of the argument can be resisted given neural holism, which proposes that conscious neural activity is inherently holistic.
  • Both vagueness and holism are controversial assumptions and only expose important weaknesses in FQA rather than refute it. The extent to which one is sceptical of the argument should depend on one’s belief in the strength of these two assumptions.
  • See paper.

How important is the end of humanity? Lay people prioritize extinction prevention but not above all other societal issues. – Matthew Coleman (Northeastern University), Lucius Caviola (Global Priorities Institute, University of Oxford) et al.

Abstract: Human extinction would mean the deaths of eight billion people and the end of humanity’s achievements, culture, and future potential. On several ethical views, extinction would be a terrible outcome. How do people think about human extinction? And how much do they prioritize preventing extinction over other societal issues? Across six empirical studies (N = 2,541; U.S. and China) we find that people consider extinction prevention a global priority and deserving of greatly increased societal resources. However, despite estimating the likelihood of human extinction to be 5% this century (U.S. median), people believe the odds would need to be around 30% for it to be the very highest priority. In line with this, people consider extinction prevention to be only one among several important societal issues. People’s judgments about the relative importance of extinction prevention appear relatively fixed and are hard to change by reason-based interventions.

See paper.

Welfare and Felt Duration (Andreas Mogensen)

The bottom line:

  • The idea that how long an experience feels (subjective duration) is more important than how long it actually lasts (objective duration) in deciding its overall value doesn't follow from any current theories about what subjective duration consists in. In fact, some plausible theories strongly suggest that subjective duration is irrelevant in itself.

My brief summary:

  • A distinction can be drawn between the subjective "felt duration" of a valenced experience (how long a pleasurable or painful experience seems to last) and its objective duration (the actual time it lasts).
  • One theory of felt duration is cognitivism, a theory suggesting that our perception of time's passage is influenced by the relative speeds of external events and our internal cognitive processes. This theory helps explain why time appears to slow down during life-threatening situations, due to an increase in the pace of our mental activity. According to Mogensen, under the cognitivism theory, variations in the subjective duration of pain should not affect our assessment of its severity. This is because, intuitively, the intensity of pain is not inherently linked to the perceived speed of our thoughts in relation to external events.
  • Another theory is the quantum theory of felt duration, positing that conscious experience is a series of discrete perceptual frames. Mogensen argues that the number of these frames during a given time period does not influence the severity of pain felt. One reason is that continuous consciousness seems to be possible, even if in fact our own experiences are discrete.
  • The overarching conclusion is that we don't currently have any theories of what subjective duration consists in on which it seems plausible that subjective duration in itself modulates the welfare significance of pains and pleasures. At least some plausible theories of what subjective duration consists in strongly suggest that subjective duration is irrelevant in itself.
  • See paper and another summary.

Estimating long-term treatment effects without long-term outcome data – David Rhys Bernard (Rethink Priorities), Jojo Lee and Victor Yaneng Wang (Global Priorities Institute, University of Oxford)

Abstract: The surrogate index method allows policymakers to estimate long-run treatment effects before long-run outcomes are observable. We meta-analyse this approach over nine long-run RCTs in development economics, comparing surrogate estimates to estimates from actual long-run RCT outcomes. We introduce the M-lassoalgorithm for constructing the surrogate approach’s first-stage predictive model and compare its performance with other surrogate estimation methods. Across methods, we find a negative bias in surrogate estimates. For the M-lasso method, in particular, we investigate reasons for this bias and quantify significant precision gains. This provides evidence that the surrogate index method incurs a bias-variance trade-off.

See paper.

Egyptology and Fanaticism (Hayden Wilkinson)

The bottom line:

  • An ‘Egyptology’ argument for Fanaticism says that rejecting Fanaticism would entail that our moral evaluations hinge on what happens in distant parts of the world unaffected by our choice (e.g., what happened in ancient Egypt). This paper salvages the Egyptology argument against a strong criticism, showing that perhaps the most compelling argument for Fanaticism remains compelling.

My brief summary:

  • Various decision theories share a troubling implication. They imply that, for any finite amount of value, it would be better to wager it all for a vanishingly small probability of some greater value. This is fanaticism.
  • It can be shown that denying fanaticism means that our evaluations of different options will sometimes hinge on the value of far-off events that are entirely unaffected by your choice (e.g., events in ancient Egypt). But this seems absurd. This is one (perhaps compelling) reason to accept Fanaticism. This argument is known as an ‘Egyptology’ argument.
  • More technically, this argument can be interpreted as follows. If the principles ‘Separability for Options’ and ‘Stochastic Dominance’ (both of which seem very plausible) are true, then Fanaticism must be true. However, there's a counterargument: Fanaticism turns out to be incompatible with the conjunction of these two principles. As such, the Egyptology argument seems to fail.
  • The paper addresses this by proposing a shift from 'Separability for Options' to a less stringent principle, 'Separability for Independent Options'. The Egyptology argument is salvaged - Fanaticism must hold if we accept Stochastic Dominance and Separability for Independent Options (and is compatible with the conjunction of these principles).
  • Furthermore, we can independently justify ‘Separability for Independent Options’ without also accepting ‘Separability for Options’, allowing us to evade the issue we identified with the latter principle.
  • The conclusion is that the Egyptology argument for Fanaticism remains compelling.
  • See paper.
Comments1


Sorted by Click to highlight new comments since:

Executive summary: This post provides brief summaries of several recent Global Priorities Institute (GPI) papers on topics including population ethics, consciousness, human extinction, and long-term impact estimation, highlighting their key arguments and conclusions.

Key points:

  1. All person-affecting views in population ethics face serious issues, implying we should do more to reduce existential risk this century.
  2. The Fading Qualia Argument suggests conscious AI systems may be possible in the near-term, but vagueness and holism of consciousness weaken confidence in the argument.
  3. People consider human extinction prevention a priority, but not the single highest priority unless the risk is very high (around 30% this century).
  4. Current theories of subjective duration of experiences do not clearly suggest that subjective duration itself affects the value of experiences.
  5. The surrogate index method for estimating long-term treatment effects before long-term data is available involves a bias-variance tradeoff.
  6. The 'Egyptology' argument, perhaps the most compelling case for Fanaticism in ethics, can be salvaged against a key objection.

 

 

This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.

Curated and popular this week
 ·  · 10m read
 · 
I wrote this to try to explain the key thing going on with AI right now to a broader audience. Feedback welcome. Most people think of AI as a pattern-matching chatbot – good at writing emails, terrible at real thinking. They've missed something huge. In 2024, while many declared AI was reaching a plateau, it was actually entering a new paradigm: learning to reason using reinforcement learning. This approach isn’t limited by data, so could deliver beyond-human capabilities in coding and scientific reasoning within two years. Here's a simple introduction to how it works, and why it's the most important development that most people have missed. The new paradigm: reinforcement learning People sometimes say “chatGPT is just next token prediction on the internet”. But that’s never been quite true. Raw next token prediction produces outputs that are regularly crazy. GPT only became useful with the addition of what’s called “reinforcement learning from human feedback” (RLHF): 1. The model produces outputs 2. Humans rate those outputs for helpfulness 3. The model is adjusted in a way expected to get a higher rating A model that’s under RLHF hasn’t been trained only to predict next tokens, it’s been trained to produce whatever output is most helpful to human raters. Think of the initial large language model (LLM) as containing a foundation of knowledge and concepts. Reinforcement learning is what enables that structure to be turned to a specific end. Now AI companies are using reinforcement learning in a powerful new way – training models to reason step-by-step: 1. Show the model a problem like a math puzzle. 2. Ask it to produce a chain of reasoning to solve the problem (“chain of thought”).[1] 3. If the answer is correct, adjust the model to be more like that (“reinforcement”).[2] 4. Repeat thousands of times. Before 2023 this didn’t seem to work. If each step of reasoning is too unreliable, then the chains quickly go wrong. Without getting close to co
JamesÖz
 ·  · 3m read
 · 
Why it’s important to fill out this consultation The UK Government is currently consulting on allowing insects to be fed to chickens and pigs. This is worrying as the government explicitly says changes would “enable investment in the insect protein sector”. Given the likely sentience of insects (see this summary of recent research), and that median predictions estimate that 3.9 trillion insects will be killed annually by 2030, we think it’s crucial to try to limit this huge source of animal suffering.  Overview * Link to complete the consultation: HERE. You can see the context of the consultation here. * How long it takes to fill it out: 5-10 minutes (5 questions total with only 1 of them requiring a written answer) * Deadline to respond: April 1st 2025 * What else you can do: Share the consultation document far and wide!  * You can use the UK Voters for Animals GPT to help draft your responses. * If you want to hear about other high-impact ways to use your political voice to help animals, sign up for the UK Voters for Animals newsletter. There is an option to be contacted only for very time-sensitive opportunities like this one, which we expect will happen less than 6 times a year. See guidance on submitting in a Google Doc Questions and suggested responses: It is helpful to have a lot of variation between responses. As such, please feel free to add your own reasoning for your responses or, in addition to animal welfare reasons for opposing insects as feed, include non-animal welfare reasons e.g., health implications, concerns about farming intensification, or the climate implications of using insects for feed.    Question 7 on the consultation: Do you agree with allowing poultry processed animal protein in porcine feed?  Suggested response: No (up to you if you want to elaborate further).  We think it’s useful to say no to all questions in the consultation, particularly as changing these rules means that meat producers can make more profit from sel
michel
 ·  · 4m read
 · 
I'm writing this in my personal capacity as someone who recently began working at the Tarbell Center for AI Journalism—my colleagues might see things differently and haven’t reviewed this post.  The rapid development of artificial intelligence could prove to be one of the most consequential technological transitions in human history. As we approach what may be a critical period in AI development, we face urgent questions about governance, safety standards, and the concentration of power in AI development. Yet many in the general public are not aware of the speed of AI development, nor the implications powerful AI models could have for them or society at large. Society’s capacity to carefully examine and publicly debate AI issues lags far behind their importance.  This post makes the case for why journalism on AI is an important yet neglected path to remedy this situation. AI journalism has a lot of potential I see a variety of ways that AI journalism can helpfully steer AI development.  Journalists can scrutinize proposed regulations and safety standards, helping improve policies by informing relevant actors of potential issues or oversights. * Example article: Euractiv’s detailed coverage of the EU AI Act negotiations helped policymakers understand the importance of not excluding general-purpose models from the act, which may have played a key role in shaping the final text.[1] * Example future opportunity: As individual US states draft AI regulations in 2025, careful analysis of proposed bills could help prevent harmful loopholes or unintended consequences before they become law. Journalists can surface (or amplify) current and future risks, providing activation energy for policymakers and other actors to address these risks.  * Example article: 404 Media's investigation revealing an a16z-funded AI platform's generation of images that “could be categorized as child pornography” led a cloud computing provider to terminate its relationship with the platf