This semester (Fall 2023), Prof Adam Elga and I will be co-instructing Longtermism, Existential Risk, and the Future of Humanity, an upper div undergraduate philosophy seminar at Princeton. (Yes, I did shamelessly steal half of our title from The Precipice.) We are grateful for support from an Open Phil course development grant and share the reading list here for all who may be interested. 

[Edit as of 19 Sept 2023: link to full syllabus—which is a bit different than the reading list below: available here]

Part 1: Setting the stage
 

Week 1: Introduction to longtermism and existential risk

Week 2: Introduction to decision theory

  • Core
  • Optional
    • Weisberg, J. (2021). Odds & Ends chapters 5-7 (these may be helpful background for understanding chapter 8, if you don’t have much background in probability).
    • Titelbaum, M. G. (2020) Fundamentals of Bayesian Epistemology chapters 3-4
       

Week 3: Introduction to population ethics

  • Core
    • Parfit, Derek. 1984. Reasons and Persons. Oxford: Oxford University Press. Read sections 4.16.120-23, 125, and 127 (pp. 355-64; 366-71, and 377-79).
    • Parfit, Derek. 1986. “Overpopulation and the Quality of Life.” In Applied Ethics, ed. P. Singer, 145–164. Oxford: Oxford University Press. Read sections 1-3. 
  • Optional
    • Remainders of Part IV of Reasons and Persons and “Overpopulation and the Quality of Life”
    • Greaves (2017) “Population Axiology” Philosophy Compass
    • McMahan (2022) “Creating People and Saving People” section 1, first page of section 4, and section 8
    • Temkin (2012) Rethinking the Good 12.2 pp. 416-17 and section 12.3 (esp. pp. 422-27)
    • Harman (2004) “Can We Harm and Benefit in Creating?”
    • Roberts (2019) “The Nonidentity Problem” SEP
    • Frick (2022) “Context-Dependent Betterness and the Mere Addition Paradox”
    • Mogensen (2019) “Staking our future: deontic long-termism and the non-identity problem” sections 4-5
       

Week 4: Longtermism: for and against

Part 2: Philosophical problems

 

Week 5: Fanaticism

 

Week 6: Cluelessness

  • Core
    • Lenman, J. (2000). “Consequentialism and Cluelessness.” Philosophy and Public Affairs29(4), 342–370. Read sections I, II, and VI. 
    • Greaves, H. (2016). “Cluelessness.” Proceedings of the Aristotelian Society, 116 (3): 311–339. Read sections V and VI (and also section III if you would like more background about the principle of indifference in order to understand section V).
    • Mogensen, A., & MacAskill, W. (2021). “The Paralysis Argument.” Philosophers’ Imprint21(15). Read sections 1-2.
  • Optional
    • Mogensen (2021) “Maximal Cluelessness”
    • Unruh (2023) “The Constraint Against Doing Harm and Long-term Consequences”
    • Re-read section on cluelessness in Greaves and MacAskill “The Case for Strong Longtermism”


Week 7: The Asymmetry

Part 3: Our place in history and what we ought to do

 

Week 8: Moral uncertainty

  • Core
    • MacAskill, W., Bykvist, K., & Ord, T. (2020). Moral Uncertainty. Oxford: Oxford University Press. Read Introduction pp. 1-2, Chapter 1 sections I-II, and Conclusion pp. 213-14.
    • Harman, Elizabeth. (2015). “The Irrelevance of Moral Uncertainty.” In R. Shafer-Landau (Ed.), Oxford Studies in Metaethics, Volume 10. Oxford University Press. Read sections 3.1 - 3.3. 
    • Weatherson, Brian. (2014). “Running risks morally.” Philosophical Studies, 167(1), 141–163. Read sections 1, 3, and 4.
  • Optional
    • MacAskill, Bykvist, and Ord (2020) Moral Uncertainty chapter 2 pp. 33-35
    • Barnett, Z. (2021). “Rational Moral Ignorance.” Philosophy and Phenomenological Research, 102(3), 645–664. https://doi.org/10.1111/phpr.12684
    • Weatherson, B. (2019). Normative Externalism. Oxford University Press. https://doi.org/10.1093/oso/9780199696536.001.0001 Chapter 3 (“Against asymmetry”)
    • Weatherson (2019) Normative Externalism chapter 7 (“Level-crossing principles”) and chapter 8 (“Higher-order evidence”)
    • Tarsney (2021) review of Weatherson Normative Externalism in Mind

 

Week 9: Moral uncertainty and stakes-sensitivity

Week 10: Transformative Artificial Intelligence

 

Week 11: Possibilities for our future and what we can do

  • Core
    • Ord, Toby. 2020. The Precipice: Existential Risk and the Future of Humanity. London: Bloomsbury. Read chapter 8.
    • MacAskill, William. 2022. What We Owe the Future. New York: Basic Books. Read chapter 10. 
  • Optional

 

Further readings (all optional)
 

Discounting

  • Cowen and Parfit (1992) “Against the Social Discount Rate” (in Peter Laslett & James S. Fishkin (eds.) Justice Between Age Groups and Generations pp. 144–161)
  • Mogensen (2022) “The only ethical argument for positive δ? Partiality and pure time preference”
  • Russell (2022) “Problems for Intergenerational Equity” Parfit Memorial Lecture 
  • Ord (2020) The Precipice Appendix A
  • Greaves (2017) “Discounting for Public Policy” section 7 (pp. 404-09)
  • Kelleher (2017) “Pure time preference in intertemporal welfare economics”


The hinge of history hypothesis


Global catastrophic biological risks

  • Ord (2020) The Precipice chapter 5 section on pandemics
  • Piper (2022) “Why experts are terrified of a human-made pandemic — and what we can do to stop it” Vox
  • Lewis (2020) “Reducing global catastrophic biological risks” 80,000 Hours

 

Coordination problems and great power conflict

 

Human enhancement

  • Buchanan (2011) Beyond Humanity?: The Ethics of Biomedical Enhancement (especially chapter 2)
  • Bostrom (2013) “Why I want to be a posthuman when I grow up”
  • Bostrom and Ord (2006) “The Reversal Test: Eliminating Status Quo Bias in Applied Ethics”
  • Chiang (2019) “It’s 2059, and the Rich Kids Are Still Winning” NYT
     

Non-consequentialist concern for the future

  • Scheffler (2013) Death and the Afterlife
  • Scheffler (2018) Why Worry about Future Generations?
  • Caney (2018) “Justice and Future Generations”
  • Meyer (2021) “Intergenerational Justice” SEP
  • Finneron-Burns (2016) “Contractualism and the Non-Identity Problem”
  • Kumar (2018) “Risking Future Generations”
     

Impossibility results in population axiology

88

0
0

Reactions

0
0

More posts like this

Comments7
Sorted by Click to highlight new comments since: Today at 3:44 AM

So cool! And thanks for sharing your syllabus :) Do you have any interest in collaborating with the Princeton EA club this semester? Hit me up at anovick@princeton.edu

This is so cool to see! Thanks for putting it together and for posting :)

Just an FYI, Week 11 refers to the 80,000 Hours career guide, but actually links to our key ideas series, which we've now stopped updating.

Thanks for catching this, Bella! I've updated the link here and on our syllabus. 

Will this be recorded? I'd love to watch!

Hi Saul, since this is a discussion-based seminar rather than a lecture course, we won't be recording. However, I plan to teach this course again in the future and may change the format - so future iterations may be recorded.

Very cool!

random thought: could include some of Yoshua Bengio's or Geoffrey Hinton's writings/talks on AI risks concerns in week 10 (& could include Lecun for counterpoint to get all 3), since they're very-well cited academics & Turing Award Winners for deep learning

I haven't looked through their writings/talks to find the most directly relevant, but some examples: https://yoshuabengio.org/2023/05/22/how-rogue-ais-may-arise/ https://yoshuabengio.org/2023/06/24/faq-on-catastrophic-ai-risks/

Thanks for the recs! What's the Lecun you mention? 

Curated and popular this week
Relevant opportunities