In his recent interview on the 80000 Hours Podcast, Toby Ord discussed how
nonstandard analysis and its notion of hyperreals may help resolve some apparent
issues arising from infinite ethics (link to transcript). For those interested
in learning more about nonstandard analysis, there are various books and online
resources. Many involve fairly high-level math as they are aimed at putting what
was originally an intuitive but imprecise idea onto rigorous footing. Instead of
those, you might want to check out a book like that of H. Jerome Keisler's
Elementary Calculus: An Infinitesimal Approach, which is freely available
online. This book aims to be an introductory calculus textbook for college
students, which uses hyperreals instead of limits and delta-epsilon proofs to
teach the essential ideas of calculus such as derivatives and integrals. I
haven't actually read this book but believe it is the best known book of this
sort. Here's another similar-seeming book by Dan Sloughter.
One problem in AI/policy/ethics is that it seems like we sometimes have
inconsistent preferences. For example, I could want to have a painting in my
office, and simultaneously not want to have a painting in my office. This is a
problem because classical (preference) logic can’t really deal with
contradictions, so it relies on the assumption that they don't exist.
Since it seems like we do sometimes experience them, I created a non-classical
preference logic that can deal with them. Because effective altruism is all
about logically satisfying preferences I considered putting it on this forum,
but decided against it since it's a bit too abstract.
For those of you that are interested in logic/philosophy, you can read it here.
EDIT: This type of logic has, apart from applications in epistemology and moral
philosophy, also recently seen applications in artificial intelligence and
Julia Nefsky is giving a research seminar in the Institute for Futures Studies
titled "Expected utility, the pond analogy and imperfect duties", which sounds
interesting for the community. It will be on September 27 at 10:00-11:45 (CEST)
and can be attended for free in person or online (via zoom). You can find the
abstract here and register here.
I don't know Julia or her work and I'm not philosopher, so I cannot directly
assess the expected quality of the seminar, but I've seen several seminars from
the Institute for Futures Studies that where very good (eg. from Olle Häggström
--and in Sep 20 Anders Sandberg gives one as well).
I hope this is useful information.
I'm curious what people who're more familiar with infinite ethics think of
Manheim & Sandberg's What is the upper limit of value?, in particular where they
discuss infinite ethics (emphasis mine):
I first read their paper a few years ago and found their arguments for the
finiteness of value persuasive, as well as their collectively-exhaustive
responses in section 4 to possible objections. So ever since then I've been
admittedly confused by claims that the problems of infinite ethics still warrant
concern w.r.t. ethical decision-making (e.g. I don't really buy Joe Carlsmith's
arguments for acknowledging that infinities matter in this context, same for
Toby Ord's discussion in a recent 80K podcast). What am I missing?
Have there ever been any efforts to try to set up EA-oriented funding
organisations that focus on investing donations in such a way as to fund
high-utility projects in very suitable states of the world? They could be pure
investment vehicles that have high expected utility, but that lose all their
money by some point in time in the modal case.
The idea would be something like this:
For a certain amount of dollars, to maximise utility, to first order, one has to
decide how much to spend on which causes and how to distribute the spending over
However, with some effort, one could find investments that pay off conditionally
on states of the world where specific interventions might have very high
utility. Some super naive examples would be a long-dated option structure that
pays off if the price for wheat explodes, or a CDS that pays off if JP Morgan
collapses. This would then allow organisations to intervene through targeted
measures, for example, food donations.
This is similar to the concept of a “tail hedge” - an investment that pays off
massively when other investments do poorly, that is when the marginal utility of
owning an additional dollar is very high.
Usually, one would expect such investments to carry negatively, that is, to be
costly over time possibly even with negative unconditional expected returns.
However, if an EA utility function is sufficiently different from a typical
market participant, this need not be the case, even in dollar terms (?).
Clearly, the arguments here would have to be made a lot more rigorous and
quantitative to see whether this might be attractive at all. I’d be interested
in any references etc.
Steelmanning is typically described as responding to the “strongest” version of
an argument you can think of.
Recently, I heard someone describe it a slightly different way, as responding to
the argument that you “agree with the most.”
I like this framing because it signals an extra layer of epistemic humility: I
am not a perfect judge of what the best possible argument is for a claim. In
fact, reasonable people often disagree on what constitutes a strong argument for
a given claim.
This framing also helps avoid a tone of condescension that sometimes comes with
steelmanning. I’ve been in a few conversations in which someone says they are
“steelmanning” some claim X, but says it in a tone of voice that communicates
* The speaker thinks that X is crazy.
* The speaker thinks that those who believe X need help coming up with a sane
justification for X, because X-believers are either stupid or crazy.
It’s probably fine to have this tone of voice if you’re talking about flat
earthers or young earth creationists, and are only “steelmanning” X as a silly
intellectual exercise. But if you’re in a serious discussion, framing
“steelmanning” as being about the argument you "agree with the most" rather than
the "strongest" argument might help signal that you take the other side
Anyone have thoughts on this? Has this been discussed before?
I think we separate causes and interventions into "neartermist" and
"longtermist" causes too much.
Just as some members of the EA community have complained that AI safety is
pigeonholed as a "long-term" risk when it's actually imminent within our
lifetimes, I think we've been too quick to dismiss conventionally
"neartermist" EA causes and interventions as not valuable from a longtermist
perspective. This is the opposite failure mode of surprising and suspicious
convergence - instead of assuming (or rationalizing) that the spaces of
interventions that are promising from neartermist and longtermist perspectives
overlap a lot, we tend to assume they don't overlap at all, because it's more
surprising if the top longtermist causes are all different from the top
neartermist ones. If the cost-effectiveness of different causes according to
neartermism and longtermism are independent from one another (or at least
somewhat positively correlated), I'd expect at least some causes to be valuable
according to both ethical frameworks.
I've noticed this in my own thinking, and I suspect that this is a common
pattern among EA decision makers; for example, Open Phil's "Longtermism" and
"Global Health and Wellbeing" grantmaking portfolios don't seem to overlap.
Consider global health and poverty. These are usually considered "neartermist"
causes, but we can also tell a just-so story about how global development
interventions such as cash transfers might also be valuable from the perspective
* People in extreme poverty who receive cash transfers often spend the money on
investments as well as consumption. For example, a study by GiveDirectly
found that people who received cash transfers owned 40% more durable goods
(assets) than the control group. Also, anecdotes show that cash transfer
recipients often spend their funds on education for their kids (a type of
human capital investment), starting new businesses, building infrastructure
Load more (8/21)
status: very rambly. This is an idea I want to explore in an upcoming post about
longtermism, would be grateful to anyone's thoughts. For more detailed context,
see https://plato.stanford.edu/entries/time/ for debate on the nature of time in
Does rejecting longtermism require rejecting the B-Theory of Time (i.e.
eternalism, the view that the past, present, and future have the same
ontological status)? Saying that future people don't exist (and therefore can't
be harmed, can't lose out by not existing, don't have the same moral rights as
present 'existing' people) also implies that the future they live in (i.e. the
city, country, planet, solar system, galaxy etc) doesn't exist. This seems to
fly in the face of our best understandings of physics (i.e. Relativity and
Quantum Physics don't have a place for a special 'present' afaik, in fact
Relativity suggests that there is no such thing as a universal present) I'm
not sure people grapple with that contradiction.
Furthermore, if you want to claim the special 'present' moment does exist, and
that our moral obligations only hold in that frame of time, doesn't that mean
that our obligations to the past don't matter? Many have criticised Bostrom's
use of 'mere ripples' to describe historical atrocities, but doesn't
believing in a unique present moment imply that the past doesn't exist, and
hence those atrocities don't exist either and that we have no obligations that
refer to past events or people? One could get around this by assuming a 'growing
block' theory of time, where the past exists, as does the present, but the
future doesn't. But if you believe in a special, unique present then asserting
the past exists but the future doesn't seems no more valid than the other way
I actually think that has legs as more than a niche philosophical point, but
would be very interested in hearing others' thoughts.
Physicists please correct me if I've made egregious mistakes of