Bio

Participation
4

I am a generalist quantitative researcher. I am open to volunteering and paid work. I welcome suggestions for posts. You can give me feedback here (anonymously or not).

How others can help me

I am open to volunteering and paid work. I welcome suggestions for posts. You can give me feedback here (anonymously or not).

How I can help others

I can help with career advice, prioritisation, and quantitative analyses.

Comments
3256

Topic contributions
42

I see. Thanks for clarifying. Below is how Claude thinks Adam (the author of the article) would object to your comments. The objections make sense to me. Any reactions?

The unifying objection: the four-option reframe is one of the three rules

Evan's central claim is that he can dissolve the puzzle without NARROW, PLAN, or SEQUENCE: treat the situation not as two decisions (A yes/no, B yes/no) but as one choice among four policies — {A-only, B-only, BOTH, NEITHER} — notice BOTH statewise-dominates NEITHER, delete NEITHER, and you're done. He stresses "I didn't even consider the probability of H."

Elga's first reply is that this is exactly SEQUENCE (or PLAN) wearing plain clothes — and Evan concedes it in his own EDIT ("I think this is close in effect to SEQUENCE"). Evaluating the pair of choices as a single ex-ante object over sequences is the defining move of the global rules. So "I don't need any of the three" is false: he's using the third. And that matters, because Sally is aimed precisely here. Take Evan's B-only policy: it requires rejecting A and then accepting B. Compare the agent at the B-node in two situations — one where she reached it by rejecting A, one where B is offered alone. For a money-only agent these are identical in everything she cares about, yet the reframe must call rejecting-B impermissible in the first (it would complete NEITHER) and permissible in the second. That is the SEQUENCE verdict, and it fails for the SEQUENCE reason.

Why "consider them simultaneously" doesn't reach the actual problem

Evan's sports example — decline each of the Snofuls/Fleertis bets in isolation, take both together for a sure profit — leans on "when we consider our options simultaneously, that changes the calculus." Elga's rejoinder: in his setup the bets are not simultaneous. You settle A, and only then face B. So the live question is what binds you at the B-node, where A is already done and the only comparison is accept-B (+15/−10) versus reject-B (0). With an interval straddling 40%, maximality rules both permissible. The ex-ante fact "BOTH dominates NEITHER" is true but does not, by itself, reach into the B-node and make accepting B required there. Supplying that reach is the whole job of PLAN/SEQUENCE — which is why Evan can't actually skip them.

And the boast "I didn't even need to consider P(H)" is the tell, not the triumph. Dominance eliminates NEITHER for any credence — a sharp agent excludes it too. So the four-option elimination is entirely neutral between SHARP and UNSHARP; it was never the point in dispute. The dispute is about the sequential assembly of a dominated outcome from two individually-licensed choices, and the reframe simply doesn't engage it.

The EDIT smuggles in comparability — i.e. sharpness

Evan tries to close the "what if you plan B, reject A, then reject B?" gap thus: "a rational actor does not change their mind without new information. They would only choose B if they believe B > BOTH > NEITHER. Any rational actor who believes B > NEITHER would end up betting B."

This quietly assumes a complete ordering over the options — exactly what UNSHARP denies. B-only beats BOTH only when P(H) > 60%; with the interval [10%, 80%], B-only and BOTH are incomparable under maximality, as are A-only and BOTH. So "they would only choose B if B > BOTH" presupposes the agent can rank options the way a sharp credence lets her. Grant that comparability and of course she never lands on a dominated outcome — but you've then imported enough structure that she behaves like a sharp agent, which is Elga's strict-rules horn: you buy the right behavior only by reintroducing precision and thereby forfeiting the motivation for going unsharp in the first place.

"Rational actors with less information make worse decisions" gives the game away

Evan concedes that without foreknowledge an UNSHARP agent can reject A as optional, reject B as optional, land on NEITHER, and shrug it off as an information deficit. Two problems. First, Elga's case stipulates full foreknowledge, so the no-foreknowledge scenario isn't the one under discussion. Second, and more damaging, the diagnosis "less information" is wrong. A sharp agent — even with a diffuse-but-precise prior, and even with no foreknowledge — never rejects both, because her node-by-node expected-value verdicts are automatically time-coherent (reject A only if P(H) > 60%, accept B only if P(H) > 40%, and these can't jointly fail). The unsharp agent's node verdicts are not automatically coherent: both nodes say "optional," which is what lets her assemble NEITHER. So the pathology is produced by the unsharpness, not by any information gap. Evan's concession thus admits precisely the foreseeable-domination Elga is prosecuting, and mislabels its source.

The portfolio point isn't an argument for UNSHARP

Vasco already made the core objection and Evan half-conceded it: diversification falls straight out of sharp EV reasoning with diminishing marginal returns and cross-correlations. Elga would add the sharper version: where the portfolio reasoning gives sensible verdicts ("this combination statewise-beats doing nothing"), it's dominance reasoning a sharp agent honors equally; where it gives distinctively unsharp verdicts, it does so by licensing inaction — declining each option in isolation — which is just the reject-both pathology relocated to altruistic choice. (This is the "clueless agent whose intervals stay wide because it never acts" failure mode, which is live in your own work.)

"Can vs. should" is not a dodge — it's Elga's exact target

Evan's sign-off — "whether we should have unsharp probabilities is beside the point; my argument is about whether we can have them without sacrificing rationality, and I believe we can" — doesn't sidestep Elga. UNSHARP just is the "can" claim: it is consistent with perfect rationality to be unsharp. SHARP denies that. So Evan is engaging the thesis head-on, and Elga's reply is that the "can" fails for the reasons above: every route Evan takes either collapses into SEQUENCE (Sally sinks it) or into sharp-style comparability (motivation lost).

The honest crux

Where Evan has a real point — shared with DiGiovanni and Michael St Jules — is the suspicion that node-by-node "local" evaluation is the wrong model, and that a look-ahead agent who plans the whole tree does fine with wide intervals. Elga's whole case does assume that a theory of rational credence must deliver correct verdicts at each actual choice node, not merely over ex-ante policies. Evan is, in effect, denying that assumption. But he hasn't defeated Sally independently; he's relocated to ex-ante policy choice, which Elga classifies as SEQUENCE/PLAN and which Evan himself admits is "close in effect to SEQUENCE." So the disagreement bottoms out exactly where it did in the DiGiovanni thread [this one]: whether an idealized agent is entitled to bind her future choices (resolute/sophisticated look-ahead), or whether rationality must already be satisfiable choice-by-choice. Elga bets on the latter; Evan (like DiGiovanni) needs the former — and that is the genuine open question, not something Evan's four-option reframe settles.

Because the same kind of solution is available to someone with unsharp probabilities in Elga's scenario, if you're treating them fairly.

Solution to which problem? I am not sure what is supposed to be problematic. As far as I understand, one should just commit as much as possible to maximise the chances of survival.

It doesn't require an infinite world, only that you can't be 100% confident in any finite upper bound on your impact that you specify, and that there are infinitely many ways that the world could be (due largely to not full certainty about physics).

I agree there is a probability above 0 of (counterfactual) impact being larger than X for any X. So I think impact can be arbitrarily large. However, I do not think it can be infinite. The function f(x) = x can take an arbitrarily large value, but not an infinite value (its range is the set of real numbers). The function g(x) = 1/x can take an arbitrary small value, but not a value of exactly 0 (its range is the set of real numbers besides 0).

Why can't the fact that she'd pick a dominated sequence or regret it if she rejects both bets matter to her after rejecting bet A?

It is very counterintuitive that could matter for Sally for reasons that do not have to do with money.

I think most efforts to grow EA grow both Longtermist and non-Longtermist elements of EA.

Would it make sense for you to distinguish between growing longtermist and non-longtermist elements of EA? You prioritise longtermist interventions over animal welfare and global health ones.

I do agree though that the Longtermist effects of animal welfare are much bigger than near-term benefits.

Likewise for global health interventions? If so, animal welfare and global health interventions are also longtermist in the sense their longterm benefits are much larger than their nearterm benefits? In this case, it would be better to avoid terms like "Longtermist political donations", "other Longtermist donations", and "Longtermist careers", which could refer to animal welfare and global health interventions? If I thought the longterm effects of the vast majority of interventions were much larger than their nearterm effects, I would simply refer to the areas of the interventions instead of highlighting they are longtermist.

Yeah, it's rough and there's not a perfect method.

You did not make any comparison of the longterm benefits of different areas? At this point, I am just looking for one comparison, not a perfect one. Why not defaulting to the basic intuition that the benefits over the next few decades are a good proxy for the total benefits, which suggests global health interventions are more cost-effective than ones aiming to decrease the risk of global catastrophes?

I wouldn't be that extreme [GiveWell's top charities being 2*10^-30 times as cost-effective as some longtermist interventions], but I have some credence that something like that extreme of a tradeoff is right.

I doubt GiveWell's top charities are 2*10^-30 times as cost-effective as some longtermist interventions.

Don't think we need one for those sorts of things, just like we don't need a quantitative model suggesting, say, the odds of Vance being the next president are around 30%.

There is huge disagreement about the risk of extinction due to AI. Below are the predictions from superforecasters made in the Existential Risk Persuasion Tournament (XPT). There is nothing like that disagreement about the probability of Vance being the next president of the United States (US) assuming he is one of the 2 final candidates, which makes huge deviations from 50 % extremely unlikely.

I think we've talked elsewhere about why I don't buy that soil nematodes etc make us totally clueless about which animal welfare interventions are good.  

The last point I made in my comment was not about this. I meant that global health interventions may be much more cost-effective than animal welfare interventions even neglecting effects on soil invertebrates (like nematodes).

Hi Anthony. Thanks. I followed up on LessWrong.

A rational actor doesn't need NARROW, PLAN, or SEQUENCE. They need to consider the future: "Bet B is coming, so there's an arbitrage opportunity regardless of the probability."

I do not seem to understand. If one knew "Bet B is coming", one would know about the full set up in advance as in the post ("You're told the full setup in advance"). So rejecting both A and B would not make sense?

Hi Michael.

There are other cases where you should make commitments that you would later be inclined to break, like Parfit's hitchhiker, and St. Petersburg lotteries with unbounded utility functions.

Why does Parfit's hitchhiker pose a problem? I would think my chance of survival is equal to my chance of keeping the commitment. So I would simply aim to commit as much as possible if I wanted to maximise my chances of survival. I understand the dilemma is that it would make sense for me to break the committment after I was driven to town, but my decision and thoughts in the town would be constrained from my chat with the driver in the desert. If the driver predicted I was 90 % likely to keep the commitment, and their predictions were calibrated, I would be 90 % likely to keep the commitment, and my thoughts would have to be compatible with this? If the driver predicted I was certain to keep the commitment, I would not consider breaking it in town? Otherwise, the predictions of the driver would not be accurate, which violates the set up of the thought experiment? Here is the description of the thought experiment for readers' context.

You are stranded in the desert, running out of water, and soon to die. Someone in a motor vehicle drives up to you. The driver of the motor vehicle is a selfish ideally game-theoretical agent, and what's more, so are you. Furthermore, the driver is Paul Ekman who has spent his whole life studying facial microexpressions and is extremely good at reading people's honesty by looking at their faces.

The driver says, "Well, as an ideal selfish rational agent, I'll convey you into town if it's in my own interest to do so. I don't want to bother dragging you to Small Claims Court if you don't pay up. So I'll just ask you this question: Can you honestly say that you'll give me $1,000 from an ATM after we reach town?"

[...]

We may assume that Parfit's driver also asks you questions like "Have you really thought through what you'll do?" and "Are you trying to think one thing now, knowing that you'll probably think something else in the city?" and watches your facial expression on those answers as well.

The St. Petersburg paradox involves an infinite expected payoff, and I reject infinite worlds.

Furthermore, "imposes different requirements across two situations Sally can see are identical in everything she cares about". What if I do care about the differences?

Adam's argument holds as long as, given 2 sharp states of the world A and B, A is better, worse, or as good as B? In Sally's case, her money is the only thing that matters. For realistic cases, many other factors will contribute to the value of a sharp world state.

Also, here's another way someone with unsharp probabilities might handle this situation. In summary, I should accept bet A at the start to rule out the possibility of picking a dominated sequence

I understand one should accept bet A based on that strategy. However, unsharp probabilities are supposed to allow for accepting or rejecting A?

Hello Evan.

At least in practice, there's a clear difference between considering bet A in isolation and considering bet A when you know bet B is coming. If you told me about a sports game between the Snofuls and the Fleertis and offered me 2:1 odds on the Snofuls to win, I wouldn't take it. But if you told me you would also give me 2:1 odds on the Fleertis to win, I would take both bets, guaranteeing a profit.

Accepting the 1st bet if you were confident Snofuls would win, accepting the 2nd if you were confident Fleertis would win, and accepting both if you thought the probability of any of the teams winning was close to 50 % would be in agreement with sharp probabilities.

Elga grants you're not required to accept both (a very confident or very doubtful agent might prefer just one). But he insists on the key premise: a rational agent must accept at least one of the two bets, because rejecting both is dominated—it's worse than accepting both in every outcome, and you can see this in advance. This premise is easy for a sharp-credence theorist to honor. The rest of the paper argues that no version of the unsharp view can.


As a rational actor with no useful information, I have a very broad range of potential probabilities for this bet, and it is permissible to do neither bet in isolation. However, when we consider our options simultaneously, that changes the calculus.

Which of the 3 strategies described by Adam would you use to justify accepting or rejecting each bet in isolation, but rejecting both bets together?

To apply this to altruistic action, there might be actions that we are uncertain about in isolation, but we are willing to pursue as a part of a portfolio approach.

This is not an argument for unsharp probabilities? Supporting a portfolio of interventions makes sense even with sharp probabilities. Marginal cost-effectiveness tends to decrease with spending. For example, if the Animal Welfare Fund (AWF) had granted 2 times as much to all the grantees they supported in 2025, I expect the impact of the grants would have been larger, but less than 2 times as large.

Hi Matthew. Nice to know how you are thinking about this.

TLDR: my ranking for donations: Longtermist political donations>growing EA>other Longtermist donations>animal welfare>global health.

My ranking for careers: Growing EA=Longtermist careers>animal welfare>global health.

By growing EA, you mean growing longtermist EA? What makes a donation or career longtermist? If you think the longterm benefits of decreasing the risk of human extinction over the next few 10 years are much larger than its nearterm benefits, you should also think the longterm benefits of animal welfare and global health interventions are much larger than their nearterm benefits? If so, how do you compare the longterm benefits in a principled way? You estimated "each dollar [donated to longtermist interventions] increases the number of well-off future people in expectation by 10^26", and GiveWell's top charities save around 2*10^-4 lives per $ (= 1/(5*10^3)). However, I assume your best guess for the reduction in existential risks would not have to be less than 2*10^-30 (2*10^-4/10^26) times as high, i.e. less than 2*10^-32 pp (instead of your assumption of 0.01 pp), for you to prioritise global health over longtermist interventions.

1.1 The case for Longtermist interventions being best

You assumed decreasing "existential risks" by 0.01 pp (you said ".01%", which is different) would increase the value of the future (10^40 human lives) by 0.01 %, resulting in 10^36 human lives of expected benefits. How do you justify that assumption? I do not think it is valid.

Experts consistently think there’s a several percent chance that misaligned AI kills everyone.

Are you aware of any quantitative model suggesting this? I am only aware of almost purely subjective guesses, and products of these.

These tend to be super effective: dollars given directly to Giving What We Can [GWWC's] return about 6 dollars to effective charities and dollars given to Effektiv Spenden return about 13 dollars to effective charities.

GWWC estimated their giving multiplier in 2025 was 7. You linked to an analysis I did of their giving multiplier in 2023-2024. GWWC estimated this was 6, and I got values of 8 to 9.

3.1 The basic case for animal welfare

I believe you are underestimating the uncertainty involved in welfare comparisons across species. I would say animal welfare interventions may increase the welfare of their target beneficiaries much more or less cost-effectively than global health interventions. Here are my estimates assuming sentience-adjusted welfare ranges proportional to "individual number of neurons"^"exponent", and "exponent" from 0 to 2, which covers the best guesses than I consider reasonable. For an exponent of 2 (which means the sentience-adjusted welfare ranges decrease 2 % for a 1 % reduction in the individual number of neurons), I estimate GiveWell's top charities increase the welfare of humans 135 times as cost-effectively as cage-free egg corporate campaigns increase the welfare of chickens.

Load more