Some thoughts about using the "random option" as the default:

1. Under increasing model ambiguity, evenly allocating your portfolio maximizes the minimum expected value. Now, a random option is not a deterministic even allocation across the options. But if our resoucres (money, time, attention/motivation) are divisible, then randomly assiging each minimum unit will tend to approximate the even allocation, and increasingly so the more we divide the units, by the law of large numbers.^[1]
2. If the random option is taken to be a random micro-action, e.g. a small muscle contraction, a fraction of a second of thought, then almost all of the random options are basically just noise, sequentially uncoordinated, and achieve nothing. In practice, the random option would be functionally equivalent to "Do nothing".

1. ^{^}

   Obviously dividing your time totally randomly into tiny non-contiguous units is horrible for actually achieving anything. Maybe we just combine them into bigger contiguous blocks by assumption. Or we allow some kind of cooperation or positive-sum trades that will often in practice lead to contiguous blocks of time.

Do you (Michael) see your views about precise and imprecise credences significantly affecting what you would actually do in the real world in a scenario where you had to blame Jones or Smith?

Probably not. I see it as more illustrative of important cases. Imagine instead it's between supporting an intervention or not, and it has similar complexity and considerations going in each direction.

More relevant examples to us could be: crops vs nature for wild animals, climate change on wild animals, fishing on wild animals, the far future effects of our actions, the acausal influence of our actions. These are all things I feel clueless enough about to mostly bracket away and ignore when they are side effects of direct interventions I'm interested in supporting. I'm not ignoring them because I think they're small. I think they are likely much larger than the effects I'm not ignoring.

I may also want to further study some of them, but I'm often not that optimistic about making much progress (especially for far future effrcts and acausal influence) and for that progress to be used in a way that isn't net negative overall by my lights.

If I asked you to actually decide who's more likely to be the culprit, how would you do it?

What do you do if you don't have reference class information for each part of the problem? How do you weigh the conflicting evidence? I'm imaginging that at many steps, you'd have to rely on direct impressions or numbers that just came to mind.

Would you feel like whatever came out was very arbitrary and depended too much on direct impressions or numbers that just came to mind? Would you actually believe and endorse what came out? Would you defend it to other people?

Some questions here are whether 50-50 as precise probabilities to start is reasonable and whether the approach to assign 50-50 as precise probabilities is reasonable.

If, when looking at the scenario, you would have done something like "wow, that's so complicated and I'm clueless, so 50-50", then your reaction almost certainly would have been the same if the example originally included one extra eyewitness in favour of one side. But then this tells you your initial way to assign credences was insensitive to this small difference. And yet after the initial assignment, you say it should be sensitive.

Or, if you forgot your initial judgement or the number of eyewitnesses and was just given the total and looked at the situation with fresh eyes, you'd come up with 50-50 again.

Alternatively, you could build a precise probability distribution as a function of the evidence that weighs it all, but this would be very sensitive to arbitrary choices.

In some cases, we can't gather strong enough evidence, say because:

1. they’re questions about very speculative or unprecedented possibilities and the evidence would either be too indirect and weak or come too late to be very action-guiding, e.g. often for AI risk, conscious subsystems, or
2. there will be too much noise or confounding, too small a sample size and anything like an RCT is too impractical (e.g. policy, corporate outreach) or wouldn't generalize well, or
3. the disagreements are partly conceptual, definitional or philosophical, e.g. "What is consciousness?", "What is the hedonic intensity of an experience?"
4. EDIT: generally, the window to intervene is too small to wait for the evidence.

In such cases, I think imprecise probabilities are the way to go to reduce arbitrariness. We can do sensitivity analysis. If whether the intervention looks good or bad overall depends highly on fairly arbitrary judgements or priors, we might disprefer it and prefer to support things that are more robustly positive. This is difference-making ambiguity aversion.

And/or we do can some kind of bracketing.

Also, you should think of research as an intervention itself that could backfire. Who could use the research, and could they use it in ways you'd judge as very negative? How likely is that? This will of course depend on the case and your own specific views.

It depends on the case. Do you think my answer to the above should influence which interventions I prioritise? My current top recommendations are research on i) the welfare of soil animals and microorganisms, and ii) comparisons of (expected hedonistic) welfare across species and digital systems. Could you see these changing if I thought EVs were imprecise instead of precise at a fundamental level?

I think there's a lot that could change if you very seriously weighed others' actual or possible direct impressions/intuitions without heavily privileging your own, before we even get into the question of precise vs imprecise credences. Epistemic modesty is going to do a lot of work first.

1. Holding your current normative views ~constant, with precise credences, then epistemic modesty would make infinite expected values (and possibly cardinally larger infinities) your focus, as long as there are well-defined consistent ways to handle them without always getting infinity minus infinity errors in practice. With imprecise credences, you could plausibly justify ignoring them on some versions of bracketing (also see here), say because they're so speculative and you're clueless about the direction of your impacts on infinities, including possibly even the effects of research into infinite effects (because the research could be used in ways you'd judge to be very bad).
2. (Independently of precise vs imprecise) If you're a moral realist, then you wouldn't privilege your own direct normative intuitions just for being yours either, and this would plausibly mean not privileging consequentialism, utilitarianism, hedonism, risk neutrality, etc.. This could have important implications. Your current priorities might still be among your top priorities, but your list of priorities could expand a lot.
   1. It might be impossible to compare these priorities; there's no universal common standard/unit across all normative stances. You might go for a portfolio of interventions.
   2. If you're not a moral realist, or for the part of you that isn't, you can just not care about views that conflict too much with your most important intuitions.
3. If you're doing some version of bracketing with imprecise credences, some vertebrate welfare work could be worth prioritizing. I'm clueless about whether crops or nature is better for wild animals, even though I'm suffering-focused, so I ignore conversions between nature and crops. Far future effects and acausal influence could guide some priorities unless you're clueless about them and bracket them away.
   1. Again, potentially impossible comparisons + portfolio.
4. With imprecise credences, I think you would also be more pessimistic about the marginal value of research to compare welfare ranges and sentience across types of possible moral patients. You should also be more pessimistic about the value of further research into the sign of the welfare of moral patients. That doesn't mean no such research is worth doing, but I think it would focus on scoping out possibilities and their implications and gathering evidence that could basically rule out the more extreme hypotheses (e.g. for (near-)constant welfare ranges and for welfare ranges with the most extreme ratios between potential moral patients). Arguments like the two envelopes problem, conscious subsystems, how moral weights could scale with neuron counts, gradations/vagueness, looking for more ways to assign welfare ranges with very different implications from the ones we have now. If you're gathering empirical evidence, you would aim it at shifting or ruling out extremes.
   1. Personally, I've decided to draw some lines in practice, and basically leave out nematodes and simpler systems as priorities. This depends largely on my normative views (and I'm not a moral realist, so I'm more willing to make some judgement calls about this). I think what counts as consciousness is largely normative and subjective, I have some objections to aggregation (e.g. torture vs dust specks) and I'm not entirely risk neutral or ambiguity neutral. The capacities I've observed in them don't seem so compelling. Maybe some of it is motivated reasoning, though. And maybe some sentience research on nematodes would be worth doing. If they met some of the standards here or here or we found evidence for some of the most sophisticated cognitive capacities we observe in fruit flies, I might take them pretty seriously.

I think you've simplified the problem too much. There can be special cases where we can use symmetry and just take simple averages, but many practical cases are not like that. Indeed, that's the point of the distinction between complex and simple cluelessness in the first place.

I think, ideally, we should look for and exploit as much evidential symmetry as possible, but I don’t think we'll always find enough of it to land on a unique precise distribution, I'd guess in principle impossible in many cases (probably almost all cases of intervention and cause area research) without further evidence.

It's true that direct impressions (e.g. internal states about the plausibility of the probabilities) could be considered evidence, but to the extent that for the same objective external evidence, these direct impressions can vary between people or depending on how or when you present the evidence, they seem arbitrary.

Would you take the fact that a direct impression came from your brain — from an inscrutable process, prone to cognitive biases of various kinds, and whose reliability you can at best verify by track records in limited domains where feedback is practical, and where track records may not generalize across tasks and domains well — is better evidence than a direct impression from another person's brain (with similar problems), with access to the same objective external evidence?

Or, what if there are multiple people with different distributions and different track records in relevant domains? How do you weigh them? How much should track record be worth? EDIT: What if their track records are measured in different ways, e.g. you have forecasters with Brier scores, investors or betters with measures of their gains and losses, researchers and grantmakers of various seniorities at different organizations?

And what's the range of direct impressions humans or other semi-rational agents could have, and how would you weigh them all?

I'd also be keen to get your response to this (and also this, if you have the time.)

Do you think it's reasonable for two people with all of the same evidence to disagree on precise probabilities and expected values? If so, how would you justify picking your own precise probabilities over someone else's, if you think theirs are just as defensible? 

Or would you just average yours and theirs in some way to get a new distribution? How?

And how far would you go, if you consider all the defensible precise probability distributions anyone could assign (whether or not anyone actually does so)? How do you weigh them all if there are infinitely many of them and no uniform distribution over them?

Here's another example I like.

How would you choose the distributions for the model weights in a way that's not itself arbitrary? E.g. how do you choose their forms and parameters in a way that's not arbitrary?

I do think imprecise credences have a similar problem of deciding which distributions to include in their representor. I think ultimately we need to make some arbitrary choices and should accept some, but we can be more or less arbitrary, or stop when it's no longer decision-relevant. Maybe sometimes we can hit a fixed point or see some kind of convergence in the extra steps we're taking.

On there potentially being no fact of the matter, this may be helpful. It goes further than the issue of imprecise credences/EVs.

On the nematode example, it could go further than that: we might assign an imprecise credence between X and 100% to a set of standards for sentience that nematodes don't meet (see my other post on gradations of moral weight). So, the ratio could be anywhere between 0 and 1 (assuming we're taking the absolute value, or only consider same-sign valence).

If the ratio is anywhere between 0 and 1, then whenever we're looking at affecting nematode-seconds relative to their welfare ranges more than human-seconds relative to our welfare ranges, it would be indeterminate which is affected more. I think that would be every time in practice.

If we don't need to deal with gradations/vagueness like this, then I would probably assign expected welfare ranges (conditional on sentience) between constant and roughly proportional to the number of neurons, and this could give many more practically useful comparisons. EDIT: although conscious subsystems makes me more inclined towards approximately proportional, if we’re entertaining nematode sentience.