All of slicedonions&#x27;s Comments + Replies

Human utility functions seem clearly inconsistent with infinite utility.

If you're not 100% sure that they are inconsistent then presumably my argument is still going to go through, because you'll have a non-zero credence that actions can elicit infinite utilities and so are infinite in expectation?

I don't identify 100% with future versions of myself, and I'm somewhat selfish, so I discount experiences that will happen in the distant future. I don't expect any set of possible experiences to add up to something I'd evaluate a

... (read more)

PeterMcCluskey

I think it's more appropriate to use Bostrom's Moral Parliament to deal with conflicting moral theories. Your approach might be right if the theories you're comparing used the same concept of utility, and merely disagreed about what people would experience. But I expect that the concept of utility which best matches human interests will say that "infinite utility" doesn't make sense. Therefore I treat the word utility as referring to different phenomena in different theories, and I object to combining them as if they were the same. Similarly, I use a dealist approach to morality. If you show me an argument that there's an objective morality which requires me to increase the probability of infinite utility, I'll still ask what would motivate me to obey that morality, and I expect any resolution of that will involve something more like Bostrom's parliament than like your approach.

I've found the conversation productive, thanks for taking the time to discuss.

My intuitive response is that that is an incomplete definition and we would also need to say what impartial reasons are, otherwise I don't know how to identify the impartial reasons.

Impartial reasons would be reasons that would 'count' even if we were some sort of floating consciousness observing the universe without any specific personal interests.

I probably don't have any more intuitive explanations of impartial reasons than that, so sorry if it doesn't convey my meaning!

Rohin Shah

My math-intuition says "that's still not well-defined, such reasons may not exist". To which you might say "Well, there's some probability they exist, and if they do exist, they trump everything else, so we should act as though they exist." My intuition says "But the rule of letting things that could exist be the dominant consideration seems really bad! I could invent all sorts of categories of things that could exist, that would trump everything I've considered so far. They'd all have some small probability of existing, and I could direct my actions any which way in this manner!" (This is what I was getting at with the "meta-oughtness" rule I was talking about earlier.) To which you might say "But moral reasons aren't some hypothesis I pulled out of the sky, they are commonly discussed and have been around in human discourse for millennia. I agree that we shouldn't just invent new categories and put stock into them, but moral reasons hardly seem like a new category." And my response would be "I think moral reasons of the type you are talking about mostly came from the human tendency to anthropomorphize, combined with the fact that we needed some way to get humans to coordinate. Humans weren't likely to just listen to rules that some other human made up, so the rules had to come from some external source. And in order to get good coordination, the rules needed to be followed, and so they had to have the property that they trumped any prudential reasons. This led us to develop the concept of rules that come from some external source and trump everything else, giving us our concept of moral reasons today. Given that our concept of "moral reasons" probably arose from this sort of process, I don't think that "moral reasons" is a particularly likely thing to actually exist, and it seems wrong to base your actions primarily on moral reason. Also, as a corollary, even if there do exist reasons that trump all other reasons, I'm more likely to reject the intuition that i

My main claim is that properties 1 and 2 need not be correlated, whereas you seem to have the intuition that they are, and I'm pushing on that.

I do think they are correlated, because according to my intuitions both are true of moral reasons. However I wouldn't want to argue that (2) is true because (1) is true. I'm not sure why (2) is true of moral reasons. I just have a strong intuition that it is and haven't come across any defeaters for that intuition.

A secondary claim is that if it does not satisfy property 3, then you can never i

... (read more)

Rohin Shah

Okay, cool, I think I at least understand your position now. Not sure how to make progress though. I guess I'll just try to clarify how I respond to imagining that I held the position you do. From my perspective, the phrase "moral reason" has both the connotation that it is external to humans and that it trumps all other reasons, and that's why the intuition is so strong. But if it is decomposed into those two properties, it no longer seems (to me) that they must go together. So from my perspective, when I imagine how I would justify the position you take, it seems to be a consequence of how we use language. My intuitive response is that that is an incomplete definition and we would also need to say what impartial reasons are, otherwise I don't know how to identify the impartial reasons.

Pursuing infinite positive utility at any cost

There seems to be something that makes you think that moral reasons should trump prudential reasons.

The reason I have is in my original post. Namely I have a strong intuition that it would be very odd to say that someone who had done what there was most moral reason to do had failed to do what there was most 'all things considered' reason for them to do.

If my intuition here is right then moral reasons must always trump prudential reasons. Note I don't have anything more to offer than this intuition, sorry if I made it seem like I did!

On you... (read more)

Rohin Shah

I did mean for you to replace X with a phrase, not a number. Your intuition involves the complex phrase "moral reason" for which I could imagine multiple different interpretations. I'm trying to figure out which interpretation is correct. Here are some different properties that "moral reason" could have: 1. It is independent of human desires and goals. 2. It trumps all other reasons for action. 3. It is an empirical fact about either the universe or math that can be derived by observation of the universe and pure reasoning. My main claim is that properties 1 and 2 need not be correlated, whereas you seem to have the intuition that they are, and I'm pushing on that. A secondary claim is that if it does not satisfy property 3, then you can never infer it and so you might as well ignore it, but "irreducibly normative" sounds to me like it does not satisfy property 3. Here are some models of how you might be thinking about moral reasons: a) Moral reasons are defined as the reasons that satisfy property 1. If I think about those reasons, it seems to me that they also satisfy property 2. b) Moral reasons are defined as the reasons that satisfy property 2. If I think about those reasons, it seems to me that they also satisfy property 1. c) Moral reasons are defined as the reasons that satisfy both property 1 and property 2. My response to a) and b) are of the form "That inference seems wrong to me and I want to delve further." My response to c) is "Define prudential reasons as the reasons that satisfy property 2 and not-property 1, then prudential reasons and moral reasons both trump all other reasons for action, which seems silly/strange."

Pursuing infinite positive utility at any cost

Yep that seems right, though you might want more than one believer in each in case one of the assigned people messes it up somehow.

Thanks @trammell.

Will read up on stochastic dominance, will presumably bring me back to my mirco days thinking about lotteries...

Note that I think there may be a way of dealing with it whilst staying in the expected utility framework. Where we ignore undefined expected utilities as they are not action guiding. Instead we focus on the part of our probability spaces where they don't emerge. In this case I suggest we should only focus on worlds in which you can't have both negative and positive infinities. We'd assume in our analysis that only ... (read more)

trammell

I was just saying that, thankfully, I don’t think our decision problem is wrecked by the negative infinity cases, or the cases in which there are infinite amounts of positive and negative value. If it were, though, then okay—I’m not sure what the right response would be, but your approach of excluding everything from analysis but the “positive infinity only” cases (and not letting multiple infinities count for more) seems as reasonable as any, I suppose. Within that framework, sure, having a few thousand believers in each religion would be better than having none. (It’s also better than having everyone believe in whichever religion seems most likely, of course.) I was just taking issue with “it might be best to encourage as many people as possible to adopt some form of religious belief to maximise our chances”.

Agreed that unguided evolution might give us generally reliable cognitive faculties. However, there is no obvious story we can give for how our cognitive faculties would have access to moral facts (if they exist). Moral facts don't interact with the world in a way that gives humans the way to ascertain them. They're not like visual data where reflected/emitted photons can be picked up by our eyes. So it's not clear how information about them would enter into our cognitive system?

I'd be interested if you have thoughts on a mechanism whereby information about moral facts could enter our cognitive systems?

Tiny Probabilities of Vast Utilities: A Problem for Long-Termism?

slicedonions5y4

Thanks for the really thoughtful engagement.

I don't know how to argue against this, you seem to be taking it as axiomatic.

I agree, my view stems from a bedrock of intuition, that just as the descriptive fact that 'my table has four legs' won't create normative reasons for action, neither will the descriptive fact that 'Harry desires chocolate ice-cream' create them either. It doesn't seem obvious to me that the desire fact is much more likely to create normative reasons than the table fact. If we don't think the t... (read more)

Rohin Shah

There seems to be something that makes you think that moral reasons should trump prudential reasons. The overall thing I'm trying to do is narrow down on what that is. In most of my comments, I've thought I've identified it, and so I argued against it, but it seems I'm constantly wrong about that. So let me try and explicitly figure it out: How much would you agree with each of these statements: * If there is a conflict between moral reasons and prudential reasons, you ought to do what the moral reasons say. * If it is an empirical fact about the universe that, independent of humans, there is a process for determining what actions one ought to take, then you ought to do what that process prescribes, regardless of what you desire. * If it is an empirical fact about the universe that, independent of humans, there is a process for determining what actions to take to maximize utility, then you ought to do what that process prescribes, regardless of what you desire. * If there is an external-to-you entity satisfying property X that prescribes actions you should take, then you ought to do what it says, regardless of what you desire. (For what value of X would you agree with this statement?) I also have a very low credence of that meta-normative rule. I meant to contrast it to the meta-normative rule "binding oughtness trumps regular oughtness", which I interpreted as "moral reasons trump prudential reasons", but it seems I misunderstood what you meant there, since you mean "binding oughtness" to apply both to moral and prudential reasons, so ignore that argument. This makes me mildly worried that you aren't able to imagine the worldview where prudential reasons exist. Though I have to admit I'm confused why under this view there are any normative reasons for action -- surely all such reasons depend on descriptive facts? Even with religions, you are basing your normative reasons for action upon descriptive facts about the religion. (Btw, random note, I suspect tha

Tiny Probabilities of Vast Utilities: A Problem for Long-Termism?

So my claim I'm trying to defend here is not that we should be willing to hand over our wallet in Pascal's mugging cases.

Instead its a conditional claim that if you are the type of person who finds the Mugger's argument compelling then then the logic which leads you to find it compelling actually gives you reason not to hand over your wallet as there are more plausible ways of attempting to elicit the infinite utility than dealing with the mugger.

Rohin Shah

I see, that makes sense, and I agree with it.

Thanks for the interesting post. One thought I have is developed below. Apologies that it only tangentially relates to your argument, but I figured that you might have something interesting to say.

Ignoring the possibility of infinite negative utilities. All possible actions seem to have infinite positive utility in expectation. For all actions have a non-zero chance of resulting in infinite positive utility. For it seems that for any action there's a very small chance that it results in me getting an infinite bliss pill, or to go Pascal's route t... (read more)

Rohin Shah

I and most other people (I'm pretty sure) wouldn't chase the highest probability of infinite utility, since most of those scenarios are also highly implausible and feel very similar to Pascal's mugging.

I think your argument is that we should ignore worlds without a binding oughtness.

Agreed, I'm just using 'binding oughtness' here as a (hopefully) more intuitive way of fleshing out what I mean by 'normative reason for action'.

But in worlds without a binding oughtness, you still have your own desires and goals to guide your actions. This might be what you call 'prudential' reasons

So I agree that if there are no normative reasons/'binding oughtness' then you would still have your mere desires. However these jus... (read more)

Rohin Shah

I don't know how to argue against this, you seem to be taking it as axiomatic. The one thing I can say is that it seems clearly obvious to me that your desires and goals can make some actions better to choose than others. It only becomes non-obvious if you expect there to be some external-to-you force telling you how to choose actions, but I see no reason to assume that. It really is fine if you're actions aren't guided by some overarching rule granted authority by virtue of being morality. But I suspect this isn't going to convince you. Can we simply assume that prudential reasons exist and figure out the implications? Thanks, I think I've got it now. (Also it seems to be in your appendix, not sure how I missed that before.) I know, and I think in the very next paragraph I try to capture your view, and I'm fairly confident I got it right based on your comment. This seems tautological when you define morality as "binding oughtness" and compare against regular oughtness (which presumably applies to prudential reasons). But why stop there? Why not go to metamorality, or "binding meta-oughtness" that trumps "binding oughtness"? For example, "when faced with uncertainty over ought statements, choose the one that most aligns with prudential reasons". It is again tautologically true that a person who does what there is most metamoral reason to do could not have failed to do what there was most all things considered reason for them to do. It doesn't sound as compelling, but I claim that is because we don't have metamorality as an intuitive concept, whereas we do have morality as an intuitive concept.

Yep what you suggest I think isn't far from the fact. Though note I'm open to the possibility of normative realism being false, it could be that we are all fooled and that there are no true moral facts.

I just think this question of 'what grounds this moral experience' is the right one to ask. On the way you've articulated it I just think your mere feelings about behaviours don't amount to normative reasons for action, unless you can explain how these normative properties enter the picture.

Note that normative reasons are weir... (read more)

Ben Pace

This is a really nice way of formulating the critique of the argument, thanks Max. It makes me update considerably away from the belief stated in the title of my post.

To capture my updated view, it'd be something like this: for those who have what I'd consider a 'rational' probability for theism (i.e. between 1% and 99% given my last couple of years of doing philosophy of religion) and a 'rational' probability for some mind-dependent normative realist ethics (i.e. between 0.1% and 5% - less confident here) then the result of my argument is that a substantial proportion of an agent's decision space should be governed by what reasons the agent would face if theism were true.

Another way to end up with reliable moral beliefs would be if they do provide an evolutionary benefit.

I wholeheartedly agree with this. However there is no structural reason to think that most possible sets of moral facts would have evolutionary benefit. You outline one option where there would be a connection, however that this is the story behind morality would be surprisingly lucky on our part.

We would also need to acknowledge the possibility that evolution has just tricked us into thinking that common sense morality is correct when really moral facts ... (read more)

My view is broadly that if reasons for action exist which create this sort of binding 'oughtness' in favour of you carrying out some particular thing, then there must be some story about why this binding oughtness applies to some things and not others.

It's not clear to me that mere human desires/goals are going to generate this special property that you now ought to do something. We don't think that the fact that 'my table has four legs results' in itself generates reasons for anyone to do anything, so why should the fact that... (read more)

Rohin Shah

With that terminology, I think your argument is that we should ignore worlds without a binding oughtness. But in worlds without a binding oughtness, you still have your own desires and goals to guide your actions. This might be what you call 'prudential' reasons, but I don't really understand that term -- I thought it was synonymous with 'instrumental' reasons, but taking actions for your own desires and goals is certainly not 'instrumental'. So it seems to me that in worlds with a binding oughtness that you know about, you should take actions according to that binding oughtness, and otherwise you should take actions according to your own desires and goals. You could argue that binding oughtness always trumps desires and goals, so that your action should always follow the binding oughtness that is most likely, and you can put no weight on desires and goals. But I would want to know why that's true. Like, I could also argue that actually, you should follow the binding meta-oughtness rule, which tells you how to derive ought statements from is statements, and that should always trump any particular oughtness rule, so you should ignore all of those and follow the most likely meta-oughtness rule. But this seems pretty fallacious. What's the difference?

slicedonions5y-3

This is my response to your meta-level response.

I don't trust the intellectual tradition of this argumentative style.

It's not obvious that anyone's asking you to trust anything? Surely those offering arguments are just asking you to assess an argument on its merits, rather than by the family of thinkers the argument emerges from?

But my impression of modern apologetics is primarily one of rationalization, not the source of religion's understanding of meaning, but a post-facto justification.

I'm reasonably involved in the apologeti... (read more)

Thanks Ben! I'll try and comment on your object level response in this comment and your meta level response in another.

Alas I'm not sure I properly track the full extent of your argument, but I'll try and focus on the parts that are trackable to me. So apologies if I'm failing to understand the force of your argument because I am missing a crucial part.

I see the crux of our disagreement summed up here:

My model of the person who believes the OP wants to say

"Yes, but just because you can tell a story about how evolution would give yo

... (read more)

Ben Pace

*nods* I think what I wrote there wasn't very clear. To restate my general point: I'm suggesting that your general frame contains a weird inversion. You're supposing that there is an objective morality, and then wondering how we can find out about it and whether our moral intuitions are right. I first notice that I have very strong feelings about my and others' behaviour, and then attempt to abstract that into a decision procedure, and then learn which of my conflicting intuitions to trust. In the first one, you would be surprised to find out we've randomly been selected to have the right morality by evolution. In the second, it's almost definitional that evolution has produced us to have the right morality. There's still a lot of work to do to turn the messy desires of a human into a consistent utility function (or something like that), which is a thing I spend a lot of time thinking about. Does the former seem like an accurate description of the way you're proposing to think about morality?

So I think that's broadly right but it's a much narrower argument than Plantinga's.

Plantinga's argument defends that we can't trust our beliefs *in general* if unguided evolution is the case. The argument I defend here is making a narrower claim that it's unlikely that we can trust our normative beliefs if unguided evolution is the case.

slicedonions5y4

I've never heard a plausible account of someone solving the is-ought problem, I'd love to check it out if people here have one. To me it seems structurally to not be the sort of problem that can be overcome.

I find subjectivism a pretty implausible view of morality. It seems to me that morality cannot be mind-dependent and non-universal, it can't be the sort of thing that if someone successfully brainwashes enough people then they can get morality to change. Again, I'd be interested if people here defend a sophisticated view of subjectivism that doesn't have unpalatable results.

MaxDalton

To link this to JP's other point, you might be right that subjectivism is implausible, but it's hard to tell how low a credence to give it. If your credence in subjectivism + model uncertainty (+ I think also constructivism + quasi-realism + maybe others?) is sufficiently high relative to your credence in God, then this weakens your argument (although it still seems plausible to me that theistic moralities end up with a large slice of the pie). I'm pretty uncertain about my credence in each of those views though.