M

MichaelStJules

9575 karmaJoined May 2016

Sequences
1

Welfare and moral weights

Comments
2179

Topic contributions
12

The AI Safety space has LOADS of very smart people that can't get jobs because there aren't enough organisations to hire them. It might be the biggest bottleneck in the cause area. Meanwhile, capabilities literally has dozens of billions being thrown into it

Is not enough organizations really the problem? For technical AI safety research, at least, I hear research management capacity is a bottleneck. A new technical AI safety org would compete with the others over the same potential research managers.

Another issue could be that few interventions seem net positive (maybe things have changed since that comment 3 years ago).

In case you know this off-hand or it's easy for you to get or point me in the right direction, do you know how they established SBF's intent to misuse the billions in customer funds? What I got from Googling this didn't seem very convincing, but I didn't read court documents directly. (See also https://forum.effectivealtruism.org/posts/ggkiDAZowmuzqZrnX/is-anyone-else-still-confused-about-what-exactly-happened-at )

I suspect he meant something like an improvement of 40 percentage points along the normalized 0-100% scale, so with a scale of -10 to 10, this would be adding 8 to their welfare: 8=40%*(10-(-10)).

(Or it could just be 40 percentage points along the normalized negative part of the welfare scale, so +4 on a scale of -10 to 10.)

A few years late, but this was an interesting read.

I have a few related thoughts:

  1. I've tried to characterize types of subjective welfare that seem important to me here. The tl;dr is that they're subjective appearances of good, bad, better, worse, reasons, mattering. Effectively conscious evaluations of some kind, or particular dispositions for those. That includes the types of preferences we would normally want to worry about, but not merely behavioural preferences in entirely unconscious beings or even conscious beings who would make no such conscious evaluations. But consciousness, conscious evaluation, pleasure, suffering, etc. could be vague and apparently merely behavioural preferences could be borderline cases.
  2. On preferences referring to preferences, you can also get similar issues like two people caring more about the other's preferences than their own "personal" preferences, both positively like people who love each other, or negatively, like people who hate each other. You can get a system of equations, but the solution can go off to infinity or not make any sense at all. There's some other writing on this in Bergstrom, 1989Bergstrom, 1999Vadasz, 2005, Yann, 2005 and Dave and Dodds, 2012
  3. Still, I wonder if we can get around issues from preferences referring to preferences by considering what preferences can actually be about in a way that actually makes sense physically. Preferences have to be realized physically themselves, too, after all. If the result isn't logically coherent, then maybe the preference is actually just impossible to hold? But maybe you still have problems when you idealize? And maybe indeterminacy or vagueness is okay. And since a preference is realized in a finite system with finitely many possible states (unless you idealize, or consider unbounded or infinite time), then the degrees of satisfaction they can take are also bounded.

When ice seemed like it could have turned out to be something other than the solid phase of water, we would be comparing the options based on the common facts — the evidence or data — the different possibilities were supposed to explain. And then by finding out that ice is water, you learn that there is much more water in the world, because you would then also have to count all the ice on top of all the liquid water.[13] If your moral theory took water to be intrinsically good and more of it to be better, this would be good news (all else equal).

Suppose we measure amounts by mass. The gram was in fact originally defined as the mass of one cubic centimetre of pure water at 0 °C.[1] We could imagine having defined the gram as the mass of one cubic centimetre of liquid water, but using water that isn't necessarily pure, not fixing the temperature or using a not fully fixed measure for the centimetre. This introduces uncertainty about the measure of mass itself, and we'd later revise the definition as we understood more, but we could still use it in the meantime. We'd also aim to roughly match the original definition: the revised mass of one cubic centimetre of water shouldn't be too different from 1 gram under the new definition.

This is similar to what I say we'd do with consciousness: we define it first relative to human first-person experiences and measure relative to them, but revise the concept and measure with further understanding. We should also aim to make conservative revisions and roughly preserve the value in our references, human first-person experiences.

  1. ^

    In French:

    Gramme, le poids absolu d'un volume d'eau pure égal au cube de la centième partie du mètre , et à la température de la glace fondante.

    https://web.archive.org/web/20160817122340/http://www.metrodiff.org/cmsms/index.php?page=18_germinal_an_3

However, this has only proved to be useful to make predictions in the observable universe, so extending it to the entire universe would not be empirically justifiable.

Useful so far! The problem of induction applies to all of our predictions based on past observations. Everything could be totally different in the future. Why think the laws of physics or observations will be similar tomorrow, but very different outside our observable universe? It seems like essentially the same problem to me.

As a result, I get the impression the hypothesis of an infinite universe is not falsibiable, such that it cannot meaningly be true or false.

Why then assume it's finite rather than infinite or possibly either?

What if you're in a short-lived simulation that started 1 second ago and will end in 1 second, and all of your memories are constructed? It's also unfalsifiable that you aren't. So, the common sense view is not meaningfully true or false, either.

My claim is that at least generally speaking, and I think actually always, theories that are under consideration only predict these relative differences and not the absolute amounts.

(...)

I have some ideas about why this is[1], but mainly I can't think of any examples where this is not the case. If you can think of any then please tell me as that would at least partially invalidate this scale invariance thing (which would be good).

I think what matters here is less whether they predict absolute amounts, but which ones can be put on common scales. If everything could be put on the same common scale, then we would predict values relative to that common scale, and could treat the common scale like an absolute one. But scale invariance would still depend on you using that scale in a scale-invariant way with your moral theory.

I do doubt all theories can be put on one common scale together this way, but I suspect we can find common scales across some subsets of theories at a time. I think there usually is no foundational common scale between any pair of theories, but I'm open to the possibility in some cases, e.g. across approaches for counting conscious subsystems, causal vs evidential decision theory (MacAskill et al., 2019), in some pairs of person-affecting vs total utilitarian views (Riedener, 2019, also discussed in my section here). This is because the theories seem to recognize the same central and foundational reasons, but just find that they apply differently or in different numbers. You can still value those reasons identically across theories. So, it seems like they're using the same scale (all else equal), but differently.

I'm not sure, though. And maybe there are multiple plausible common scales for a given set of theories, but this could mean two envelopes problem between those common scales, not between the specific theories themselves.

And I agree that there probably isn't a shared foundational common scale across all theories of consciousness, welfare and moral weights (as I discuss here).

I think you would also say that theories don't need to predict this overall scale parameter because we can always fix it based on our observations of absolute utility

Ya, that's roughly my position, and more precisely that we can construct common scales based on our first-person observations of utility, although with the caveat that in fact these observations don't uniquely determine the scale, so we still end up with multiple first-person observation-based common scales.

 

this is the bit of maths that I'm not clear on yet, but I do currently think this is not true (i.e. the scale parameter does matter still, especially when you have a prior reason to think there would be a difference between the theories).

Do you think we generally have the same problem for other phenomena, like how much water there is across theories of the nature of water or the strength of gravity as we moved from the Newtonian picture to general relativity? So, we shouldn't treat theories of water as using a common scale, or theories of gravity as using a common scale? Again, maybe you end up with multiple common scales for water, and multiple for gravity, but the point is that we still can make some intertheoretic comparisons, even if vague/underdetermined, based on the observations the theories are meant to explain, rather than say nothing about hiw they relate.

In these cases, including consciousness, water and gravity, it seems like we first care about the observations, and then we theorize about them, or else we wouldn't bother theorizing about them at all. So we do some (fairly) theory-neutral valuing.

When Hormel employees and other associated people gave $500k to an end-of-life care charity - a donation which is part of Lewis's data - I don't think this was a secret scheme to increase beef consumption.

Ya, I wouldn't want to count that. I didn't check what the data included.

People who work in agriculture aren't some sort of evil caricature who only donate money to oppose animal protection; a lot of their donations are probably motivated by the same concerns that motivate everyone else.

I agree. I think if the money is coming through an interest/industry group or company, not just from an employee or farmer, then it's probably usually lobbying for that interest/industry group or company or otherwise to promote the shared interests of that group. Contributions from individuals could be more motivated by political identity and other issues than just protecting or promoting whatever industry they work in.

Vegans could donate to an animal protection group, like HSUS, to lobby on their behalf. That should make it clear why they’re donating.

I doubt that was to support animal protection, though.

Load more