All of jackmalde's Comments + Replies

The psychology of population ethics

I know this wasn't directed at me but I have a few thoughts.

First question: in broad terms, what do you think moral philosophers should infer for psychological studies of this type in general, and from this one in particular? One perspective would be for moral philosophers to update their views towards that of the population - the "500 million Elvis fans can't be wrong" approach.

I think there are various useful things one can take from this study. A few main ones off the top of my head:

  • Understanding people's views allows us to potentially frame things in m
... (read more)
Gifted $1 million. What to do? (Not hypothetical)

You might find it interesting to read through this post and the comments. It covers how one might use a large sum of money to most improve the far future.

Many EAs believe that we should be focused on the far future when doing good, a school of thought called "longtermism". Let me know if you would like to read more about that and I can try to point you in a useful direction.

Can you control the past?

OK thanks, and I have read through now and seen that you discuss randomness in section 4.

Overall a very interesting read! Out of interest, is this idea of "acausal control" entirely novel or has it/something similar been discussed by others? 

2Joe_Carlsmith16dNot sure exactly what words people have used, but something like this idea is pretty common in the non-CDT literature, and I think e.g. MIRI explicitly talks about "controlling" things like your algorithm.
Can you control the past?

I haven't read the whole post, but:

But defecting in this case, I claim, is totally crazy. Why? Because absent some kind of computer malfunction, both of you will make the same choice, as a matter of logical necessity.

Is this definitely true when you take into account quantum randomness? Maybe it is, but, if so, I think it might be worth explaining why.

7Joe_Carlsmith21dI'm imagining computers with sufficiently robust hardware to function deterministically at the software level, in the sense of very reliably performing the same computation, even if there's quantum randomness at a lower level. Imagine two good-quality calculators, manufactured by the same factory using the same process, which add together the same two numbers using the same algorithm, and hence very reliably move through the same high-level memory states and output the same answer. If quantum randomness makes them output different answers, I count that as a "malfunction."
New Articles on Utilitarianism.net: Population Ethics and Theories of Well-Being

I see where you're coming from, but if this is true:

For all these reasons, utilitarians are largely united in rejecting person-affecting views, even as they continue to debate which impersonal theory provides the best way forward.

then part of me thinks that the population ethics section did in fact need to pay adequate attention to the potential drawbacks of person-affecting views and make it clear why utilitarians tend to think impersonal theories are preferable, which was always going to come across somewhat biased.

Ultimately whilst the section is an int... (read more)

Questions for Howie on mental health for the 80k podcast

My gut reaction is to think we should just make use of existing mental health resources out there which are abundant. I’m not sure why it would help for it to be EA specific.

It would certainly be useful for someone to make a summary of available resources and/or to do a meta-review of what works for mental health, but I can’t see that this would require a whole organisation to be set up. CEA could hire one person to work on this for example and that would seem to me to be sufficient.

2Charles He1moThis org would be setup to provision actual mental health services or programs at free or low cost for EAs. To be really concrete, maybe imagine a pilot with 1-2 EA founders and maybe 2-4.0 FTE practitioners or equivalent partnerships. There are perspectives where the value of reviews and compendium websites have limited value and my first impression is that it may apply here. This is a very strong statement. I have trouble relating to this belief. Your profile says you work in management consulting or economics. You also seem to live in the UK. You seem have and directly use high human capital in management or highly technical fields. In totality, you probably enjoy substantial mental health services and while such jobs can be stressful, it's usually the case they do not involve direct emotional trauma. Not all EAs enjoy the same experiences. For example: * In global health or in animal welfare, people who love animals have to pour over graphic footage of factory farming, or physically enter factory farms risking specific legal retaliation from industry, to create movies like this [https://youtu.be/LQRAfJyEsko?t=4346]. I assure you funding for such work is low and there may be no mental health support. * Similar issues exist in global health and poverty, where senior leaders often take large pay cuts and reduction of benefits. * I know an EA-like person who had to work for free during 2020 for 3-4 months in a CEO role, where they worked 20 hour days. They regularly faced pleas for medical supplies from collapsing institutions, as well as personal attacks and fires inside and outside the org, for not doing enough. Many of these roles are low status or have zero pay or benefits. Many of the people who do this work above have very high human capital and would enjoy high pay and status, but actively choose to do work because no one else will or will even understand it.
Towards a Weaker Longtermism

For the record I'm not really sure about 1030 times, but I'm open 1000s of times.

And differences of 1030 are almost impossible, because everything we do now may affect the whole far future and therefore has nontrivial expected impact on vast numbers of lives.

Pretty much every action has an expected impact on the future in that we know it will radically alter the future  e.g. by altering the times of conceptions and therefore who lives in the future. But that doesn't necessarily mean we have any idea on the magnitude or sign of this expected impact. Wh... (read more)

Towards a Weaker Longtermism

It seems to me that many longtermists believe (i) but that almost no-one believes (ii).

Really? This surprises me. Combine (i) with the belief that we can tractably influence the far future and don't we pretty much get to (ii)?

6Darius_Meissner1moNo, we probably don’t. All of our actions plausibly affect the long-term future in some way, and it is difficult to (be justified to) achieve very high levels of confidence about the expected long-term impacts of specific actions. We would require an exceptionaldegree of confidence to claim that the long-term effects of our specific longtermist intervention are astronomically (i.e. by many orders of magnitude) larger than the long-term effects of some random neartermist interventions (or even doing nothing at all). Of course, this claim is perfectly compatible with longtermist interventions being a few orders of magnitude more impactful in expectation than neartermist interventions (but the difference is most likely not astronomical). Brian Tomasik eloquently discusses this specific question in the above-linked essay [https://reducing-suffering.org/why-charities-dont-differ-astronomically-in-cost-effectiveness/] . Note that while his essay focuses on charities, the same points likely apply to interventions and causes: Brian Tomasik further elaborates on similar points in a second essay, Charity Cost-Effectiveness in an Uncertain World [https://longtermrisk.org/charity-cost-effectiveness-in-an-uncertain-world/]. A relevant quote:
Towards a Weaker Longtermism

I think Ord's favoured approach to moral uncertainty is maximising expected choice-worthiness (MEC) which he argues for with Will MacAskill.

Reading the abstract of the moral parliamentarianism paper, it isn't clear to me that he is actually a proponent of that approach, just that he has a view on the best specific approach within moral parliamentarianism.

As I say in my comment to Ben, I think an MEC approach to moral uncertainty can lead to being quite fanatical in favour of longtermism.

Towards a Weaker Longtermism

I don't think it's necessarily clear that incorporating moral uncertainty means you have to support hedging across different plausible views. If one maximises expected choiceworthiness (MEC) for example one can be fanatically driven by a single view that posits an extreme payoff (e.g. strong longtermism!).

Indeed MacAskill and Greaves have argued that strong longtermism seems robust to variations in population axiology and decision theory whilst Ord has argued reducing x-risk is robust to normative variations (deontology, virtue ethics, consequentialism). I... (read more)

What would you do if you had half a million dollars?

Thanks, I understand all that. I was confused when Khorton said:

I meant increasing the number of grantmakers who have spent significant time thinking about where to donate significant capital

I wouldn't say the lottery increases the number of grantmakers who have spent significant time thinking, I think it in fact reduces it.

I agree with you when you say however:

The overall amount of time spent is actually less than before, but the depth is far greater, and with dramatically less redundancy.


 

8Jonas Vollmer2moI think deciding between capital allocators is a great use of the donor lottery, even as a Plan A. You might say something like: "I would probably give to the Long-Term Future Fund, but I'm not totally sure whether they're better than the EA Infrastructure Fund or Longview or something I might come up with myself. So I'll participate in the donor lottery so if I win, I can take more time to read their reports and see which of them seems best." I think this would be a great decision. I'd be pretty unhappy if such a donor then felt forced to instead do their own grantmaking despite not having a comparative advantage for doing so (possibly underperforming Open Phil's last dollar), or didn't participate in the donor lottery in the first place. I think the above use case is one of the most central one that I hope to address. I tentatively agree that further diversification of funding sources might be good, but I don't think the donor lottery is the right tool for that.
What would you do if you had half a million dollars?

I think perhaps we agree then - if after significant research, you realize you can't beat an EA Fund, that seems like a reasonable fallback, but that should not be plan A.

Yeah that sounds about right to me.

I meant increasing the number of grantmakers who have spent significant time thinking about where to donate significant capital

I still don't understand this. The lottery means one / a small number of grantmakers get all the money to allocate. People who don't win don't need to think about where to donate. So really it seems to me that the lottery reduces the number of grantmakers and indeed the number of who spend time thinking about where to donate.

I still don't understand this. The lottery means one / a small number of grantmakers get all the money to allocate. People who don't win don't need to think about where to donate. So really it seems to me that the lottery reduces the number of grantmakers and indeed the number of who spend time thinking about where to donate.

The model is this:

  • A bunch of people each have $5,000 to donate.
  • Many put in a bit of effort - they spend a bit of time on the GiveWell website, read some stuff by MIRI, and chat to a couple of friends. But this isn't enough to catch the
... (read more)
What would you do if you had half a million dollars?

I'm not sure I understand how the lottery increases the diversity of funding sources / increases the number of grantmakers if one or a small number of people end up winning the lottery. Wouldn't it actually reduce diversity / number of grantmakers? I might be missing something quite obvious here...

Reading this it seems the justification for lotteries is that it not only saves research time for the EA community as a whole, but also improves the allocation of the money in expectation. Basically if you don't win you don't have to bother doing any research (so... (read more)

5Khorton2moI think perhaps we agree then - if after significant research, you realize you can't beat an EA Fund, that seems like a reasonable fallback, but that should not be plan A. Re: increasing grantmakers, I meant increasing the number of grantmakers who have spent significant time thinking about where to donate significant capital - obviously having hundreds of people donating $1k each would have more diversity but in practice I think most $1k donors defer their decision-making to someone else, like an EA Fund or GiveWell.
What would you do if you had half a million dollars?

Although whether increasing the population is a good thing depends of if you are an average utilitarian or a total utilitarian.

Yes that is true. For what it's worth most people who have looked into population ethics at all reject average utilitarianism as it has some extremely unintuitive implications like the "sadistic conclusion" whereby one can make things better by bringing into existence people with terrible lives, as long as they're still bringing up the average wellbeing level by doing so i.e. if existing people have even worse lives.

What would you do if you had half a million dollars?

I got the impression that their new, general-purpose pool would still be fairly longtermist, but it's possible they will have to make sacrifices.

To clarify it's not that I don't think they would be "longtermist" it's more that I think they may have to give to longtermist options that "seem intuitively good to a non-EA", e.g. giving to an established organisation like MIRI or CHAI, rather than give to longtermist options that may be better on the margin but seem a bit weirder at first glance like "buying out some clever person so they have more time to do s... (read more)

2Ozzie Gooen2moI think that looking at their track record is only partially representative. They used to follow a structure where they would recommend donation opportunities to particular clients. Recently they've set up a fund that works differently; people would donate to the fund, then the fund will make donations at their will. My guess is that this will help a bit around this issue, but not completely. (Maybe they'll even be extra conservative, to prove to donors that they will match their preferences.) Another (minor) point is that Longview's donations can be fungible with LTFF. If they spend $300K on something that LTFF would have otherwise spent money on, then the LTFF would have $300K more to spend on whatever it wants. So if Longview can donate to, say, only 90% of interesting causes, up to $10Mil per year, the last 10% might not be that big of a deal.
What would you do if you had half a million dollars?

Yeah you probably should - unless perhaps you think there are scale effects to giving which makes you want to punt on being able to give far more.

Worth noting of course that Patrick didn’t know he was going to give to a capital allocator when he entered the lottery though, and of course still doesn’t. Ideally all donor lottery winners would examine the LTFF very carefully and honestly consider whether they think they can do better than LTFF. People may be able to beat LTFF, but if someone isn’t giving to LTFF I would expect clear justification as to why they think they can beat it.

I disagree. One of the original rationales for the lottery if I recall correctly was to increase the diversity* of funding sources and increase the number of grantmakers. I think if the LTFF is particularly funding constrained, there's a good chance the Open Philanthropy Project or a similar organisation will donate to them. I value increased diversity and number of grantmakers enough that I think it's worth trying to beat LTFF's grantmaking even if you might fail.

*By diversity, I don't mean gender or ethnicity, I just mean having more than one grantmaker doing the same thing, ideally with different knowledge, experience and connections.

What would you do if you had half a million dollars?

Would you mind linking some posts or articles assessing the expected value of the long-term future?

You're right to question this as it is an important consideration. The Global Priorities Institute has highlighted "The value of the future of humanity" in their research agenda (pages 10-13). Have a look at the "existing informal discussion" on pages 12 and 13, some of which argues that the expected value of the future is positive.

Sure, it's possible that some form of eugenics or genetic engineering could be implemented to raise the average hedonic set-point

... (read more)
1Question Mark2moThanks. Although whether increasing the population is a good thing depends of if you are an average utilitarian or a total utilitarian [https://en.wikipedia.org/wiki/Average_and_total_utilitarianism]. With more people, both the number of hedons and dolors will increase, with a ratio between hedons to dolors skewed in favor of hedons. If you're a total utilitarian, the net hedons will be higher with more people, so adding more people is rational. If you're a total utilitarian, the ratio of hedons to dolors and the average level of happiness per capita will be roughly the same, so adding more people wouldn't necessarily increase expected utility.
What would you do if you had half a million dollars?

I'm not really sure what to think about digital sentience. We could in theory create astronomical levels of happiness, astronomical levels of suffering, or both. Digital sentience could easily dominate all other forms of sentience so it's certainly an important consideration.

It seems unlikely to me that we would go extinct, even conditional on "us" deciding it would be best.

This is a fair point to be honest!

What would you do if you had half a million dollars?

In general, it kind of seems like the "point" of the lottery is to do something other than allocate to a capital allocator.

If you enter a donor lottery your expected donation amount is the same as if you didn't enter the lottery. If you win the lottery, it will be worth the time to think more carefully about where to allocate the money than if you had never entered, as you're giving away a much larger amount. Because extra time thinking is more likely to lead to better (rather than worse) decisions, this leads to more (expected) impact overall, even though... (read more)

6Khorton2moI could imagine that happening in some situations where after a lot of careful thought you decide to defer to another grantmaker, but if you know in advance that you'd like to give your money to a grantmaker, shouldn't you just do that?
What would you do if you had half a million dollars?

There is still the possibility that the Pinkerites are wrong though, and quality of life is not improving.

Sure, and there could be more suffering than happiness in the future, but people go with their best guess about what is more likely and I think most in the EA community side with a future that has more happiness than suffering.

happiness levels in general should be roughly stable in the long run regardless of life circumstances.

Maybe, but if we can't make people happier we can always just make more happy people. This would be very highly desirable if yo... (read more)

4Question Mark2moWould you mind linking some posts or articles assessing the expected value of the long-term future? If the basic argument for the far future being far better than the present is because life now is better than it was thousands of years ago, this is, in my opinion, a weak argument. Even if people like Steven Pinker are right, you are extrapolating billions of years from the past few thousand years. To say that this is wild extrapolation is an understatement. I know Jacy Reese talks about it in this post [https://forum.effectivealtruism.org/posts/BY8gXSpGijypbGitT/why-i-prioritize-moral-circle-expansion-over-artificial#The_expected_value_of_the_far_future] , yet he admits the possibility that the expected value of the far future could potentially be close to zero. Brian Tomasik also wrote this article [https://reducing-suffering.org/near-miss/] about how a "near miss" in AI alignment could create astronomical amounts of suffering. Sure, it's possible that some form of eugenics or genetic engineering could be implemented to raise the average hedonic set-point of the population and make everyone have hyperthymia [https://en.wikipedia.org/wiki/Hyperthymic_temperament]. But you must remember that millions of years of evolution put our hedonic set-points where they are for a reason. It's possible that in the long run, genetically engineered hyperthymia might be evolutionarily maladaptive, and the "super happy people" will die out in the long run.
What would you do if you had half a million dollars?

B) the far future can be reasonably expected to have significantly more happiness than suffering

I think EAs who want to reduce x-risk generally do believe that the future should have more happiness than suffering, conditional on no existential catastrophe occurring. I think these people generally argue that quality of life has improved over time and believe that this trend should continue (e.g. Steven Pinker's The Better Angels of Our Nature). Of course life for farmed animals has got worse...but I think people believe we should successfully render factory... (read more)

5antimonyanthony2moNote that this post [https://forum.effectivealtruism.org/posts/NfkEqssr7qDazTquW/the-expected-value-of-extinction-risk-reduction-is-positive#1_3__Future_agents_could_later_decide_not_to_colonize_space__option_value_] (written by people who agree that reducing extinction risk is good) provides a critique of the option value argument.
4Linch2moI lend some credence to the trendlines argument but mostly think that humans are more likely to want to optimize for extreme happiness (or other positive moral goods) than extreme suffering (or other negatives/moral bads), and any additive account of moral goods will shake out to in expectation have a lot more positive moral goods than moral bads, unless you have really extreme inside views to think that optimizing for extreme moral bads is as as likely (or more likely) than optimizing for extreme moral goods. I do think there are nontrivial probability of P(S-risk | singularity), eg a) our descendants are badly mistaken or b) other agents follow through with credible pre-commitments to torture, but I think it ought to be surprising for classical utilitarians to believe that the EV of the far future is negative.

Of course life for farmed animals has got worse...but I think people believe we should successfully render factory farming redundant on account of cultivated meat.

I think there's recently more skepticism about cultured meat (see here, although I still expect factory farming to be phased out eventually, regardless), but either way, it's not clear a similar argument would work for artificial sentience, used as tools, used in simulations or even intentionally tortured. There's also some risk that nonhuman animals themselves will be used in space colonization,... (read more)

4Question Mark2moThere is still the possibility that the Pinkerites are wrong though, and quality of life is not improving. Even though poverty is far lower and medical care is far better than in the past, there may also be more mental illness and loneliness than in the past. The mutational load within the human population may also be increasing [https://pubmed.ncbi.nlm.nih.gov/31424543/]. Taking the hedonic treadmill [https://en.wikipedia.org/wiki/Hedonic_treadmill] into account, happiness levels in general should be roughly stable in the long run regardless of life circumstances. One may object to this by saying that wireheading may become feasible in the far future. Yet wireheading may be evolutionarily maladaptive [https://reducing-suffering.org/how-likely-is-wireheading/], and pure replicators may dominate the future instead. Andrés Gómez Emilsson has also talked about this in A Universal Plot - Consciousness vs. Pure Replicators [https://www.youtube.com/watch?v=nGmETz-wDMc]. Regarding averting extinction and option value, deciding to go extinct is far easier said than done. You can’t just convince everyone that life ought to go extinct. Collectively deciding to go extinct would likely require a singleton to exist, such as Thomas Metzinger's BAAN [https://www.edge.org/conversation/thomas_metzinger-benevolent-artificial-anti-natalism-baan] scenario. Even if you could convince a sizable portion of the population that extinction is desirable, these people will simply be removed by natural selection, and the remaining portion of the population will continue existing and reproducing. Thus, if extinction turns out to be desirable, engineered extinction would most likely have to be done without the consent of the majority of the population. In any case, it is probably far easier to go extinct now while we are confined to a single planet than it would be during the age of galaxy-wide colonization.
What would you do if you had half a million dollars?

I have to say I'm pretty glad you won the lottery as I like the way you’re thinking! I have a few thoughts which I put below. I’m posting here so others can respond, but I will also fill out your survey to provide my details as I would be happy to help further if you are interested in having my assistance!

TLDR: I think LTFF and PPF are the best options, but it’s very hard to say which is the better of the two.

  • Longview Philanthropy: it’s hard to judge this option without knowing more about their general-purpose fund - I didn’t see anything on this on their
... (read more)
3Jonas Vollmer2moI really liked this comment. Three additions: * I would take a close look at who the grantmakers are and whether their reasoning seems good to you. Because there is significant fungibility and many of these funding pools have broad scopes, I personally expect the competence of the grantmakers to matter at least as much as the specific missions of the funds. * I don't think it's quite as clear that the LTFF is better than the EA Infrastructure Fund; I agree with your argument but think this could be counterbalanced by the EA Infrastructure Fund's greater focus on talent recruitment, or other factors. * I don't know to what degree it is hard for Longview to get fully unrestricted funding, but if that's hard for Longview, giving it unrestricted funding may be a great idea. They may run across promising opportunities that aren't palatable to their donors, and handing them over to EA Funds or Open Philanthropy may not be straightforwardly easy in some cases. (Disclosure: I run EA Funds, which hosts the LTFF and EA Infrastructure Fund. Opinions my own, as always.)
2Ozzie Gooen2moThanks for the comment, this raises a few good points. Good point. I got the impression that their new, general-purpose pool would still be fairly longtermist, but it's possible they will have to make sacrifices. We'll ping them about them (or if any of them are reading this, please do reply directly!) > If you find someone / a team who you think is better than the LTFF grant team then fine, but I’m sceptical you will. To be clear, one of the the outcomes could be that this person decides to give to the LTFF. These options aren't exclusive. But I imagine in this case, they shouldn't have that much work to do, they would essentially be making a choice from the options we list above.
What would you do if you had half a million dollars?

I'm not saying that reducing S-risks isn't a great thing to do, nor that it would reduce happiness, I'm just saying that it isn't clear that a focus on reducing S-risks rather than on reducing existential risk  is justified if one values reducing suffering and increasing happiness equally.

I think robustness (or ambiguity aversion) favours reducing extinction risks without increasing s-risks and reducing s-risks without increasing extinction risks, or overall reducing both, perhaps with a portfolio of interventions. I think this would favour AI safety, especially that focused on cooperation, possibly other work on governance and conflict, and most other work to reduce s-risks (since it does not increase extinction risks), at least if we believe CRS and/or CLR that these do in fact reduce s-risks. I think Brian Tomasik comes to an overall pos... (read more)

9Question Mark2moIf one values reducing suffering and increasing happiness equally, it isn't clear that reducing existential risk is justified either. Existential risk reduction and space colonization means that the far future can be expected to have both more happiness and more suffering, which would seem to even out the expected utility. More happiness + more suffering isn't necessarily better than less happiness + less suffering. Focusing on reducing existential risks would only seem to be justified if either A) you believe in Positive Utilitarianism, i.e. increasing happiness is more important than reducing suffering, B) the far future can be reasonably expected to have significantly more happiness than suffering, or C) reducing existential risk is a terminal value in and of itself.
What would you do if you had half a million dollars?

My understanding is that Brian Tomasik has a suffering-focused view of ethics in that he sees reducing suffering as inherently more important than increasing happiness - even if the 'magnitude' of the happiness and suffering are the same.

If one holds a more symmetric view where suffering and happiness are both equally important it isn't clear how useful his donation recommendations are.

8Question Mark2moEven if you value reducing suffering and increasing happiness equally, reducing S-risks [https://www.youtube.com/watch?v=jiZxEJcFExc] would likely still greatly increase the expected value of the far future. Efforts to reduce S-risks would almost certainly reduce the risk of extreme suffering being created in the far future, but it's not clear that they would reduce happiness much.
What are the long term consequences of poverty alleviation?

Probably depends on how you're reducing poverty...and how long-term your "long-term" is. Something like removing trade restrictions is likely to have very different long-term effects than distributing bednets. Even then I really don't have good answers for you on the nature of these differences.

You might want to check out the persistence studies literature. For example work by Nathan Nunn, who Will MacAskill references in this talk. This may not precisely align to what you're asking for, but Nunn has studies finding for example that:

  • Countries that adopted
... (read more)
9zdgroff2moI think the persistence studies stuff is the best bet. One thing to note there is that the literature is sort of a set of existence proofs. It shows that there are various things that have long-term impacts, but it might not give you a strong sense of the average long-term impact of poverty alleviation.
What should we call the other problem of cluelessness?

Could also go for tractable and intractable cluelessness?

Also I wonder if we should be distinguishing between empirical and moral cluelessness - with the former being about claims about consequences and the latter about fundamental ethical claims.

The case for strong longtermism - June 2021 update

Thanks Robert. I've never seen this breakdown of cluelessness and it could be a useful way for further research to define the issue.

The Global Priorities Institute raised the modelling of cluelessness in their research agenda and I'm looking forward to further work on this. If interested, see below for the two research questions related to cluelessness in the GPI research agenda. I have a feeling that there is still quite a bit of research that could be conducted in this area.

------------------

Forecasting the long-term effects of our actions often requires... (read more)

The case for strong longtermism - June 2021 update

Yeah I meant ruling out negative EV in a representor may be slightly extreme, but I’m not really sure - I need to read more.

The case for strong longtermism - June 2021 update

Thanks, I really haven't given sufficient thought to the cluelessness section which seems the most novel and tricky. Fanaticism is probably just as important, if not more so, but is also easier to get one's head around.  

I agree with you in your other comment though that the following seems to imply that the authors are not "complexly clueless" about AI safety:

For example, we don’t think any reasonable representor even contains a probability function according to which efforts to mitigate AI risk save only 0.001 lives per $100 in expectation.

I mean I ... (read more)

2MichaelStJules3moYa, maybe your representor should be a convex set, so that for any two functions in it, you can take any probabilistic mixture of them, and that would also be in your representor. This way, if you have one with expected value x and another with expected value y, you should have functions with each possible expected value between. So, if you have positive and negative EVs in your representor, you would also have 0 EV. Do you mean negative EV is slightly extreme or ruling out negative EV is slightly extreme? I think neglecting to look into and address ways something could be negative (e.g. a probability difference, EV) often leads us to unjustifiably assuming a positive lower bound, and I think this is an easy mistake to make or miss. Combining a positive lower bound with astronomical stakes would make the argument appear very compelling.
The case for strong longtermism - June 2021 update

On their (new) view on what objections against strong longtermism are strongest - I think that this may be the most useful update in the paper. I think it is very important to pinpoint the strongest objections to a thesis, to focus further research. 

It is interesting that the authors essentially appear to have dismissed the intractability objection. It isn’t clear if they no longer think this is a valid objection, or if they just don’t think it is as strong as the other objections they highlight this time around. I would like to ask them about this in... (read more)

The case for strong longtermism - June 2021 update

On addressing cluelessness - for the most part I agree with the authors’ views, which includes the view that there needs to be further research in this area.

I do find it odd however that they attempt to counter the worry of ‘simple cluelessness’ but not that of ‘complex cluelessness’ i.e. to counter the possibility that there could be semi-foreseeable unintended consequences of longtermist interventions that make us ultimately uncertain on the sign of the expected-value assessment of these interventions. Maybe they see this as obviously not an issue...but I would have appreciated some thoughts on this.

3RobertHarling3moThanks for this post Jack, I found it really useful as I haven't got round yet to reading the updated paper. This break down in the cluelessness section was a new arrangement to me. Does anyone know if this break down has been used elsewhere? If not this seems like useful progress in better defining the cluelessness objections to longtermism.
4MichaelStJules3moI think complex cluelessness is essentially covered by the other subsections in the Cluelessness section. It's an issue of assigning numbers arbitrarily to the point that what you should do depends on your arbitrary beliefs. I don't think they succeed in addressing the issue, though, since they don't sufficiently discuss and address ways each of their proposed interventions could backfire despite our best intentions (they do discuss some in section 4, though). The bar is pretty high to satisfy any "reasonable" person.
The case for strong longtermism - June 2021 update

On the new definition - as far as I can tell it does pretty much the same job as the old definition, but is clearer and more precise, bar a small nitpick I have...

One deviation is from “a wide class of decision situations” to “the most important decision situations facing agents today”. As far as I can tell, Greaves and MacAskill don’t actually narrow the set of decision situations they argue ASL applies to in the new paper. Instead, I suspect the motivation for this change in wording was because “wide” is quite imprecise and subjective (Greaves concedes t... (read more)

Help me find the crux between EA/XR and Progress Studies

You mention that some EAs oppose progress / think that it is bad. I might be wrong, but I think these people only "oppose" progress insofar as they think x-risk reduction from safety-based investment is even better value on the margin. So it's not that they think progress is bad in itself, it's just that they think that speeding up progress incurs a very large opportunity  cost. Bostrom's 2003 paper outlines the general reasoning why many EAs think x-risk reduction is more important than quick technological development.

Also, I think most EAs intereste... (read more)

EA Survey 2020: Demographics

I would absolutely expect EAs to differ in various ways to the general population. The fact that a greater proportion of EAs are vegan is totally expected, and I can understand the computer science stat as well given how important AI is in EA at the moment. 

However when it comes to sexuality it isn't clear to me why the EA population should differ. It may not be very important to understand why, but then again the reason why could be quite interesting and help us understand what draws people to EA in the first place. For example perhaps LGBTQ+ people ... (read more)

1david_reinstein4moLet's presume that the 'share non-straight is' a robust empirical finding and not an artifact of sample selection or of how the question was asked, or of the nonresponse etc. (We could dig into this further if it merited the effort)... It is indeed somewhat surprised, but I am not who surprised, as I expect a group that is very different in some ways from the general population may likely be very different in other ways, and we may not always have a clear story for why. If we did want to look into it further it further, we might look into what share of the vegan population, or of the 'computer science population', in this mainly very-young age group, is not straight-identified. (of course those numbers may also be very very difficult together, particularly because of the difficulty of getting a representative sample of small populations, as I discuss here [https://rethinkpriorities.github.io/ea_data_public/sensitivity.html#eas_challenges] . This may be very interesting from a sociological point of view but I am not sure if it is a first order important for us right now. That said, if we have time we may be able to get back to it.
9David_Moss4moI agree that there's not a direct explanation of why we would expect this difference in the EA community, unlike in the case of veganism and computer science. I also agree that properties of our sampling itself don't seem to offer good explanations of these results (although of course we can't rule this out). This would just push the explanation back a level, and it seems even harder to explain why we'd heavily oversample nonheterosexual (and especially female nonheterosexual EAs compared to female heterosexual EAs) than to explain why we might observe these differences in the EA community. That said, we do have good reason to think that the EA community (given its other characteristic) would have higher percentages of nonheterosexual responses. Elite colleges in general also have substantially higher rates than the general population (it looks like around 15% [https://features.thecrimson.com/2017/freshman-survey/makeup-yale/] at Harvard and Yale) and of course the EA community contains a very disproportionate percentage of elite college graduates. Also, although we used slightly different questions, it seems like the EA community may be more liberal than elite colleges, and we might expected higher self-reported nonheterosexuality in more liberal populations (comparing to Harvard and Yale again, they have around 12% somewhat/very conservatives, we have around 3% centre right or right- we have more 'libertarians', but if these are mostly socially liberal then the same might apply). As I noted before though, I think that this is probably just a result of the particular question format used. I would expect more nonheterosexual responses where people can write in a free response compared to where they have to select either heterosexual or some other fixed category.
EA Survey 2020: Demographics

Thanks. Any particular reason why you decided to do unguided self-description?

You could include the regular options and an "other (please specify)" option too. That might give people choice, reduce time required for analysis, and make comparisons to general population surveys easier.

2David_Moss4moThis was a specific question and question format that it was requested we include. I wouldn't like to speculate about the rationale, but I discuss some of the pros and cons associated with each style of question in general here [https://forum.effectivealtruism.org/posts/kpRu5zSdmLWmwLBdM/please-take-the-2020-ea-survey?commentId=hMKGSLG2H5NfxeWMQ] .
EA Survey 2020: Demographics

We observed extremely strong divergence across gender categories. 76.9% of responses from male participants identified as straight/heterosexual, while only 48.6% of female responses identified as such. 

The majority of females don't identify as heterosexual? Am I the only one who finds this super interesting? I mean in the UK around 2% of females in the wider population identify as LGB.

Even the male heterosexual figure is surprising low. Any sociologists or others want to chime in here?

2david_reinstein4moI was also surprised, but obviously we are far from a random sample of the population, there is a very unusual 'selection' process to * know about EA * identify with EA * take the survey E.g., (and its not a completely fair analogy but) about 30% of 2019 respondents said they were vegan vs about vs about 1-3% of comparable populations [https://en.wikipedia.org/wiki/Vegetarianism_by_country#:~:text=as%20of%202003.-,United%20States,are%20vegetarian%20and%200.5%25%20vegan] Perhaps better analogy: Looking quickly at the 2018-2019 data, roughly half of responded studied computer science. This compares to about 5% of the US degrees granted , 10% if we include all engineering degrees [https://nces.ed.gov/programs/digest/d19/tables/dt19_322.10.asp]. But is this worth pursuing further? Should we dig into surprising was the EA/EA-survey population differs from the general population?
4reallyeli4moYounger people and more liberal people are much more likely to identify as not-straight, and EAs are generally young and liberal. I wonder how far this gets you to explaining this difference, which does need a lot of explaining since it's so big. Some stats [https://news.gallup.com/poll/329708/lgbt-identification-rises-latest-estimate.aspx] on this (in the US).
9David_Moss4moThat's not quite right because some responses were coded as "unclear": around 33% of female responses were coded as not heterosexual, which is almost 3x as many for male respondents. Both those percentages are still relatively high, of course. Unfortunately I don't think it's clear what to make of them, due to a number of factors: i) results for the general population surveys are inconsistent across different surveys, ii) results seem to vary a lot by age (for example, see here [https://www.statista.com/statistics/719685/american-adults-who-identify-as-homosexual-bisexual-transgender-by-generation/] ) and likely by other demographic factors (e.g. how liberal or educated the population is) which I would expect to raise EA numbers, iii) most surveys don't use the question format that we were asked to use (people writing in an unguided self-description) making comparison difficult, iv) as a result of that survey choice the results for our survey are very hard to interpret with large percentages of responses being explicitly coded as unclassifiable. As such, while interesting, I think these results are hard to make anything of with any degree of confidence, which is why we haven't looked into them in any detail.
2BrianTan4moYeah I find this interesting and surprising too.
Ending The War on Drugs - A New Cause For Effective Altruists?

I think that's fair but I also think that non-neglectedness is actually bad for two reasons:

  1. Diminishing returns (which may not be the case if people are solving the problem poorly)
  2. Crowdedness meaning it's harder to change direction even if people are solving the problem poorly (although this point is really tractability so one needs to be careful about not double-counting when doing ITN).

I'm thinking number 2 could be quite relevant in this case. Admittedly it's quite relevant for any EA intervention that involves systemic change, but I get the impression that other systemic change interventions may be even higher in importance.

Concerns with ACE's Recent Behavior

The only thing of interest here is what sort of compromise ACE wanted. What CARE said in response is not of immediate interest, and there's certainly no need to actually share the messages themselves.

Perhaps you can understand why one might come away from this conversation thinking that ACE tried to deplatform the speaker? To me at least it feels hard to interpret "find a compromise" any other way.

[Note that I have no idea whatsoever about what actually happened here. This is purely hypothetical.]

FWIW if I was in a position similar to ACE's here are a few potential "compromises" I would have explored. (Of course, which of these would be acceptable to me would depend on exactly why I'm concerned and how strongly, etc.) I think some of them wouldn't typically be considered deplatforming, though I would imagine that people who are against deplatforming would find many if not all of them at least somewhat objectionable (I would also guess that some who ... (read more)

Concerns with ACE's Recent Behavior

Thanks for writing this comment as I think you make some good points and I would like people who disagree with Hypatia to speak up rather than stay silent.

Having said that, I do have a few critical thoughts on your comment. 

Your main issue seems to be the claim that these harms are linked, but you just respond by only saying how you feel reading the quote, which isn't a particularly valuable approach.

I don’t think this was Hypatia’s main issue. Quoting Hypatia directly, they imply the following are the main issues:

  • The language used in the statement ma
... (read more)
Avoiding the Repugnant Conclusion is not necessary for population ethics: new many-author collaboration.

I don't find your comment to have much in the way of argument as to why it might be bad if papers like this one become more widespread. What are you actually worried would happen? This isn't super clear to me at the moment.

I agree a paper that just says "we should ignore the repugnant conclusion" without saying anything else isn't very helpful, but this paper does at least gather reasons why the repugnant conclusion may be on shaky ground which seems somewhat useful to me.

6AppliedDivinityStudies5moI received a nice reply from Dean which I've asked if I can share. Assuming he says yes, I'll have a more thought out response to this point soon. Here are some quick thoughts: There are many issues in all academic fields, the vast majority of which are not paid the appropriate amount of attention. Some are overvalued, some are unfairly ignored. That's too bad, and I'm very glad that movements like EA exist to call more attention to pressing research questions that might otherwise get ignored. What I'm afraid of is living in a world where researchers see it as part of their charter to correct each of these attentional inexactitudes, and do so by gathering bands of other academics to many-author a paper which basically just calls for a greater/lesser amount of attention to be paid to some issue. Why would that be bad? 1. It's not a balanced process. Unlike the IGM Experts Panel [https://www.igmchicago.org/igm-economic-experts-panel/], no one is being surveyed and there's no presentation of disagreement or distribution of beliefs over the field. How do we know there aren't 30 equally prominent people willing to say the Repugnant Conclusion is actually very important? Should they go out and many-author their own paper? 2. A lot of this is very subjective, you're just arguing that an issue receives more/less attention than is merited. That's fine as a personal judgement, but it's hard for anyone else to argue against on an object-level. This risks politicization. 3. There are perverse incentives. I'm not claiming that's what's at play here, but it's a risk this precedent sets. When academics argue for the (un)importance of various research questions, they are also arguing for their own tenure, departmental funding, etc. This is an unavoidable part of the academic career, but it should be limited to careerist venues, not academic publications. Again, those are some quick thoughts from an outsider, so I wouldn't
Confusion about implications of "Neutrality against Creating Happy Lives"

My short answer is that 'neutrality against creating happy lives' is not a mainstream position in the EA community. Some do hold that view, but I think it's a minority. Most think that creating happy lives is good.

0Alex HT5moI agree with this answer. Also, lots of people do think that temporal position (or something similar, like already being born) should affect ethics. But yes OP, accepting time neutrality and being completely indifferent about creating happy lives does seem to me to imply the counterintuitive conclusion you state. You might be interested in this excellent emotive piece [https://forum.effectivealtruism.org/posts/HGLK3igGprWQPHfAp/against-neutrality-about-creating-happy-lives] or section 4.2.1 of this philosophy thesis [https://rucore.libraries.rutgers.edu/rutgers-lib/40469/PDF/1/play/,]. They both argue that creating happy lives is a good thing.
On the longtermist case for working on farmed animals [Uncertainties & research ideas]

Thanks for writing this Michael, I would love to see more research in this area. 

Thus, it seems plausible that expanding a person’s moral circle to include farm animals doesn’t bring the “boundary” of that person’s moral circles any “closer” to including whatever class of beings we’re ultimately concerned about (e.g., wild animals or artificial sentient beings). Furthermore, even if expanding a person’s moral circle to include farm animals does achieve that outcome, it seems plausible that that the outcome would be better achieved by expanding moral c

... (read more)
2MichaelA5moWhen I read your comment, I thought "I think you've correctly highlighted one reason we might want to focus on advocating for impartial utilitarianism or for moral concern for 'all sentient beings', but I think there are many other considerations that are relevant and that could easily tip the balance in favour of some other framing. E.g., it's also good for a framing to be easy to understand and get behind, and relatively unlikely to generate controversy." So then I decided to try to come up with considerations/questions relevant to which framing for MCE advocacy would be best (especially from a longtermist perspective). Here's my initial list: * Which existing or potential future beings actually are moral patients? * And how much moral weight [https://forum.effectivealtruism.org/tag/moral-weight] and capacity for welfare [https://forum.effectivealtruism.org/posts/4Xg3dC6rrW4WFnSne/comparisons-of-capacity-for-welfare-and-moral-status-across] does/will each have? * And how numerous is/will each type of being be? * Which framing will spread the most? I.e., which framing is most memetically [https://forum.effectivealtruism.org/tag/memetics] fit (most memorable, most likely to be shared, etc.)? * Which framing will be most convincing? * Which framing will generate the least opposition, the lowest chance of PR issues, or similar? * E.g., perhaps two framings are both likely to be quite convincing for ~10% of people who come across them, while causing very little change in the beliefs or behaviours of most people who come across them, but one framing is also likely to cause ~10% of people to think the person using that framing is stupid, sanctimonious, and/or immoral. That would of course push against using the latter, more controversial framing. * Which framing will be most likely to change actual behaviours, and especially important ones? * Which framing is most l
3MichaelA5moI also think this is plausible, though I should also note that I don't currently have a strong view on: * whether that's a better bet than other options for moral advocacy * how valuable the best of those actions are relative to other longtermist actions Readers interested in this topic might want to check out posts tagged moral advocacy / values spreading [https://forum.effectivealtruism.org/tag/moral-advocacy-values-spreading], and/or the sources collected here [https://docs.google.com/document/d/1r3ToITGtBX94JPri5gA3dte6An-sPZ4mJfFb_z8jPZw/edit#heading=h.46wkpx249zdc] on the topic of "How valuable are various types of moral advocacy? What are the best actions for that?" (this collection is assocated with my post on Crucial questions for longtermists [https://forum.effectivealtruism.org/posts/wicAtfihz2JmPRgez/crucial-questions-for-longtermists] ).
9Darius_Meissner5moAnother way to approach this is to ensure that people who arealreadyinterested in learning about utilitarianism are able to find high-quality resources that explicitly cover topics like the idea of the expanding moral circle, sentiocentrism/pathocentrism, and the implications for considering the welfare of geographically distant people, other species, and future generations. Improving educational opportunities of this kind was one motivation for writing this section on utilitarianism.net [http://utilitarianism.net]: Chapter 3: Utilitarianism and Practical Ethics: The Expanding Moral Circle [https://www.utilitarianism.net/utilitarianism-and-practical-ethics#the-expanding-moral-circle] .
Possible misconceptions about (strong) longtermism

To be honest I'm not really sure how important there being a distinction between simple and complex cluelessness actually is. The most useful thing I took from Greaves was to realise there seems to be an issue of complex cluelessness in the first place - where we can't really form precise credences in certain instances where people have traditionally felt like they can, and that these instances are often faced by EAs when they're trying to do the most good.

Maybe we're also complexy clueless about what day to conceive a child on, or which chair to sit on, b... (read more)

Possible misconceptions about (strong) longtermism

So far, I feel I've been able to counter any proposed example, and I predict I would be able to do so for any future example (unless it's the sort of thing that would never happen in real life, or the information given is less than one would have in real life).

I think simple cluelessness is a subjective state.  In reality one chair might be slightly older, but one can be fairly confident that it isn't worth trying to find out (in expected value terms). So I think I can probably just modify my example to one where there doesn't seem to be any subjectiv... (read more)

2MichaelA5moI haven't read the relevant papers since last year, but I think I recall the idea being not just that we currently don't have a sense of what the long-term effects of an action are, but also that we basically can't gain information about that. In line with that memory of mine, Greaves writes here [https://forum.effectivealtruism.org/posts/LdZcit8zX89rofZf3/evidence-cluelessness-and-the-long-term-hilary-greaves#Response_five___Go_longtermist_] that the long-term effects of short-termist interventions are "utterly unpredictable" - a much stronger claim that just that we currently have no real prediction. (And I think that that idea is very problematic, as discussed elsewhere in this thread. [https://forum.effectivealtruism.org/posts/ocmEFL2uDSMzvwL8P/possible-misconceptions-about-strong-longtermism?commentId=X4dXQ5eYKmzMfMCtL] .)
2MichaelA5mo(I'm writing this while a little jetlagged, so it might be a bit incoherent or disconnected from what you were saying.) I don't think this is right. I think the key thing is to remember that doing more analysis (thinking, discussing, researching, whatever) is itself a choice, and itself has a certain expected value (which is related to how long it will take, how likely it is to change what other decision you make, and how much of an improvement that change might be). Sometimes that expected value justifies the opportunity cost, and sometimes it doesn't. This can be true whether you can or can't immediately see any difference in the expected value of the two "concrete choices" (this is a term I'm making up to exclude the choice to do further analysis). E.g., I don't spend time deciding which of two similar chairs to sit in, and this is the right decision for me to make from a roughly utilitarian perspective, and this is because: * It seems like that, even after quite a while spent analysing which chair I should sit in, the expected value I assign to each choice would be quite similar * There are other useful things I can do with my time * The expected value of just choosing a chair right away and then doing certain other things is higher than the expected value of first spending longer deciding which chair to sit in (Of course, I don't explicitly go through that whole thought process each time I implicitly make a mundane decision.) But there are also some cases where the expected values we'd guess each of two actions would have are basically the same and yet we should engage in further analysis. This is true when the opportunity cost of the time spent on that analysis seems justified, in expectation, by the probability that that analysis would cause us to change our decision and the extent to which that change might be an improvement. So I don't think the concept of "simple cluelessness" is necessary, and I think it's unhelpful in that: * It so
Possible misconceptions about (strong) longtermism

Your critique of the conception example might be fair actually. I do think it's possible to think up circumstances of genuine 'simple cluelessness' though where, from a subjective standpoint, we really don't have any reasons to think one option may be better or worse than the alternative. 

For example we can imagine there being two chairs in front of us and making a choice of which chair to sit on. There doesn't seem to be any point stressing about this decision (assuming there isn't some obvious consideration to take into account), although it is cert... (read more)

4MichaelA5moSo far, I feel I've been able to counter any proposed example, and I predict I would be able to do so for any future example (unless it's the sort of thing that would never happen in real life, or the information given is less than one would have in real life). Some off-the-top-of-my-head reasons we might not have perfect evidential symmetry here: * One chair might be closer, so walking to it expends less energy and/or takes less time, which has various knock-on effects * One chair will be closer to some other object in the world, making it easier for you to hear what's going on over there and for people over there to hear you, which could have various knock-on effects * One chair might look very slightly older, and thus very slightly more likely to have splinters or whatever I totally agree, but this is a very different claim from there being a qualitative, absolute distinction between simple and complex cluelessness. My independent impression is that, for the purpose of evaluating longtermism and things like that, we could basically replace all discussion of simple vs complex cluelessness with the following points: * You'll typically do a better job achieving an objective (in expectation) if you choose a plan that was highlighted in an effort to try to achieve that objective, rather than choosing a plan that was highlighted in an effort to try to achieve some other objective * This seems like commonsense, and also is in line with the "suspicious convergence" idea * Plans like "donate to AMF" were not highlighted to improve the very long-term future * Plans like "donate to reduce AI x-risk" were highlighted largely to improve the very long-term future * A nontrivial fraction of people highlighted this plan for other reasons (e.g., because they wanted to avoid extinction for their own sake or the sake of near-term generations), but a large fraction highlighted it for approximately longtermist rea
Possible misconceptions about (strong) longtermism

Thanks for all your comments Michael, and thanks for recommending this post to others!

I have read through your comments and there is certainly a lot of interesting stuff to think about there. I hope to respond but I might not be able to do that in the very near future.  

I'd suggest editing the post to put the misconceptions in the headings in quote marks

Great suggestion thanks, I have done that.

The Epistemic Challenge to Longtermism (Tarsney, 2020)

Thanks yeah, I saw this section of the paper after I posted my original comment. I might be wrong but I don't think he really engages in this sort of discussion in the video, and I had only watched the video and skimmed through the paper. 

So overall I think you may be right in your critique. It might be interesting to ask Tarsney about this (although it might be a fairly specific question to ask).

2MichaelA5moYeah, I plan to suggest some questions for Rob to ask Tarsney later today. Perhaps this'll be one of them :)
The Epistemic Challenge to Longtermism (Tarsney, 2020)

OK that's clearer, although I'm not immediately sure why the paper would have achieved the following:

I somewhat updated my views regarding: 

  • how likely such a lock-in is
    • and in particular how likely it is that a state that looks like it might be a lock-in would actually be a lock-in
      • ...

I think Tarsney implies that institutional reform is less likely to be a true lock-in, but he doesn't really back this up with much argument. He just implies that this point is somewhat obvious. Under this assumption, I can understand why his model would lead to the follow... (read more)

2MichaelA5mo[Writing this comment quickly] I think it makes sense to be a bit confused about what claim I'm making and why. I read the paper and made the initial version of these note a few weeks ago, so my memory of what the paper said and how it changed my views is slightly hazy. But I think the key point is essentially the arguably obvious point that the rate of ENEs can be really important, and that that rate seems likely to be much higher when the target state is something like "a very good system of government or set of values" or "a very bad system of government or set of values" (compared to when the target state is whether an intelligent civilization exists). It does seem much more obvious that extinction or non-extinction are each stronger attractor states that particularly good or particularly bad non-extinction outcomes are. This is basically something I already knew, but I think Tarsney's models and analysis made the point a bit more salient, and also made it clearer how important it is (since the rate of ENEs seems like probably one of the most important factors influencing the case for longtermism). But what I've said above kind-of implicitly accepts Tarsney's focus (for the sake of his working example) on simply whether there is an intelligent civilization around, rather than what it's doing. In reality, I think that what the civilization is doing is likely also very important.[1] So the above point about particularly good or particularly bad non-extinction outcomes maybe being only weak attractor states might also undermine the significance of keeping an intelligent civilization around. But here's one way that might not be true: Maybe we think it's easier to have a lock-in of - or natural trends that maintain - a good non-extinction outcome than a bad non-extinction outcome. (I think Ord essentially implies this in The Precipice. I might soon post something related to this. It's also been discussed in some other places, e.g. here.) If so, then the point ab
The Epistemic Challenge to Longtermism (Tarsney, 2020)

In case anyone is interested, Rob Wiblin will be interviewing Tarsney on the 80,000 Hours podcast next week. Rob is accepting question suggestions on Facebook (I think you can submit questions to Rob on Twitter or by email too).

Load More