All of RobertHarling's Comments + Replies

TL;DR: A 'risky' career “failing” to have an impact doesn’t mean your career has “failed” in the conventional sense, and probably isn’t as bad it intuitively feels.

• You can fail to have an impact with your career in many ways. One way to break it down might be:
  • The problem you were trying to address turns out to not be that important
  • Your method for addressing the problem turns out to not work
  • You don’t succeed in executing your plan
• E.g. you could be aiming to have an impact by reducing the risk of future pandemics, and you do this by aiming to become a

Thanks for this post! I think I have a different intuition that there are important practical ways where longtermism and x-risk views can come apart.  I’m not really thinking about this from an outreach perspective, more from an internal prioritisation view. (Some of these points have been made in other comments also, and the cases I present are probably not as thoroughly argued as they could be).
 

• Extinction versus Global Catastrophic Risks (GCRs)
  • It seems likely that a short-termist with the high estimates of risks that Scott describes would focu

Thanks for this interesting analysis! Do you have a link to  Foster's analysis of MindEase's impact?

How do you think the research on MindEase's impact compares to that of GiveWell's top charities? Based on your description of Hildebrandt's analysis for example, it seems less strong than e.g. the several randomized control trials supporting distributing bed nets.  Do you think discounting based on this could substantially effect the cost-effectiveness? (Given how much lower Foster's estimate of impact is though and that this is more heavily used in the overall cost-effectiveness, I would be interested to see whether this has a stronger evidence base?)

Thanks for this post Jack, I found it really useful as I haven't got round yet to reading the updated paper. This break down in the cluelessness section was a new arrangement to me. Does anyone know if this break down has been used elsewhere? If not this seems like useful progress in better defining the cluelessness objections to longtermism. 

Thanks very much for your post! I think this a really interesting idea and it's really useful to learn from your experience in this area. 

What would you think of the concern that these types of ads would be a "low fidelity" way of spreading EA that could risk misinforming people about EA?   I think from my experience community building, it's really useful to be able to describe and discuss EA ideas in detail, and that there are risks to giving someone an incorrect view of EA. These risks include someone being critical of what they believe EA is, ... (read more)

Thanks for sharing this paper, I had not heard of it before and it sounds really interesting.

Thanks for your comment Jack, that's a really great point. I suppose that we would seek to influence AI slightly differently for each reason:

1. Reduce chance of unaligned/uncontrolled AI
2. Increase chance of useful AI
3. Increase chance of exactly aligned AI

e.g. you could reduce the chance of AI risk by stopping all AI development but then lose the other two benefits, or you could create a practically useful AI but not one that would guide humanity towards an optimal future. That being said I reckon in practice a lot of work to improve the development of AI would hit all 3. Though maybe if you view one reason as much more important than the others then you focus on a specific type of AI work.

Thank you very much for this post, I found it very interesting. I remember reading the original paper and feeling a bit confused by it. It's not too fresh in my mind so I don't feel too able to try to defend it. I appreciate you highlighting how the method they use to estimate f_l is unique and drives their main result.

A range of 0.01 to 1 for fl in your preferred model seems surprisingly high to me, though I don't understand the Lineweaver Davis paper well enough to really comment on its result which I think your range is based on.  I think they ment... (read more)

Thanks for your comment athowes. I appreciate your point that I could have done more in the post to justify this "binary" of good and optimal. 

Though the simulated minds scenario I described seems at first to be pretty much optimal, it could be much larger if you thought it would last for many more years. Given large enough uncertainty about future technology, maybe seeking to identify the optimal future is impossible.

I think your resources, value and efficiency model is really interesting. My intuition is that values is the limiting factor. I can bel... (read more)

Thanks again for creating this post Neel. I can confirm I managed to write and publish my post in time! 

I think without commiting to writing it here, my post would either have been made a few months later, or perhaps not been published at all.

Thanks for your comment!

I hadn't thought to think about selection effects, thanks for pointing that out. I suppose Bostrom actually describes black balls as technologies that cause catastrophe but doesn't set the bar as high as extinction. Then drawing a black ball doesn't affect future populations drastically, so perhaps selection effects don't apply?

Also, I think in The Precipice Toby Ord makes some inferences for natural extinction risk given the length of time humanity has existed for? Though I may not be remembering correctly. I think the logic was so... (read more)

Commitment: I commit to writing a post on a vague idea about where most of the value of the long term future is and how sensitive it is to different values by 7pm on 11th December.

Thanks for suggesting this Neel!

Thanks for this post Akash, I found it really interesting to read. I definitely agree with your point about how friendly EAs can be when you reach out to them. I think this is something I've been aware of for a while, but it still takes me time to internalise and make myself more willing to reach out to people. But it's definitely something I want to push myself to do more, and encourage other people to do. No one is  going to be unhappy about someone showing an interest in their work and ideas!

This is a really interesting idea. I think I instinctively have a couple of concerns about such an idea

1) What is the benefit of such statements? Can we expect the opinion of the EA community to really carry much weight beyond relatively niche areas?

2) Can the EA community be sufficiently well defined to collect opinion? It is quite hard to work out who identifies as an EA, not least because some people are unsure themselves. I would worry any attempt to define the EA community too strictly (such as when surveying the community's opinion) could come across as exclusionary and discourage some people from getting involved.

Thanks for your response!

I definitely see your point on the value of information to the future civilisation. The technology required to reach the moon and find the cache is likely quite different to the level required to resurrect humanity from the cache so the information could still be very valuable.

An interesting consideration may be how we value a planet being under human control vs control of this new civilisation. We may think we cannot assume that the new civilisation would be doing valuable things but that a human planet would be quite valuable. ... (read more)

Ah yes! Thanks for pointing that out!

Thanks for your comment, I found that paper really interesting and it was definitely an idea I'd not considered before.

My main two questions would be:

1) What is the main value of humanity being resurrected? - We could inherently value the preservation of humanity and it's culture. However, my intuition would be that humanity would be resurrected in small numbers and these humans might not even have very pleasant lives if they're being analysed or experimented on. Furthermore the resurrected humans are likely to have very little agency, being... (read more)

Thanks for your comment. "Caring about more" is quite a vague way of describing what I wanted to say. I think I was just trying to say that the risk of a true existential event from A is about 7 times greater than the risk from B (as 0.7/0.095 =~ 7.368) so it would be 7 times not 70 times?

Considering evolutionary timelines is definitely very hard because it's such a chaotic process. I don't have too much knowledge about evolutionary history and am hoping to research this more. I think after most human existential events, the complexity of the life that remains would be much greater than that for most of the history of the Earth. So although it took humans 4.6 billion years to evolve "from scratch", it could take significantly less time for intelligent life to re-evolve after an existential event as a lot of the hard evol... (read more)

I believe it is the probability that a nuclear war occurs AND leads to human extinction, as described in The Precipice. I think I would agree that if it was the just the probability of nuclear war, this would be too low, and a large reason the number is small is because of the difficulty for a nuclear war to cause human extinction.

Thanks for the elaboration. I haven't given much consideration to "desired dystopias" before and they are really interesting to consider.

Another dystopian scenario to consider could be one in which humanity "strands" itself on Earth through resource depletion. This could also prevent future life from achieving a grand future.

That makes a lot of sense. If the probability of intelligent life re-evolving is low, or if the probability of it doing morally valuable things is low then this reduces the importance of considering the effect on other species.

Hi Michael, thanks for this comment!

This is a really good point and something I was briefly aware of when writing but did not take the time to consider fully. I've definitely conflated extinction risk with existential risk. I hope that when restricting everything I said just to extinction risk, the conclusion still holds.

A scenario where humanity establishes it's own dystopia definitely seems comparable to the misaligned AGI scenario. Any "locked-in" totalitarian regime would probably prevent the evolution of other intelligent life. This could cause us to increase the risk posed by such dystopian scenarios and weigh these risks more highly.

Thanks for your comment Matthew. This is definitely an interesting effect which I had not considered. I wonder whether though the absolute AI risk may increase, it would not affect our actions as we would have no way to affect the development of AI by future intelligent life as we would be extinct. The only way I could think of to affect the risk of AI from future life would be to create an aligned AGI ourselves before humanity goes extinct!

Hi Michael, thank you very much for your comment.

I was not aware of some of these posts and will definitely look into them, thanks for sharing! I also eagerly await a compilation of crucial questions for longtermists which sounds very interesting and useful.

I definitely agree that I have not given consideration to what moral views re-evolved life would have. This is definitely a big question. One assumption I may have implicitly used but not discussed is that

"While the probability of intelligent life re-evolving may be somewhat soluble and differ b... (read more)

Thanks for your comment! I'm very interested to hear about a modelling approach. I'll look at your model and will probably have questions in the near-future!

Hi, thanks for your questions!

(1) I definitely agree with P1. For P2, would it not be the case that the risk of extinction of humans is strictly greater than the the risk of extinction of humans and future possible intelligent life as the latter is a conjuction of the former? Perhaps a second premise could instead be

P2 The best approaches for reducing human existential risk are not necessarily the best approaches for reducing existential risk to humans and all future possible intelligent life

With a conclusion

C We should focus on the best methods of preve... (read more)

Hi Carl,

Thank you very much for your comment! I agree with your comment on the human extinction risks that 99% is probably not high enough to cause extinction. I think I wanted to provide examples of human extinction event, but should have been more careful on the exact values and situations I described.

On re-evolution after an asteroid impact, my understanding is that although species such as humans eventually evolved after the impact, had humanity existed at the time of the impact it would not have survived as nearly all land mammals over 25kg went ext... (read more)