Effective Altruism Forum
EA Forum

Hide table of contents
Comment Permalink
Answer by Gordon Seidoh Worley10
0
0

Develop and deploy a system to protect Earth from impacts from large asteroids, etc.

Reply
Showing 3 of 4 replies (Click to show all)
Linch3
0
0

Edit: I no longer believe this and now think the expected impact (pun unintended) is more likely negative than positive, including for reasons Ben mentioned.

Reply
14
Ben_Harack
While I'm sympathetic to this view (since I held it for much of my life), I have also learned that there are very significant risks to developing this capacity naively. To my knowledge, one of the first people to talk publicly about this was Carl Sagan, who discussed this in his television show Cosmos (1980), and in these publications: Harris, A., Canavan, G., Sagan, C. and Ostro, S., 1994. The Deflection Dilemma: Use Vs. Misuse of Technologies for Avoiding Interplanetary Collision Hazards. Ben's summary: * Their primary concern and point is that a system built to defend humanity from natural asteroids would actually expose us to more risk (of anthropogenic origin) than it would mitigate (of natural origin). * Opportunities for misuse of the system depend almost solely on the capability of that system to produce delta-V changes in asteroids (equivalently framed as “response time”). A system capable of ~1m/s delta V would be capable of about 100 times as many misuses as its intended uses. That is, it would see ~100 opportunities for misuse for each opportunity for defending Earth from an asteroid. * They say that a high capability system (capable of deflection with only a few days notice) would be imprudent to build at this time. Sagan, C. and Ostro, S.J., 1994. Dangers of asteroid deflection. Nature, 368(6471), p.501. Sagan, C., 1992. Between enemies. Bulletin of the Atomic Scientists, 48(4), p.24. Sagan, C. and Ostro, S.J., 1994. Long-range consequences of interplanetary collisions. Issues in Science and Technology, 10(4), pp.67-72. Two interesting quotes from the last one: * “There is no other way known in which a small number of nuclear weapons can destroy global civilization.” * “No matter what reassurances are given, the acquisition of such a package of technologies by any nation is bound to raise serious anxieties worldwide.” More recently, my collaborator Kyle Laskowski and I have reviewed the relevant technologies (and likely incentives) and
2
MichaelA🔸
You may already be aware of this, and/or the window of relevance may have passed, but just thought I'd mention that Toby Ord discusses a similar matter in The Precipice. He seems to come to roughly similar conclusions to you and to Sagan et al., assuming I'm interpreting everyone correctly. E.g. he writes: This seems like an interesting and important point, and an example of how important it can be to consider issues like downside risks, the unilateralist’s curse, etc. - perhaps especially in the area of existential risk reduction. And apparently even with what we might see as one of the rare "obviously" good options! Something I find slightly odd, and that might conflict with yours or Sagan et al.'s views, was that Ord also wrote: I don't really know anything about this area, but it seems strange to hear that the option involving nuclear explosions is the safer one. And I wonder if the increased amount of explosives, development of tech for delivering it to asteroids, etc., could increase risks independently of asteroid-deflection, such as if it can be repurposed for just directly harming countries on Earth. Or perhaps it could reduce the safety benefits we'd get from having colonies on other moons/planets/asteroids/etc.? Again, though, this is a field I know almost nothing about. And I assume Ord considered these points. Also, obviously there are many nuclear weapons and delivery mechanisms already.
See in context
by Eli Rose
1 min read6 answers 0

22

Existential risk
Frontpage

Consider all the actions possible tomorrow, which individuals or groups could take. Are there any which would "obviously" (meaning: you believe it with high probability, and you expect that belief to be uncontroversial) result in decreased x-risk?

(For example, consider reducing the size of Russia and the US's nuclear stockpiles. I'm curious if this is on the list.)

(I include "which individuals or groups" could take because I am interested in what actions we could take if we all coordinated perfectly. For example, neither Russia nor the US can unilaterally reduce both their stockpiles, and perhaps it would increase x-risk for one of them to lower only theirs, but the group consisting of US and Russia's government could theoretically agree to lower both stockpiles.)

22

0
0

Reactions

0
0
New Answer
New Comment


6 Answers sorted by
Top

Ben_West🔸

18
0
0

ALLFED and related projects like seed banks seem pretty uncontroversially likely to reduce the risk of human extinction.

Max_Daniel10
0
0

If "uncontroversially" means something like "we can easily see that their net effect is to reduce extinction risk," then I disagree. To give just two examples, the known availability of alternative foods might decrease the perceived cost of nuclear war, thus making it more likely; and it might instill a sense of false confidence in decision-makers, effectively diverting attention and funding from more effective risk-reduction measures. I'm perhaps willing to believe that, after weighing up all these considerations, we can all agree that the net effect is to reduce risk, but I think this is far from obvious.

Reply
3
Ben_West🔸
Yeah, if you count "may distract from an even better intervention" as a reason why something is "not obviously good", then I think that basically nothing is obviously good. (Which might be true, just pointing out that this criticism seems pretty general.)
7
Max_Daniel
I agree, and in fact "nothing is obviously good" describes my (tentative) view reasonably well, at least if (i) the bar for 'obviously' is sufficiently high and (ii) 'good' is to be understood as roughly 'maximizing long-term aggregate well-being.' Depending on the question one is trying to answer, this might not be a useful perspective. However, I think that when our goal is to actually select and carry out an altruistic action, this perspective is the right one: I'd want to simply compare the totality of the (wellbeing-relevant) consequences with the relevant counterfactual (e.g., no action, or another action), and it would seem arbitrary to me to exclude certain effects because they are due to a general or indirect mechanism. (E.g., suppose for the sake of the argument that I'm going to die in a nuclear war that would not have happened in a world without seed banks. - I'd think that my death makes the world worse, and I'd want someone deciding about seed banks today to take this disvalue into account; this does not depend on whether the mechanism is that nuclear bombs can be assembled from seeds, or that seed banks have crowded out nuclear deproliferation efforts, or whatever.)
2
Kirsten
I think it's a better idea to first identify ideas that are better than doing nothing - which in itself can be difficult! - and then prioritize those. I think there are talented people who could be convinced to work on the long term future if they are given a task to do which is uncontroversially better than doing nothing. I agree it's better to prioritize actions than just work on the first one you think of, but starting with a bar of 'optimal' seems too high.
6
Max_Daniel
I agree. However, your reply makes me think that I didn't explain my view well: I do, in fact, believe that it is not obvious that, say, setting up seed banks is "better than doing nothing" - and more generally, that nothing is obviously better than doing nothing. I suspect that my appeal to "diverting attention and funding" as a reason for this view might have been confusing. What I had in mind here was not an argument about opportunity cost: while true, I did not want to say that an actor that set up a seed bank could perhaps have done better by doing something else instead (say, donating to ALLFED). Instead, I was thinking of effects on future decisions (potentially by other actors), as illustrated by the following example: * Compare the world in which, at some time t0, some actor A decides to set up a seed bank (say, world w1) with the world w2 in which A decides to do nothing at t0. * It could be the case that, in w2, at some later time t1, a different actor B makes a decision that: * Causes a reduction in the risk of extinction from nuclear war that is larger than the effect of setting up a seed bank at t0. (This could even be, say, the decision to set up two seed banks.) * Happened only because A did not set up a seed bank at t0, and so in particular does not occur in world w1. (Perhaps a journalist in w2 wrote a piece decrying the lack of seed banks, which inspired B - who thus far was planning to become an astronaut - to devote her career to setting up seed banks.) Of course, this particular example is highly unlikely. And worlds w1 and w2 would differ in lots of other aspects. But I believe considering the example is sufficient to see that extinction risk from nuclear war might be lower in world w2 than in w1, and thus that setting up a seed bank is not obviously better than doing nothing.

Gordon Seidoh Worley

10
0
0

Develop and deploy a system to protect Earth from impacts from large asteroids, etc.

Ben_Harack14
0
0

While I'm sympathetic to this view (since I held it for much of my life), I have also learned that there are very significant risks to developing this capacity naively.

To my knowledge, one of the first people to talk publicly about this was Carl Sagan, who discussed this in his television show Cosmos (1980), and in these publications:

Harris, A., Canavan, G., Sagan, C. and Ostro, S., 1994. The Deflection Dilemma: Use Vs. Misuse of Technologies for Avoiding Interplanetary Collision Hazards.

Ben's summary:

  • Their primary concern and point is that a system built to defend humanity from natural asteroids would actually expose us to more risk (of anthropogenic origin) than it would mitigate (of natural origin).
  • Opportunities for misuse of the system depend almost solely on the capability of that system to produce delta-V changes in asteroids (equivalently framed as “response time”). A system capable of ~1m/s delta V would be capable of about 100 times as many misuses as its intended uses. That is, it would see ~100 opportunities for misuse for each opportunity for defending Earth from an asteroid.
  • They say that a high capability system (capable of deflection with o
... (read more)
Reply
2
MichaelA🔸
You may already be aware of this, and/or the window of relevance may have passed, but just thought I'd mention that Toby Ord discusses a similar matter in The Precipice. He seems to come to roughly similar conclusions to you and to Sagan et al., assuming I'm interpreting everyone correctly. E.g. he writes: This seems like an interesting and important point, and an example of how important it can be to consider issues like downside risks, the unilateralist’s curse, etc. - perhaps especially in the area of existential risk reduction. And apparently even with what we might see as one of the rare "obviously" good options! Something I find slightly odd, and that might conflict with yours or Sagan et al.'s views, was that Ord also wrote: I don't really know anything about this area, but it seems strange to hear that the option involving nuclear explosions is the safer one. And I wonder if the increased amount of explosives, development of tech for delivering it to asteroids, etc., could increase risks independently of asteroid-deflection, such as if it can be repurposed for just directly harming countries on Earth. Or perhaps it could reduce the safety benefits we'd get from having colonies on other moons/planets/asteroids/etc.? Again, though, this is a field I know almost nothing about. And I assume Ord considered these points. Also, obviously there are many nuclear weapons and delivery mechanisms already.
Linch7
0
0

This is the only answer here I'm moderately confident is correct. A pity the EV is so low!

Reply
3
Linch
Edit: I no longer believe this and now think the expected impact (pun unintended) is more likely negative than positive, including for reasons Ben mentioned.

Kirsten

9
0
0

I'd suggest work that would allow vaccines to be developed much more quickly falls into this category - it was mentioned in the 80,000 Hours podcast with Tom Kalil.

"I was able to get some additional funding for this new approach [to develop vaccines more quickly] and my primary motivation for it was, maybe it’ll help in Ebola, but almost certainly if it works it will improve our ability to respond to future emerging infectious diseases, or maybe even a world of engineered pathogens."

https://80000hours.org/podcast/episodes/tom-kalil-government-careers/

Max_Daniel10
0
0

Again, I disagree this is obvious. Just some ways in which this could be negative:

  • It could turn out that some of the research required for rapid vaccine development can be misused or exacerbate other risk.
  • The availability of rapid vaccine manufacturing methods could lead to a false sense of confidence among decisionmakers, leading to them effectively neglecting other important prevention and mitigation measures against biorisk.
Reply

Pablo

8
0
0

In this talk on 'Crucial considerations and wise philanthropy', Nick Bostrom tentatively mentions some actions that appear to be robustly x-risk reducing, including promoting international peace and cooperation, growing the effective altruism movement, and working on solutions to the control problem.

LintzA3
0
0

Just to play devil's advocate with some arguments against peace (in a not so well thought out way)... There's a book called 'The Great Leveler' which puts forward the hypothesis that the only time when widespread redistribution has happened is after wars. This means that without war we might expect consistently rising inequality. This effect has been due to mass mobilization ('Taxing the Rich' asserts that there has only been mass political willpower to increase redistribution with the claims of veterans having served and feeling they should be compensated

... (read more)
Reply
7
Pablo
This seems like point worth highlighting, especially vis-à-vis Bostrom's own views about the importance of global governance in 'The Vulnerable World Hypothesis'. Worth also noting that the League of Nations was created in the aftermath of WW1.
8
Ben_Harack
This line of inquiry (that rebuilding after wars is quite different from other periods of time) is explored in G. John Ikenberry's After Victory: Institutions, Strategic Restraint, and the Rebuilding of Order After Major Wars. A quick and entertaining summary of the book - and how it has held up since its publication - was written by Ikenberry in 2018: Reflections on After Victory.
4
Pablo
Thank you for those references!

RomeoStevens

6
0
0

Increasing the ease/decreasing the formality of world leaders talking to each other as per the Red Phone. World leaders mostly getting educated at the same institutions helps enormously with communication as well, though it does increase other marginal risks due to correlated blind spots.

Biorisk mitigation becoming much higher status a field and thus attracting more top talent.

Pakistan not having nukes.

Max_Daniel9
0
0

Again, I disagree that any of these is obvious:

  • Ease of communication also opens up more opportunities for rash decisions and premature messages, can reduce the time available for decisions, and the potential for this infrastructure to be misused by malign actors.
  • Biorisk mitigation being higher status could contribute to making the dangers of bioweapons more widely known among malign actors, thus making it more likely that they're being developed.
  • Pakistan not having nukes would alter the geopolitical situation in South Asia in major ways, with repercussions for the relationships between the major powers India, China, and the US. I find it highly non-obvious what the net effect of this would be.
Reply
2
RomeoStevens
I'd suggest keeping brainstorming and debates about obviousness thresholds separate as the latter discourages people from ideating.
8
Pablo
Personally, I don't find that skeptical comments like Max's discourage me from ideating. And the suggestion to keep ideation and evaluation separate might discourage the latter, since it's actually not obvious how to operationalize 'keeping separate'.
6
Kirsten
I've previously read a study that suggested evaluation during brainstorming led to less ideas - I don't remember where. Personally, I feel less inclined to post when I know someone will tell me my idea is wrong. Edit: A Harvard Business Review article about brainstorming and 'evaluation anxiety' led me to this article, which I have not been able to read yet. https://hbr.org/2015/03/why-group-brainstorming-is-a-waste-of-time https://onlinelibrary.wiley.com/doi/abs/10.1111/1467-8616.00154
[anonymous]16
0
0

A general comment about this thread rather than a reply to Khorton in particular: The original post didn't suggest that this should be a brainstorming thread, and I didn't interpret it like that. I interpreted it as a question looking for answers that the posters believe, rather than only hypothesis generation/brainstorming.

Reply
4
Larks
When I was studying maths it was made clear to us that some things were obvious, but not obviously obvious. Furthermore, many things I thought were obvious were in fact not obvious, and some were not even true at all!
4
Max_Daniel
Thanks for this suggestion. I agree that in general brainstorming and debates are best kept separate. I also wouldn't want to discourage anyone from posting an answer to this question - in fact, I'm unusually interested in more answers to this question. I'm not sure if you were saying that you in particular feel discouraged from ideating as a response to seeing my comment, but I'm sorry if so. I'm wondering if you would have liked me to explain why I was expressing my disagreement, and to explicitly say that I value you suggesting answers to the original question (which I do)?
1
Gordon Seidoh Worley
+1 Further the OP gives a specific notion of obviousness to use here: This doesn't leave a lot of room for debate about what is "obvious" unless you want to argue that a person doesn't believe it with high probability and they are wrong about their own belief about how controversial it is.

atlas

2
0
0
  • Most actions that seem to make arms races or war more unlikely, e.g. the world's major powers committing to strengthening international institutions and multilateralism.
  • Any well-connected and well-resourced actor dedicating themselves to research ways to improve decision-making that affects the long-term in large institutions.
  • Everyone in the AI research community taking a few weeks to engage deeply with AI risk arguments.

Curated and popular this week
Paul Present
 ·  · 28m read
 · 
Note: I am not a malaria expert. This is my best-faith attempt at answering a question that was bothering me, but this field is a large and complex field, and I’ve almost certainly misunderstood something somewhere along the way. Summary While the world made incredible progress in reducing malaria cases from 2000 to 2015, the past 10 years have seen malaria cases stop declining and start rising. I investigated potential reasons behind this increase through reading the existing literature and looking at publicly available data, and I identified three key factors explaining the rise: 1. Population Growth: Africa's population has increased by approximately 75% since 2000. This alone explains most of the increase in absolute case numbers, while cases per capita have remained relatively flat since 2015. 2. Stagnant Funding: After rapid growth starting in 2000, funding for malaria prevention plateaued around 2010. 3. Insecticide Resistance: Mosquitoes have become increasingly resistant to the insecticides used in bednets over the past 20 years. This has made older models of bednets less effective, although they still have some effect. Newer models of bednets developed in response to insecticide resistance are more effective but still not widely deployed.  I very crudely estimate that without any of these factors, there would be 55% fewer malaria cases in the world than what we see today. I think all three of these factors are roughly equally important in explaining the difference.  Alternative explanations like removal of PFAS, climate change, or invasive mosquito species don't appear to be major contributors.  Overall this investigation made me more convinced that bednets are an effective global health intervention.  Introduction In 2015, malaria rates were down, and EAs were celebrating. Giving What We Can posted this incredible gif showing the decrease in malaria cases across Africa since 2000: Giving What We Can said that > The reduction in malaria has be
Neel Nanda
 ·  · 1m read
 · 
TL;DR Having a good research track record is some evidence of good big-picture takes, but it's weak evidence. Strategic thinking is hard, and requires different skills. But people often conflate these skills, leading to excessive deference to researchers in the field, without evidence that that person is good at strategic thinking specifically. I certainly try to have good strategic takes, but it's hard, and you shouldn't assume I succeed! Introduction I often find myself giving talks or Q&As about mechanistic interpretability research. But inevitably, I'll get questions about the big picture: "What's the theory of change for interpretability?", "Is this really going to help with alignment?", "Does any of this matter if we can’t ensure all labs take alignment seriously?". And I think people take my answers to these way too seriously. These are great questions, and I'm happy to try answering them. But I've noticed a bit of a pathology: people seem to assume that because I'm (hopefully!) good at the research, I'm automatically well-qualified to answer these broader strategic questions. I think this is a mistake, a form of undue deference that is both incorrect and unhelpful. I certainly try to have good strategic takes, and I think this makes me better at my job, but this is far from sufficient. Being good at research and being good at high level strategic thinking are just fairly different skillsets! But isn’t someone being good at research strong evidence they’re also good at strategic thinking? I personally think it’s moderate evidence, but far from sufficient. One key factor is that a very hard part of strategic thinking is the lack of feedback. Your reasoning about confusing long-term factors need to extrapolate from past trends and make analogies from things you do understand better, and it can be quite hard to tell if what you're saying is complete bullshit or not. In an empirical science like mechanistic interpretability, however, you can get a lot more fe
 ·  · 15m read
 · 
“I” refers to Zach, the Centre for Effective Altruism's CEO. Oscar is CEA’s Chief of Staff. We are grateful to all the CEA staff and community members who have contributed insightful input and feedback (directly and indirectly) during the development of our strategy and over many years. Mistakes are of course our own. Exec summary As one CEA, we are taking a principles-first approach to stewardship of the EA community. During the search for a new CEO, the board and search committee were open to alternative strategic directions, but from the beginning of my tenure, we’ve committed to a strategy under which we will: * Operate as one CEA, rather than winding down, breaking up or renaming the organization. Instead of optimizing for each of our team’s programs, we’ll be optimizing for EA as a whole. * Take a principles-first approach to EA, rather than becoming an AI org or otherwise re-orienting ourselves to specific causes. * Take greater responsibility for stewardship of the EA community, rather than restricting ourselves to passively providing infrastructure and support. This post explores stewardship in greater detail. Stewardship is about actors taking more responsibility for reaching and raising EA’s ceiling, and we believe CEA should play a leading role in steering, supporting and coordinating the community. Importantly, however, stewardship of EA is not ownership of EA: we don’t want to be the only leaders, and we do want a close collaboration with the community. During 2024 we focussed on building strong foundations that CEA will require to succeed at stewarding the community, including making over 20 hires (having started the year with 34 staff) while cutting a quarter of our costs, and developing our strategy for 2025 and 2026, including by listening to and learning from members of the EA community during visits I made to over half a dozen countries and in more than 200 one-on-one meetings. I feel good about the foundations we built and having priori
Recent opportunities in Existential risk
35
Center on Long-Term Risk: Summer Research Fellowship 2025
Center on Long-Term Risk
· · 1m read
0
0
78
AI Moral Alignment: The Most Important Goal of Our Generation
Ronen Bar
· · 10m read
14
14
28
Apply to the Cambridge ERA:AI Fellowship 2025
Harrison 🔸
· · 3m read
0
0
View more