All of Denkenberger's Comments + Replies
I think the main reason that EA focuses relatively little effort on climate change is that so much money is going to it from outside of EA. So in order to be cost effective, you have to find very leveraged interventions, such as targeting policy, or addressing extreme versions of climate change, particularly resilience, e.g. ALLFED (disclosure, I'm a co-founder).
I have been recently asking around whether someone has compiled how much money is going into different ways of mitigating GCBRs, so this is quite relevant! Do you have estimates of the current EA (or otherwise) spending in these or similar buckets?
- Prevention: AI misuse, DNA synthesis screening, etc
- Suppression: Pathogen-agnostic early warning, planning for rapid response lockdowns, etc
- Containment: UV systems, P4E stockpiling, plans for keeping vital workers onsite, backup plans for providing food, energy and water non-industrially with low human contac
I think this is a very valuable project.
...But this is still a combination of two questions, the latter of which longtermists have never, to my knowledge, considered probabilistically:[3]
- What is the probability that the event kills all living humans?
- What effect does the event otherwise have on the probability that we eventually reach an interstellar/existentially secure state, [4] given the possibility of multiple civilisational collapses and ‘reboots’? (where the first reboot is the second civilisation)
3^
The closest thing I know to such an attempt
Thanks for mentioning resilient foods! It is true that more food storage would give more time to scale up resilient foods. Stored food could be particularly valuable for some countries in loss of trade scenarios. Some have suggested that getting the World Trade Organization to change its rules would result in more food storage automatically. Still, I think the priority now is spending a few hundred million dollars total on resilient foods to research, pilot, and plan for them. If we extend your proposal for 20 years and for the world, then you are up to ~$...
I love the cumulative probability graph!
There is a little probability mass on things which are a reasonable fraction of the great or hellish futures — mostly corresponding to worlds in which the lightcone is divided in some way
- Trade means that the probability of such outcomes isn’t so high, and I’ll set them aside for now; however, I think that this would be a natural place to extend this analysis
Let's say the positive side of your graph has a logarithmic horizontal axis. I think there would be some probability mass that we have technological stagnation an...
...I think the reviewer may be concluding from the above that, given no international food trade, calorie consumption would be much lower, and therefore increasing food production via new food sectors would become much more important relative to distribution. I agree with the former, but not the latter. Loss of international food trade is more of a problem of food distribution than production. If this increased thanks to new food sectors, but could not be distributed to low-income food-deficit countries (LIFDCs) due to loss of trade, there would still be many
This was very helpful! I found the diagrams particularly useful. Visible lighting design for rooms has a similar problem of uniform illumination, but it is mitigated by the fact that there is significant reflection of the light, which I presume does not apply for far UVC.
Has there been any work on planning to relocate existing UV systems to the most critical tasks, if an extreme pandemic hit soon, of making more super PPE/UV systems?
...One unpublished study by a Russian academic and a CDC researcher allegedly estimated that the cost of 1 ACH by ventilat
Why is flesh weaker than diamond?
I don't think this is a fair comparison. If nature wanted skin to be harder, it can do that, for instance with scales (particularly hard in the case of turtle shells). Of course your logic explains why diamond is harder than bone. But if you want a small thing that could penetrate flesh, we already have it in the form of parasites.
One of the points in the book Strangers Drowning was that very dedicated altruists (some EAs included) live like it is war time all the time. Basically, the urgency of people dying from poverty, animals suffering, and humanity's future at risk demand the sacrifices that are typically reserved for war time. Another example is if existential risk were high, some argue that we should be on "extreme war footing" and dedicate a large portion of society's resources to reducing the risk. I'm interested in your perspective on these thoughts.
This is a decent summary, but there are a couple corrections:
ALLFED increased paid team members, but much less than doubled (we have capacity to expand more quickly with additional funding).
We do have 17 advisory board members, but they represent 4 countries, not 9 (the 9 countries were represented by the 17 team members at the retreat).
Nice post!
The model does not predict much differences between the different scenarios until 2020-2030. Therefore, we only know that the model has not been falsified so far, but it is still unclear what the path is we are currently on.
I think it would be helpful to see an overlay of our actual trajectory. Though the absolute values of the models are not that different for the period 2000 to 2020, the slopes are quite different. I think there was a paper analyzing the fits including the slopes. The increase of production of food since the year 2000 has been ...
Thanks for all you have done!
Finally, EAs have treated EtG as increasingly more weird, especially offline, defeating the original argument for engaging.
This is very disappointing, especially because, if you disregard "still deciding", EtG was the second most popular route to impact among EAs in the 2022 survey.
(leading a - dare I say - successful effective nonprofit)
Sure - go ahead and dare. :)
My day job is associate professor of mechanical engineering at University of Canterbury in New Zealand, and I volunteer for ALLFED. Nearly 100% of my donations are to ALLFED. I think that ALLFED is the most cost-effective way of improving the long run future at the margin (see here and here, though I'm not quite as bullish as the mean survey/poll results in those papers), but there are orders of magnitude of uncertainty, and I think more total money should be put into AGI ...
As one who donates 50%, it doesn't seem like it should be that uncommon. One way I think about it is earning like upper-middle-class, living like middle-class, and donating like upper-class. Tens of percent of people work for tens of percent less money in sectors like nonprofits and governments. And I've heard of quite a few non-EAs who have taken jobs for half the money. And yet most people think about donating that large of a percent very differently than taking a job that pays less. I'm still not sure why - other than that it is uncommon or "weird."
I agree that most academic research is a bad ROI but I find that a lot of this sort of 'nobody reads research' commentary is equating reads with citations which seems completely wrong. By that metric most forum posts would also not be read by anyone.
I agree-for one, the studies I've seen saying that the median publication is not cited are including conference papers, so if one is talking about the peer-reviewed literature, citations are significantly greater. I've estimated the average number of citations per paper is around 30 for the peer-reviewed litera...
I was assuming 50 % reduction in international trade, and 50 % of that reduction being caused by climatic effects, so only 25 % (= 0.5^2) caused by climatic effects. I have changed "50 % of it" to "50 % of this loss" in my original reply to clarify.
That makes sense. Thanks for putting the figure in!
I guess famine deaths due to the climatic effects are described by a logistic function, which is a strictly increasing function, so I agree with the above. However, I guess the increase will be pretty small for low levels of soot.
If it were linear starting...
Very interesting!
- Space colonies. Fertility is low in wealthy countries with large unsettled territories (Canada, Australia), even though they are far more hospitable than other planets. There is no reason to think that space colonies alone will reverse the fertility decline.
I think the incentive for fertility depends on the level of connection with the Earth. If it were fully independent from Earth, it would have a strong incentive to increase population because there are large economies of scale in terms of increasing the standard of living, including being able to create more living space per person, more advanced electronics, media, etc.
>Half of the impact of the total loss of international food trade would cause 2.6% to >die according to Xia 2022. So why is it not 4.43%+2.6% = 7.0% mortality?
In my BOTEC with "arguably more reasonable assumptions", I am assuming just a 50 % reduction in international food trade, not 100 %.
That's why I only attributed half of the impact of total loss of international food trade. If I attributed all the impact, it would have been 4.43%+5.2% = 9.6% mortality. I don't see how you are getting 5.67% mortality.
...My famine deaths due to the climatic effects a
For arguably more reasonable assumptions of 50 % loss of international food trade, and 50 % of it being caused by the climatic effects, linearly interpolating, the increase in the death rate would be 25 % (= 0.5^2). So the new death rate would be 5.67 % (= 0.0443 + (0.0940 - 0.0443)*0.25), i.e. 1.28 (= 0.0567/0.0443) times my value.
Half of the impact of the total loss of international food trade would cause 2.6% to die according to Xia 2022. So why is it not 4.43%+2.6% = 7.0% mortality?
...It is still the case that I would get a negative death rate inputting 5
In that case, I would only be overestimating the amount of soot by 10 %, which is a small factor in the context of the large uncertainty involved (my 95th percentile famine deaths due to the climatic effects is 62.3 times my best guess).
Do you mean underestimating? I agree that it's not that large of an effect.
...For reference, maintaining my famine deaths due to climatic effects negligible up to an injection of soot into the stratosphere of 11.3 Tg, if I had assumed a total loss of international food trade fully caused by the climatic effects, I would have o
I thought this was comprehensive, and it was clever how you avoided doing a Monte Carlo simulation for most of the variables. The expected amount of soot to the stratosphere was similar to my and Luisa's numbers for a large-scale nuclear war. So the main discrepancies are the expected number of fatalities and the impact on the long-term future.
From Figure 4 of Wagman 2020, the soot injected into the stratosphere for an available fuel per area of 5 g/cm^2 is negligible[14].
At 5 g/cm^2, Still most of soot makes it into the upper troposphere, so I think ...
See the reply to the first comment on that post. Paul's "most humans die from AI takeover" is 11%. There are other bad scenarios he considers, like losing control of the future, or most humans die for other reasons, but my understanding is that the 11% most closely corresponds to doom from AI.
It is good that 80k is making simple videos to explain the risks associated with EA
Do you mean "risks associated with AI"?
Were these commenters expecting it to be much cheaper to save a life by preventing the loss of potential in an extinction, than to save a life using near-termist interventions?
I think that commenters are looking at the cost-effectiveness they could reach with current budget constraints. If we had way more money for longtermism, we could go to a higher cost per basis point. That is different than the value of reducing a basis point, which very well could be astronomical, given GiveWell costs for saving a life (though to be consistent, one should try to estimate the long-term impacts of a GiveWell intervention as well).
A nuclear war into a supervolcano is just really unlikely.
A nuclear war happening at the same time as a supervolcano is very unlikely. However, it could take a hundred thousand years to recover population, so if the frequency of supervolcanic eruptions is roughly every 30,000 years, it's quite likely there would be one before we recover.
Plus if there were 1000 people then there would be so much human canned goods left over - just go to a major city and sit in a supermarket.
The scenario I'm talking about is one where the worsening climate and loss of techno...
Neglectedness in the classic sense. Although not as crowded as climate change, there are other large organizations / institutions that address nuclear risk and have been working in this space since the early Cold War.
I agree that the nuclear risk field as a whole is less neglected than AGI safety (and probably than engineered pandemic), but I think that resilience to nuclear winter is more neglected. That's why I think overall cost-effectiveness of resilience is competitive with AGI safety.
I'm not Matt, but I do work on nuclear risk. If we went down to 1000 to 10,000 people, recovery would take a long time, so there is significant chance of supervolcanic eruption or asteroid/comet impact causing extinction. People note that agriculture/cities developed independently, indicating it is high probability. However, it only happened when we had a stable moderate climate, which might not recur. Furthermore, the Industrial Revolution only happened once, so there is less confidence that it would happen again. In addition, it would be more difficult w...
He did it because he felt good doing it, and also to be healthy. He started thinking more and more about meaninglessness in maintaining for long-term health.
I think it's also helpful to point out that we should be good Bayesians and not believe anything 100%. It seems to me plausible that in 20 years, AI may not change everything, but maybe we will be able to reverse aging (or maybe AI will change everything and we can upload our brains). With some chance of indefinite lifespan, I think some effort into health in the next 20 years even if one is relatively young could have a big expected value.
I would recommend patenting, but then committing to donate part of the profits. That has been my strategy.
Actually, I think they only simulate the fires, and therefore soot production, for 40 min. So you may well have a good point. I do not know whether it would be a difference by a factor of 10. Figure 6 of Reisner 2018 may be helpful to figure that out, as it contains soot concentration as a function of height after 20 and 40 min of simulation. Do the green and orange curves look like they are closely approaching stationary state?
Wow - only 40 minutes - my understanding is actual firestorms take hours. This graph is for the low loading case, which did not pr...
For the high fuel load of 72.62 g/cm^2, Reisner 2019 obtains a firestorm
Thanks for the correction. Unfortunately, there is no scale on their figure, but I'm pretty sure the smoke would be going into the upper troposphere (like Livermore finds). Los Alamos only simulates for a few hours, so that makes sense that hardly any would have gotten to the stratosphere. Typically it takes days to loft to the stratosphere. So I think that would resolve the order of magnitude disagreement on percent of soot making it into the stratosphere for a firestorm.
I think the s...
Los Alamos: Even for a fuel loading of 72.62 g/cm^2, it is 6.21 % (= 0.196/3.158).
So basically, no matter how much fuel Los Alamos puts in, they cannot reproduce the firestorms that were observed in World War II. I think this is a red flag for their model (but in fairness, it is really difficult to model combustion - I've only done computational fluid dynamics modeling - combustion is orders of magnitude more complex).
Thanks for clarifying. If instead one uses a mean (though I do think the tails should be weighted more heavily) closer to Luisa's and my analysis of 30 Tg, then Xia predicts about 1.6 billion starvation fatalities and about 110 million direct fatalities (though this latter number would probably be higher because Xia assumes that all areas hit would firestorm, which I don't, so I think more area would be hit to produce that amount of soot to the stratosphere). This is pessimistic in that it assumes no international food trade, no planting location adaptatio...
This study found 5 billion dead, but this is an obvious overestimate for a realistic response to massive global cooling. I suspect the death estimates they give are out by at least two orders of magnitude, given the various unrealistic assumptions they use
50 million dying from starvation (more than 50 million would die from the direct impacts of the nuclear war) is possible with a ~90% reduction in non-adapted agriculture (with current applications of fertilizers, pesticides, etc), but trade, resilient foods, and subsidies would have to go very well. I hav...
Deep bunkers like that are expensive and rare, and even if the bunker itself survived, ground bursts are messy and would likely leave it inaccessible.
There are thousands of underground mines in the US (14000 active mines, but many are surface mines), and I think it would only require 1 or a few to store thousands of nuclear weapons. Maybe the weapons would be spread out over many mines. It would not be feasible to make thousands of mines inaccessible.
...Missile warheads are only of use as a source of raw materials, and while you might be able to g
First, remind me why we're looking at 1500 countervalue weapons? Do we really expect them to just ignore the ICBM silos?
My understanding is that the warning systems are generally designed such that the ICBMs could launch before the attacking warheads reach the silos. I do have significant probability on counterforce scenarios, but I can't rule out counter value scenarios, so I think it's an important question to estimate what would happen in these counter value scenarios.
...Possibly the single most important goal of the deployed warheads is
US suburbs may have a lot of building mass in aggregate, but it's also really spread out and generally doesn't contain that much which is likely to draw nuclear attack.
There are only 55 metropolitan areas in the US with greater than 1 million population. Furthermore, the mostly steel/concrete city centers are generally not very large, so even with a nuclear weapon targeted at the city center, it would burn a significant amount of suburbs. So with 1500 nuclear weapons countervalue even spread across NATO, a lot of the area hit would be suburbs.
...Yeah, sorry,
It argues Toon 2008 has overestimated the soot ejected into the stratosphere following a nuclear war by something like a factor of 191[1] (= 1.5*2*2*(1 + 2)/2*(2 + 3)/2*(4 + 13)/2).
I think a geometric mean would be more appropriate, so (48*468)^0.5 = 150. But I disagree with a number of the inputs.
They also assume 4,400 warheads from the US and Russia alone, significantly higher than current arsenals.
Current US + Russia arsenals are around 11,000 warheads, but current deployed arsenals are only about 3000. With Putin pulling out of New START, many nuc...
>Current US + Russia arsenals are around 11,000 warheads, but current deployed arsenals are only about 3000. With Putin pulling out of New START, many nuclear weapons that are not currently deployed could become so.
Possibly the single most important goal of the deployed warheads is to stop the other side from deploying their warheads, both deployed and non-deployed. Holding to deployed only is probably a reasonable assumption given that some of the deployed will not make it, and most of the non-deployed definitely won't. And this was written...
A counterfactual marginal multiplier of z means the effective giving organisation would have caused z $ of donations for each additional dollar it had spent...
The effective giving organisations is underfunded if z < 1, as long as the counterfactual marginal multiplier includes all relevant effects.
Do you mean z > 1?
I agree that Will's statement is correct for the near term. But Will also said that his vision is that, like science is the agreed way of getting to the truth, EA should be the agreed way of getting to the good. I think that would imply that EA has become a mass movement.
Thanks for the link. This shows that 3% of global wealth is in billionaires. Though richer people generally give a larger percent of their income, it's not clear they give a larger percent of their wealth. This is because many people with near zero wealth still have significant income, and still donate to charity. So I would guess ~3% of donations from individuals/foundations would be from billionaires. Corporations you point out are 6% of the US total. It's not clear to me how to classify this, but generously you could go with market capitalization. I wou...
Wealth is heavily fat-tailed, so it’s very likely that one or a small number of funders end up accounting for most funding.
Most philanthropy is not from billionaires, so the fact that most EA philanthropy is from billionaires means that EA has been unusually successful at recruiting billionaires. This could continue, or it could mean revert. So I do think there is hope for more funding diversification.
I'm curious how you would count endowments. For instance, Princeton has an endowment equal to about 10 years of expenditure, and about 17 years of expenditure net of non-philanthropic income. My understanding is that most of this would be restricted, e.g. to scholarships or athletics. So if 20% were unrestricted, would that mean you would calculate 2 or 3.4 years of unrestricted runway?
Wouldn't interstellar travel close to the speed of light require a huge amount of energy, and a level of technological transformation that again seems much higher than most people expect?
Not really - about six hours of the energy produced by the sun. If molecular manufacturing could double every day (many bacteria double much faster), we would get there very fast.
I think that saving lives in a catastrophe could have more flow-through effects, such as preventing collapse of civilization (from which we may not recover), reducing the likelihood of global totalitarianism, and reducing the trauma of the catastrophe, perhaps resulting in better values ending up in AGI.