'Predicted' in the title is pretty clickbaity/misleading given that the market was created and driven by insider traders. 'Knew About' or 'Leaked Information About' seem much more accurate.
Otherwise, I found this very interesting. I hadn't heard of this market before, and appreciate the analysis of what seems like it might be a very important case study, both for how to handle the leaking of embargoed or otherwise sensitive information and for what to do about insider trading.
I like that you can interact with this. It makes understanding models so much easier.
Playing with the calculator, I see that the result is driven to a surprising degree by the likelihood that "Compute needed by AGI, relative to a human brain (1e20-1e21 FLOPS)" is <1/1,000x (i.e. the bottom two options).[1]
I think this shows that your conclusion is driven substantially by your choice to hardcode "1e20-1e21 FLOPS" specifically, and then to treat this figure as a reasonable proxy for what computation an AGI would need. (That is, you suggest ~~1x as the mid...
It seems extremely clear that working with the existing field is necessary to have any idea what to do about nuclear risk. That said, being a field specialist seems like a surprisingly small factor in forecasting accuracy, so I’m surprised by that being the focus of criticism.
I was interested in the criticism (32:02), so I transcribed it here:
...Jeffrey Lewis: By the way, we have a second problem that arises, which I think Wizards really helps explain: this is why our field can’t get any money.
Aaron Stein: That’s true.
Jeffrey Lewis: Because it’s extremely har
During the calls I've offered to people considering applying to work as a grantmaker, I've sent this post to about 10 people already. Thanks for writing it!
I agree that this is the most common misconception about grantmaking. To be clear, (as we've discussed) I think there are some ways to make a difference in grantmaking which are exceptions to the general rule explained here, but think this approach is the right one for most people.
This was the single most valuable piece on the Forum to me personally. It provides the only end-to-end model of risks from nuclear winter that I've seen and gave me an understanding of key mechanisms of risks from nuclear weapons. I endorse it as the best starting point I know of for thinking seriously about such mechanisms. I wrote what impressed me most here and my main criticism of the original model here (taken into account in the current version).
This piece is part of a series. I found most articles in the series highly informative, but this particula...
For those interested in Triplebyte's approach, there's also Kelsey Piper's thoughts on why and how the company gives feedback, and why others don't.
Thanks for this.
Without having the data, it seems the controversy graph could be driven substantially by posts which get exactly zero downvotes.
Almost all posts get at least one vote (magnitude >= 1), and balance>=0, so magnitude^balance >=1. Since the controversy graph goes below 1, I assume you are including the handling which sets controversy to zero if there are zero downvotes, per the Reddit code you linked to.
e.g. if a post has 50 upvotes:
0 downvotes --> controversy 0 (not 1.00)
1 downvote --> controversy 1.08
2 downvotes --> controve...
(I downvoted this because a large fraction of the basic facts about what organisations are doing appear to be incorrect. See other comments. Mostly I think it's unfortunate to have incorrect things stated as fact in posts, but going on to draw conclusions from incorrect facts also seems unhelpful.)
I'm totally not a mod, but I thought I'd highlight the "Is it true? Is it necessary? Is it kind?" test. I think it's right in general, but especially important here. The Forum team seems to have listed basically this too: "Writing that is accurate, kind, and relevant to the discussion at hand."
I'm also excited to highlight another piece of their guidance "When you disagree with someone, approach it with curiosity: try to work out why they think what they think, and what you can learn from each other." On this:
I think this is the best intro to investing for altruists that I've seen published. The investment concepts it covers are the most important ones, and the application to altruists seems right.
(For context: I used to work as a trader, which is somewhat but not very relevant, and have thought about this kind of thing a bit.)
I would guess that the decision of which GiveDirectly programme to support† is dominated by the principle you noted, of
the dollar going further overseas.
Maybe GiveDirectly will, in this case, be able to serve people in the US who are in comparable need to people in extreme poverty. That seems unlikely to me, but it seems like the main thing to figure out. I think your 'criteria' question is most relevant to checking this.
† Of course, I think the most important decision tends to be deciding which problem you aim to help solve, which would precede the question of whether and which cash transfers to fund.
The donation page and mailing list update loosely suggest that donations are project-specific by default. Likewise, GiveWell says:
GiveDirectly has told us that donations driven by GiveWell's recommendation are used for standard cash transfers (other than some grant funding from Good Ventures and cases where donors have specified a different use of the funds).
(See the donation page for what the alternatives to standard cash transfers are.)
If funding for different GiveDirectly projects are sufficiently separate, your donation would pretty much just incr...
For the record, I wouldn't describe having children to 'impart positive values and competence to their descendants' as a 'common thought' in effective altruism, at least any time recently.
I've been involved in the community in London for three years and in Berkeley for a year, and don't recall ever having an in-person conversation about having children to promote values etc. I've seen it discussed maybe twice on the internet over those years.
--
Additionally: This seems like an ok state of affairs to me. Having childre...
In the '2% RGDP growth' view, the plateau is already here, since exponential RGDP growth is probably subexponential utility growth. (I reckon this is a good example of confusion caused by using 'plateau' to mean 'subexponential' :) )
In the 'accelerating view', it seems that whether there is exponential utility growth in the long term comes down to the same intuitions about whether things keep accelerating forever that are discussed in other threads.
Thanks!
In my understanding, [a confident focus on extinction risk] relies crucially on the assumption that the utility of the future cannot have exponential growth in the long term
I wanted to say thanks for spelling that out. It seems that this implicitly underlies some important disagreements. By contrast, I think this addition is somewhat counterproductive:
and will instead essentially reach a plateau.
The idea of a plateau brings to mind images of sub-linear growth, but all that is required is sub-exponential growth, a much weaker claim. I think this will...
I’m curious to know what you think the difference is. Both problems require greenhouse gas emissions to be halted.
I agree that both mainline and extreme scenarios are helped by reducing greenhouse gas emissions, but there are other things one can do about climate change, and the most effective actions might turn out to be things which are specific to either mainline or extreme risks. To take examples from that link:
Thanks for this. I found it interesting to think about. Here are my main comments.
Mainline and extreme risks
I think it would be better to analyse mainline risks and extreme risks separately.
Regarding increasing marginal returns (IMR), which seems to be the primary contribution of this paper and not obviously addressed by replies to other types of systemic change objections:
Perhaps rather than 'Are IMR commonly found in cause areas?', I would ask 'where are IMR found?' and, for the purposes of testing the critique, 'in which cases are relevant actors not already aware of those IMR?' This is because I expect the prevalence of IMR to vary substantially between areas. (I see that you also call for concrete examples i...
Punting strategies, in contrast, affect future generations primarly via their effect on the people alive in the most influential centuries.
That seems like a sufficiently precise definition. Whether there are any interventions in that category seems like an open question. (Maybe it is a lot more narrow than Will's intention.)
Not appropriated: lost to value drift. (Hence, yes, the historical cases I draw on are the same as in my comment 3 up in this thread.) I'm thinking of this quantity as something like the proportion of resources which will in expectation be dedicated 100 years later to the original mission as envisaged by the founders, annualised.
Got it. Given the inclusion of (bad) value drift in 'appropriated (or otherwise lost)', my previous comment should just be interpreted as providing evidence to counter this claim:
But I shouldn’t think that the chance of my funds being appropriated (or otherwise lost) is as high as 2% per year.
[Recap of my previous comment] It seems that this quote predicts a lower rate than there has ever† been before. Such predictions can be correct! However, a plan for making the prediction come true is needed.
It seems that the plan should be different to what ...
Thanks. I agree that we might endorse some (or many) changes. Hidden away in my first footnote is a link to a pretty broad set of values. To expand: I would be excited to give (and have in the past given) resources to people smarter than me who are outcome-oriented, maximizing, cause-impartial and egalitarian, as defined by Will here, even (or especially) if they plan to use them differently to how I would. Similarly, keeping the value 'do the most good' stable maybe means something like keeping the outcome-oriented, maximizing, cause-impartial a...
I was very surprised to see that 'funds being appropriated (or otherwise lost)' is the main concern with attempting to move resources 100 years into the future. Before seeing this comment, I would have been confident that the primary difficulty is in building an institution which maintains acceptable values† for 100 years.
Some of the very limited data we have on value drift within individual people suggests losses of 11% and 18% per year for two groups over 5 years. I think these numbers are a reasonable estimate for people who have held certain ...
Sorry - 'or otherwise lost' qualifier was meant to be a catch-all for any way of the investment losing its value, including (bad) value-drift.
I think there's a decent case for (some) EAs doing better at avoiding this than e.g. typical foundations:
Thanks! I hadn't seen the Cotton-Barratt piece before.
Extinction risk reduction punts on the question of which future problems are most important to solve, but not how best to tackle the problem of extinction risk specifically. Building capacity for future extinction risk reduction work punts on how best to tackle the problem of extinction risk specifically, but not the question of which future problems are most important to solve. They seem to do more/less punting than one another along different dimensions, so, depending on one's definition of ...
I agree that, among other things, discussion of mechanisms for sending resources to the future would needed to make such a decision. I figured that all these other considerations were deliberately excluded from this post to keep its scope manageable.
However, I do think that one can interpret the post as making claims about a more insightful kind of probability: the odds with which the current century is the one which will have the highest leverage-evaluated-at-the-time (in contrast to an omniscient view / end-of-time evaluation, which is what this thread m...
This was very thought-provoking. I expect I'll come back to it a number of times.
I suspect that how the model works depends a lot on exactly how this definition is interpreted:
a time t is more influential (from a longtermist perspective) than a time t iff you would prefer to give an additional unit of resources,[1] that has to be spent doing direct work (rather than investment), to a longtermist altruist living at t rather than to a longtermist altruist living at t.
In particular, I think you intend direct work to include extinction risk reduction,...
How I see it:
Extinction risk reduction (and other type of "direct work") affects all future generations similarly. If the most influential century is still to come, extinction risk reduction also affects the people alive during that century (by making sure they exist). Thus, extinction risk reduction has a "punting to future generations that live in hingey times" component. However, extinction risk reduction also affects all the unhingey future generations directly, and the effects are not primarily mediated through the people alive in the most influential
...Using a distribution over possible futures seems important. The specific method you propose seems useful for getting a better picture of maxcentury most leveraged. However, what we want in order to make decisions is something more akin to maxleverage of century . The most obvious difference is that scenarios in which the future is short and there is little one can do about it score highly on expected ranking and low on expected value. I am unclear on whether a flat prior makes sense for expectancy, but it seems more reasonable than for proba...
While I agree with you that is not that action relevant, it is what Will is analyzing in the post, and think that William Kiely's suggested prior seems basically reasonable for answering that question. As Will said explicitly in another comment:
Agree that it might well be that even though one has a very low credence in HoH, one should still act in the same way. (e.g. because if one is not at HoH, one is a sim, and your actions don’t have much impact).
I do think that the focus on is t...
I stand corrected. I think those quotes overstate matters a decent amount, but indeed the security of fissile material is a significantly more important barrier to misuse.
Thanks! Here are some places you might start. (People who have done deeper dives into nuclear risk might have more informed views on what resources would be useful.)
I recently started to feel that celebrating Petrov was a bad choice: he just happened to be in the right place in the right time, and as you say, there were many false positives at the time. Petrov's actions were important, but they provide no lessons to those who aspire to reduce x-risk.
A better example might be Carl Sagan, who (if I'm correct) researched nuclear winter and succesfully advocated against nuclear weapons by conveying the risk of nuclear winter. This seemed to have contributed to Gorbachov's conviction to mitigate nuclear war risk. This stor
...Open Phil (then GiveWell Labs) explored climate change pretty early on in their history, including the nearer-term humanitarian effects. Giving What We Can also compared climate change efforts to health interventions. (Each page is a summary page which links to other pages going into more detail.)
I'm very excited to see people doing empirical work on what things we care about are in fact dominated by their extremes. At least after adjusting for survey issues, statements like
The bottom 90% accounts for 30% of incidents
seem to be a substantial improvement on theoretical arguments about properties of distributions. (Personal views only.)
I'm less optimistic about the use of surveys on whether people think tryptamines will/did work:
Huh. The winning response, one of the six early responses, also engages explicitly with the arguments in the main post in its section 1.2 and section 2. This one discussed things mentioned in the post without explicitly referring to the post. This one summarises the long-term-focused arguments in the post and then argues against them.
I worry I'm missing something here. Dismissing these responses as 'cached arguments' seemed stretched already, but the factual claim made to back that decision up, that 'None of these engaged with the pro-p...
I also came to note that the request was for 'the best arguments against psychedelics, not for counter-arguments to your specific arguments in favour'.
However, I also wrote one of the six responses referred to, and I contest the claim that
None of these engaged with the pro-psychedelic arguments I made in the main post
The majority of my response explicitly discusses the weakness of the argumentation in the main post for the asserted effect on the long-term future. To highlight a single sentence which seems to make this clear, I say:
I don't se...
Huh. The winning response, one of the six early responses, also engages explicitly with the arguments in the main post in its section 1.2 and section 2. This one discussed things mentioned in the post without explicitly referring to the post. This one summarises the long-term-focused arguments in the post and then argues against them.
I worry I'm missing something here. Dismissing these responses as 'cached arguments' seemed stretched already, but the factual claim made to back that decision up, that 'None of these engaged with the pro-p...
On the specific questions you're asking about whether empirical data from the Kuwaiti oil field destruction is taken into account: it seems that the answer to each is simply 'yes'. The post says that the data used is adapted from Toon et al. (2007), which projects how much smoke would reach the stratosphere specifically. The paper explicitly considers that event and what the model would predict about them:
Much interest in plume rise was directed at the Kuwati oil fires set by Iraqi forces in 1991. Small (1991) estimated that oil well fires produce energy a...
On your general point about paying attention to political biases, I agree that's worthwhile. A quibble related to that which might matter to you: the Wikipedia article you're quoting seems to attribute the incorrect predictions to TTAPS but I could only trace them to Sagan specifically. I could be missing something due to dead/inaccessible links.
There are a whole bunch of things I love about this work. Among other things:
I have one material issue with the model structure, which I think may reverse your bottom line. The scenario full-scale countervalue attack against Russia has a median smoke estimate of 60Tg and a scenario probability of 0.27 x 0.36 = ~0.1. This means the probability of total smoke exceeding 60Tg has to be >5%, but Total smoke generated by a US-Russia nuclear exchange calculates a probability of only 0.35% for >60Tg smoke.
What seems to be going on is that the model incorporates estimated smoke from each countervalue targeting scenario as {scenario pr...
Agreed. The discussion of the likelihood of countervalue targetting throughout this piece seems very important if countervalue strikes would typically produce considerably more soot than counterforce strikes. In particular, the idea that any countervalue component of a second strike would likely be small seems important and is new to me.
I really hope the post is right that any countervalue targetting is moderately unlikely even in a second strike for the countries with the largest arsenals. That one ‘point blank’ line in the 2010 NPR was certainly surprising to me. On the other hand, I'm not compelled by most of the arguments as applied to second strikes specifically.
This is fascinating, especially with details like different survivability of US and Russian SLBMs. My main takeaway is that counterforce is really not that effective, so it remains hard to see why it would be worth engaging in a first strike. I'd be interested to hear if you ever attempt to quantify the risk that cyber, hypersonic, drone and other technologies (appear to) change this, or if this has been attempted by someone already.
Relatedly:
If improvements in technology allowed either country to reliably locate and destroy those targets, they would...
Quibbles/queries:
The one significant thing I was confused about was why the upper bound survivability for stationary, land-based ICBMs is only 25%. It looks like these estimates are specifically for cases where a rapid second strike (which could theoretically achieve survivability of up to 100%) is not attempted. Do you intend to be taking a position on whether a rapid second strike is likely? It seems like you are using these numbers in some places, e.g. when talking about ‘Countervalue targeting by Russia in the US’ in your third post, when you might be ...
This series (#2, #3) has begun as the most interesting-to-me on the Forum in a long time. Thanks very much. If you have written or do write about how future changes in arsenals may change your conclusions about what scenarios to pay the most attention to, I'd be interested in hearing about it.
In case relevant to others, I found your spreadsheet with raw figures more insightful than the discrete system in the post. To what extent do you think the survey you use for the probabilities of particular nuclear scenarios is a reliable source? (I previously di...
effect from boosting efficacy of current long-termist labor + effect from increasing the amount of long-termist labor
Let's go. Upside 1:
effect from boosting efficacy of current long-termist labor
Adding optimistic numbers to what I already said:
Psychedelic interventions seem promising because they can plausibly increase the number of capable people focused on long-termist work, in addition to plausibly boosting the efficacy of those already involved.
Pointing out that there are two upsides is helpful, but I had just made this claim:
The math for [the bold part] seems really unlikely to work out.
It would be helpful if you could agree with or contest with that claim before we move on to the other upside.
-
Rationality projects: I don't care to arbitrate what counts as EA. I'm going to steer c...
The commonsense meaning of 'I predicted X' is that I used some other information to assess that X was likely. 'I saw the announcement of X before it was published' is not that. I agree that it wasn't literally false. It just gave a false impression. Hence 'pretty clickbaity/misleading'.