All of zdgroff's Comments + Replies

I think I might add this to my DIY, atheist, animal-rights Haggadah.

[Edited to add the second sentence of the paragraph beginning, "Putting these together."]

The primary result doesn't speak to this, but secondary results can shed some light on it. Overall, I'd guess persistence is a touch less for policies with much more support, but note that the effect of proposing a policy on later policy is likely much larger than the effect of passing a policy conditional on its having been proposed.

The first thing to note is that there are really two questions here we might want to ask:

1. What is the effect of passing a policy change, c

Yep, at the risk of omitting others, Lukas Freund as well.

Yes, it's a good point that benefits and length of the period are not independent, and I agree with the footnote too.

I would note that the factors I mentioned there don't seem like they should change things that much for most issues. I could see using 50-100 years rather than, e.g., 150 years as my results would seem to suggest, but I do think 5-10 years is an order of magnitude off.

Easy Q to answer so doesn't take much time! In economics, the norm is not to publish your job market paper until after the market for various reasons. (That way, you don't take time away from improving the paper, and the department that hires you gets credit.)

We will see before long how it publishes!

1. I look at some things you might find relevant here. I try to measure the scale of the impact of a referendum. I do this two ways. I have just a subjective judgment on a five-point scale, and then I also look at predictions of the referendum's fiscal impact from the secretary of state. Neither one is predictive. I also look at how many people would be directly affected by a referendum and how much news coverage there was before the election cycle. These predict less persistence.
2. This is something I plan to do more, but they can't vary that much because when

I do look at predictors a bit—though note that it's not about what makes it harder to repeal but rather about what makes a policy change/choice influential decades later.

The main takeaway is there aren't many predictors—the effect is remarkably uniform. I can't look at things around the structure of the law (e.g., integration in a larger bill), but I'd be surprised if something like complexity of language or cross-party support made a difference in what I'm looking at.

Yeah, Jack, I think you're capturing my thinking here (which is an informal point for this audience rather than something formal in the paper). I look at measures of how much people were interested in a policy well before the referendum or how much we should expect them to be interested after the referendum. It looks like both of these predict less persistence. So the thought is that things that generally are less salient when not on the ballot are more persistent.

See my reply to Neil Dullaghan—I think that gives somewhat of a sense here. Some other things:

• I don't have a ton of observations on any one specific policy, so I can't say much about whether some special policy area (e.g., pollution regulation) exhibits a different pattern.
• I look at whether this policy, or a version of it, is in place. This should capture anything that would be a direct and obvious substitute, but there might be looser substitutes that end up passing if you fail to pass an initial policy. The evidence I do have on this suggests it's small,

I didn't write down a prior. I think if I had, it would have been less persistence. I think I would have guessed five years was an underestimate. (I think probably many people making that assumption would also have guessed it was an underestimate but were airing on the side of conservatism.)

Yes, basically (if I understand correctly). If you think a policy has impact X for each year it's in place, and you don't discount, then the impact of causing it to pass rather than fail is something on the order of 100 * X. The impact of funding a campaign to pass it is bigger, though, because you presumably don't want to count the possibility that you fund it later as part of the counterfactual (see my note above about Appendix Figure D20).

 Some things to keep in mind:

1. Impacts might change over time (e.g., a policy stops mattering in 50 years even if

A few things:

• I do find these patterns when I look at a few different types of policies (referendums, legislation, state vs. Congress, U.S. vs. international), so there's some reason to think it's not just state referendums. 
• There's a paper on the repeals of executive orders that finds an even lower rate of repeals there, but that doesn't tell us the counterfactual (i.e., would someone else have done this if the president in question did not).
• There's suggestive evidence that when policies are more negotiable, there's less persistence. In my narrative/c

I think there are probably ways to tackle that but don't have anything shovel-ready. I'd want to look at the general evidence on campaign spending and what methods have been used there, then see if any of those would apply (with some adaptations) to this case.

Thanks a lot! And good luck on the job market to you—let's connect when we're through with this (or if we have time before then).

I'm very glad to see you working and thinking about this—it seems pretty neglected within the EA community. (I'm aware of and agree with the thought that speeding up space settlement is not a priority, but making sure it goes well if it happens does seem important.)

Oh, that's a good idea. I had thought of something quite different and broader, but this also seems like a promising approach.

Yeah, I think that would reduce the longevity in expectation, maybe by something like 2x. My research includes things that could hypothetically fall under congressional authority and occasionally do. (Anything could fall under congressional authority, though some might require a constitutional amendment.) So I don't think this is dramatically out of sample, but I do think it's worth keeping in mind.

The former, though I don't have estimates of the counterfactual timeline of corporate campaigns. (I'd like to find a way to do that and have toyed with it a bit but currently don't have one.)

I believe 4 years is very conservative. I'm working on a paper due November that should basically answer the question in part 1, but suffice it to say I think the ballot measures should look many times more cost-effective than corporate campaigns.

From what I can tell, the climate change one seems like the one with the most support in the literature. I'm not sure how much the consensus in favor of the human cause of megafauna extinctions (which I buy) generalizes to the extinction of other species in the Homo genus. Most of the Homo extinctions happened much earlier than the megafauna ones. But it could be—I have not given much thought to whether this consensus generalizes.

The other thing is that "extinction" sometimes happened in the sense that the species interbred with the larger population of Homo sapiens, and I would not count that as the relevant sort of extinction here.

Yeah, this is an interesting one. I'd basically agree with what you say here. I looked into it and came away thinking (a) it's very unclear what the actual base rate is, but (b) it seems like it probably roughly resembles the general species one I have here. Given (b), I bumped up how much weight I put on the species reference class, but I did not include the human subspecies as a reference class here given (a).

From my exploration, it looked like there had been loose claims about many of them going extinct because of Homo sapiens, but it seemed like this w... (read more)

Very strong +1 to all this. I honestly think it's the most neglected area relative to its importance right now. It seems plausible that the vast majority of future beings will be digital, so it would be surprising if longtermism does not imply much more attention to the issue.

I take 5%-60% as an estimate of how much of human civilization's future value will depend on what AI systems do, but it does not necessarily exclude human autonomy. If humans determine what AI systems do with the resources they acquire and the actions they take, then AI could be extremely important, and humans would still retain autonomy.

I don't think this really left me more or less concerned about losing autonomy over resources. It does feel like this exercise made it starker that there's a large chance of AI reshaping the world beyond human extinction. ... (read more)

I guess I would think that if one wants to argue for democracy as an intrinsic good, that would get you global democracy (and global control of EA funds), and it's practical and instrumental considerations (which, anyway, are all the considerations in my view) that bite against it.

Yes, that seems right.

It seems like the critics would claim that EA is, if not coercing or subjugating, at least substantially influencing something like the world population in a way that meets the criteria for democratisation. This seems to be the claim in arguments about billionaire philanthropy, for example. I'm not defending or vouching for that claim, but I think whether we are in a sufficiently different situation may be contentious.

Well I think MIC relies on some sort of discontinuity this century, and when we start getting into the range of precedented growth rates, the discontinuity looks less likely.

But we might not be disagreeing much here. It seems like a plausibly important update, but I'm not sure how large.

This is a valuable point, but I do think that giving real weight to a world where we have neither extinction nor 30% growth would still be an update to important views about superhuman AI. It seems like evidence against the Most Important Century thesis, for example.

It might be challenging to borrow (though I'm not sure), but there seem to be plenty of sophisticated entities that should be selling off their bonds and aren't. The top-level comment does cut into the gains from shorting (as the OP concedes), but I think it's right that there are borrowing-esque things to do.

I'm trying to make sure I understand: Is this (a more colorful version) of the same point as the OP makes at the end of "Bet on real rates rising"?

The other risk that could motivate not making this bet is the risk that the market – for some unspecified reason – never has a chance to correct, because (1) transformative AI ends up unaligned and (2) humanity’s conversion into paperclips occurs overnight. This would prevent the market from ever “waking up”.

However, to be clear, expecting this specific scenario requires both: 

1. Buying into spe

It doesn't seem all that relevant to me whether traders have a probability like that in their heads. Whether they have a low probability or are not thinking about it, they're approximately leaving money on the table in a short-timelines world, which should be surprising. People have a large incentive to hunt for important probabilities they're ignoring.

Of course, there are examples (cf. behavioral economics) of systemic biases in markets. But even within behavioral economics, it's fairly commonly known that it's hard to find ongoing, large-scale biases in financial markets.

Do you have a sense of whether the case is any stronger for specifically using cortical and pallial neurons? That's the approach Romain Espinosa takes in this paper, which is among the best work in economics on animal welfare.

Yes, and thank you for the detailed private proposal you sent the research team. I didn't see it but heard about it, and it seems like it was a huge help and just a massive amount of volunteer labor. I know they really appreciated it.

I'm an ACE board member, so full disclosure on that, though what I say here is in my personal capacity.

I'm very glad about a number of improvements to the eval process that are not obvious from this post. In particular, there are now numeric cost-effectiveness ratings that I found clarifying, overall explanations for each recommendation, and clearer delineation of the roles the "programs" and "cost-effectiveness" sections play in the reviews. I expect these changes to make recommendations more scope sensitive. This leaves me grateful for and confident in the new review framework.

As I noted on the nuclear post, I believe this is based on a (loosely speaking) person-affecting view (mentioned in Joel and Ben's back-and-forth below). That seems likely to me to bias the cost-effectiveness downward.

Like Fin, I'm very surprised by how well this performs given takes in other places (e.g. The Precipice) on how asteroid prevention compares to other x-risk work.

Worth flagging that I believe this is based on a (loosely speaking) person-affecting view (mentioned in Joel and Ben's back-and-forth below). That seems to me to bias the cost-effectiveness of anything that poses a sizable extinction risk dramatically downward.

At the same time, I find both the empirical work and the inside-view thinking here very impressive for a week's work, and it seems like even those without a person-affecting view can learn a lot from this.

Thanks for writing this. I think about these sorts of things a lot. Given the title, do you know of examples of movements that did not start academic disciplines and appear to have suffered as a result?

The Global Priorities Institute and clusters of work around that do work in economics, including welfare economics. I'd also be curious to hear what you think they should do differently.

I'm toying with a project to gather reference classes for AGI-induced extinction and AGI takeover. If someone would like to collaborate, please get in touch.

(I'm aware of and giving thought to reference class tennis concerns but still think something like this is neglected.)

I don't think it's right that the broad project of alignment would look the same with and without considering religion. I'm curious what your reasoning is here and if I'm mistaken.

One way of reading this comment is that it's a semantic disagreement about what alignment means. The OP seems to be talking about the  problem of getting an AI to do the right thing, writ large, which may encompass a broader set of topics than alignment research as you define it.

Two other ways of reading it are that (a) solving the problem the OP is addressing (getting an AI... (read more)

One thing that's  sad and perhaps not obvious to people is that, as I understand it, Nathan Robinson was initially sympathetic to EA (and this played a role in at-times vocal advocacy for animals). I don't know that there's much to be done about this. I think the course of events was perhaps inevitable, but that's relevant context for other Forum readers who see this.

And worth noting that Ben Franklin was involved in the constitution, so at least some of his longtermist time seems to have been well spent.

I don't have a strong view on the original setup, but I can clarify what the argument is. For the first point, that we maximize $E\left(C_E\right)+S\left(C_S\right)$. The idea is that we want to maximize the likelihood that the organism chooses the action that leads to enjoyment (the one being selected for). That probability is a function of how much better it is to choose that action than the alternative. So if you get E from choosing that action and lose S from choosing the alternative, the benefit from choosing that action is E - (-S) = E + S. However, you only pay to produ... (read more)

Like others, I really appreciate these thoughts, and it resonates with me quite a lot. At this point, I think the biggest potential failure mode for EA is too much drift in this direction. I think the "EA needs megaprojects" thing has generated a view that the more we spend, the better, which we need to temper. Given all the resources, there's a good chance EA is around for a while and quite large and powerful. We need to make sure we put these tools to good use and retain the right values.

EA spending is often perceived as wasteful and self-serving

It's int... (read more)

Yes, that's an accurate characterization of my suggestion. Re: digital sentience, intuitively something in the 80-90% range?

Yes, all those first points make sense. I did want to just point to where I see the most likely cruxes.

Re: neuron count, the idea would be to use various transformations of neuron counts, or of a particular type of neuron.  I think it's a judgment call whether to leave it to the readers to judge; I would prefer giving what one thinks is the most plausible benchmark way of counting and then giving the tools to adjust from there, but your approach is sensible too.

Thanks for writing this post. I have similar concerns and am glad to see this composed. I particularly like the note about the initial design of space colonies. A couple things:

• My sense is that the dominance of digital minds (which you mention as a possible issue) is actually the main reason many longtermists think factory farming is likely to be small relative to the size of the future. You're right to note that this means future human welfare is also relatively unimportant, and my sense is that most would admit that. Humanity is instrumentally important,

Research institute focused on civilizational lock-in

Values and Reflective Processes, Economic Growth, Space Governance, Effective Altruism

One source of long-term risks and potential levers to positively shape the future is the possibility that certain values or social structures get locked in, such as via global totalitarianism, self-replicating colonies, or widespread dominance of a single set of values. Though organizations exist dedicated to work on risks of human extinction, we would like to see an academic or independent institute focused on other eve... (read more)

Consulting on best practices around info hazards

Epistemic Institutions, Effective Altruism, Research That Can Help Us Improve

Information about ways to influence the long-term future can in some cases give rise to information hazards, where true information can cause harm. Typical examples concern research into existential risks, such as around potential powerful weapons or algorithms prone to misuse. Other risks exist, however, and may also be especially important for longtermists. For example, better understanding of ways social structures and values can ... (read more)