Wow, this is exciting. I agree this is one of the most important things we should be working on right now.

I very much agree that we need less deference and more people thinking for themselves, especially on cause prioritization. I think this is especially important for people who have high talent/skill in this direction, as I think it can be quite hard to do well.

It’s a huge problem that the current system is not great at valuing and incentivizing this type of work, as I think this causes a lot of the potentially highly competent cause prioritization people to go in other directions. I’ve been a huge advocate for this for a long time.

I think it is somewhat hard to systematically address, but I’m really glad you are pointing this out and inviting collaboration on your work, I do think concentration of power is extremely neglected and one of the things that most determines how well the future will go (and not just in terms of extinction risk but upside/opportunity cost risks as well.)

Going to send you a DM now!

Hey Trevor, it’s been a while, I just read Kuhan’s quick take which referred to this quick take, great to see you’re still active!

This is very interesting, I’ve been evaluating a cause area I think is very important and potentially urgent—something like the broader class of interventions of which “the long reflection” and “coherent extrapolated volition” are examples, essentially how do we make sure the future is as good as possible conditional on aligned advanced AI.

Anyways, I found it much easier to combine tractability and neglectedness into what I called “marginal tractability,” meaning how easy is it to increase success of a given cause area by, say, 1% at the current margin. 

I feel like trying to abstractly estimate tractability independent of neglectedness was very awkward, and not scalable; i.e. tractability can change quite unpredictably over time, so it isn’t really a constant factor, but something you need to keep reevaluating as conditions change over time. 

Asking the tractability question “If we doubled the resources dedicated to solving this problem, what fraction of the problem would we expect to solve?” isn’t a bad trick, but in a cause area that is extremely neglected this is really hard to do because there are so few existing interventions, especially measurable ones. In this case investigating some of the best potential interventions is really helpful.

I think you’re right that the same applies when investigating specific interventions. Neglectedness is still a factor, but it’s not separable from tractability; marginal tractability is what matters, and that’s easiest to investigate by actually looking at the interventions to see how effective they are at the current margin.

I feel like there’s a huge amount of nuance here, and some of the above comments were good critiques…

But for now gotta continue on the research. The investigation is at about 30,000 words, need to finish, lightly edit, and write some shorter explainer versions, would love to get your feedback when it’s ready!
 

You forgot to mention that Anthropic’s name literally means “as seen from a narrowly human point of view”, far cry from moral circle expansion or doing the most good possible

Thanks Tyler! I think this is spot on. I am nearing the end of writing a very long report on this type of work so I don’t have time at the moment to write a more detailed reply (and what I’m writing is attempting to answer these questions). One thing that really caught my eye was when you mentioned:

Populating and refining a list of answers to this last question has been a lot of the key work of the field over the past few years.

I am deeply interested in this field, but not actually sure what is meant by “the field.”   Could you point me to what search terms to use and perhaps the primary authors or research organizations who have published work on this type of thing?”

@William_MacAskill, what are the main characteristics we should aim for “Viatopia” to have?

Will MacAskill stated in a recent 80,000 hours podcast that he believes marginal work on trajectory change toward a best possible future rather than a mediocre future seems likely significantly more valuable than marginal work on extinction risk.

Could you explain what the key crucial considerations are for this claim to be true, and a basic argument for why think each of the crucial considerations resolves in favor of this claim?

Would also love to hear if others have any other crucial considerations they think weigh in one direction or the other.

Great, thank you!

Hi, hate to bother you again, just wondering where things are at with this contest?

Yes… So basically what you’re saying is this argument goes through if you make the summation of all bubble universes at any individual time step, but longtermist arguments would go through if you take a view from outside the metaverse and make the summation across all points of time in all bubble universes simultaneously?

I guess my main issue is that I’m having trouble philosophically or physically stomaching this, it seems to touch on a very difficult ontological/metaphysical/epistemological question of whether or not it is coherent to do the summation of all points in space-time across infinite time as though all of the infinite future already “preexists” in some sense. On the other hand, it could be the case that taking such an “outside view” of infinite space-time as though calculations could be “all at once” may not be an acceptable operation to perform, as such a calculation could not in reality ever actually be made by any observer, or at least could not be made at any given time

I have a very strong intuition that infinity itself is incoherent and unreal and therefore something like eternal inflation is not actually likely to be correct or may be physically possible. However, I am certainly not an expert in this and my feelings about the topic not necessarily correct; yet my sense is these sorts of questions are not fully worked out.

Part of what makes this challenging for me is that the numbers are so much ridiculously bigger than the numbers in longtermist calculations, that it would seem that even a very, very small chance that it might be correct would make me think it should get somewhat deeper consideration, at least have some specialists who work on these kinds of topics weigh in on how likely it seems something like this could be correct.