So I agree that humanity might just choose not to reach the stars. It seems unlikely to me that nobody (or nobody with sufficient resources) would want to do this post-AGI, but it's possible humanity as a whole prevents other people from expanding (eg worries about building independent power centers that might harm the safety of Earth, or spoilt negotiations, or more idiosyncratic factors). 

This is not the most likely existential risk imo, but certainly one to be aware of.

That said, the 1960s-70s moon landing was a large net resource loss. Costed ~ half a percentage point of GDP (!) annually for multiple years and didn't get anything in return other than a few innovations and one-upping the Soviets. Seems like a pretty different story!

There are two common models of space colonization people sometimes allude to, neither of which I think is particularly likely. 

Model 1 (“normal colonization”) is that space colonization will look something like Earth colonization, e.g. the way the first humans to expand to the Polynesian islands. So your boat (rover/ship/probe) hops to one island (planet), you build up a civilization, and then you send your probes onwards to the next couple of nearby planets, maybe saving up a bunch of resources if you've colonized nearby star systems (eg your galaxy) and need to send a bigger ship to more distant stars. So it looks like either orderly civilizational growth or an evolutionary process.

I don't think this model is really likely because von Neumann probes will be really cheap relative to the carrying capacity of star systems. So I don't think the intuitive "slow waves of colonization" model makes a lot of sense on a galactic scale. 

I don’t think my view here is particularly controversial. My impression is that while the first model is common in science fiction, nobody in the futurism/x-risk/etc field really believes it.

Model 2 (“mad dash”) is that you race ahead as soon as you reach relativistic speeds. So as soon as your science and industry has advanced enough for your probes to reach appreciable fractions of c, you start blasting out von Neumann probes to the far reaches of the affectable universe.

I think this model is more plausible, but still unlikely. A small temporal delay is worth it to develop more advanced spacefaring technology.

My guess is that even if all you care about is maximizing space colonization, it still makes sense to delay some time before you launch your first "serious" interstellar space probe, rather than do it as soon as possible[1].

Whether you can reach the furthest galaxies is determined by something like[2]:

total time to reach a galaxy = delay + distance/speed 

So you want to delay and keep researching until the marginal speed gain from additional R&D time is lower than the marginal cost of the delay. 

I don't have a sense of how long this is, but intuitively it feels more like decades and centuries, maybe even slightly longer, than months or years. The theoretically reachable universe is 16-18 billion years away, so a 100 years delay is worth it if you can just increase the speed by 1/100 millionth of c [4].

For energy/resource reasons you might want to expand to nearby star systems first to send the fastest possible probes but note again that delay before sending your first probe is always at worst a constant amount of time. There's the possible exception of being able to accelerate R&D in other star systems, e.g. because you need multiple star systems of compute in order to do the R&D well. But this is trickier than it looks! The lightspeed communication barrier means sending information is slow, so you're really giving up a lot in terms of latency to use up more compute. A caveat here is that you might want your supercomputer to be bigger than the home system's resources. So maybe you want to capture a nearby star system to turn that into your core R&D department. Though that takes a while to build out, too.

Here are a few models of space colonization that I think are more likely:

1. Model 3 (Deliberate + build in the home system, then spam): Research deeply until reaching ~very deep technological levels [3] then suddenly spam a ton of probes everywhere at very high fractions of c. I think this is the implicit model in Sandberg's Space Races' paper.
   1. Assuming technological maturity, Sandberg had two different models for exploration:
      1. one that races earlier to grab nearby interstellar resources
      2. and one that waits longer and saves up to send faster probes (where the constraint on probe speed is primarily energy, not knowledge)
   2. In the technological maturity case, Sandberg concludes that racing earlier is better.
   3. Note however that Sandberg’s model presumes technological maturity, so in a sense his analysis starts a bit after where mine ends.
   4. I think his model is roughly correct in worlds where reaching technological maturity is relatively quick. This assumption seems plausible enough to me, but not guaranteed.
2. Model 4 (Colonize in nearby systems, deliberate in home systems, then spam): Colonize nearby systems first, while continuously researching at home. Turn nearby systems into Dyson swarms etc so they have massive (and flexible!) industrial capacity, while simultaneously researching in the home system what are the best ways to send fast ships.
   1. In this model you’re first colonizing systems within say 50-500 years (not light-years, years) of your home system
   2. And then once your home systems figure out the optimal way to send probes, they tell the colonies what to do next (at the speed of light), and the colonies (plus maybe the home system too at that point) spam probes at near the speed of light.
3. Model 5 (Waves: Deliberate, spam, deliberate, spam, deliberate…): The home system spends enough time thinking/researching/building until they’ve reached a plausible plateau. They send probes out for a while, roughly until distances are such that sending more probes from home will be slower to reach distant shores than it’d take probes from colonies to reach them. Then they switch back to deliberation mode and keep deliberating until/if they invent a new mode of transportation that’s faster than the head start colonies have, then start sending probes again to distant stars, intending to overtake the front wave of probes from the colonies at sufficiently distant stars.
   1. The colonies repeat the same strategy as home, first building up and sending a bunch of probes out, and then switching to “research” mode.
   2. This keeps going until we are very confident you can’t send faster ships, and then the expanding core switches from deliberation and spread into spending their energy on more terminal moral goods.
4. Model 6 (Deliberate, spam, deliberate and signal): Like the previous model, but after the first wave of probes, the home system  (and other systems in the expanding core) no longer sends more probes. Instead they switch to spending all their time on research and deliberations. If/when they discover a faster mode of transportation, they signal (at exactly c) the new strategy to distant systems at the frontier, to switch their expansion technology.
   1. Compared to Model 5, this strategy has the advantage of the speed of light always being faster than whatever mode of transportation you have for physical ships. So if your colonies are “on the way” to distant stars, it’s always faster to tell the colonies what to do than to send your own probes.
   2. This strategy might seem strictly superior to Model 5. But this isn’t necessarily the case! For example, galaxies tend to be ~2D, whereas the affectable universe is ~3D. So for different galaxies not on the same plane, it might often be more efficient to send probes directly than to wait for light-speed communication to hit a colony on the current frontier, and then send a probe from there.
      1. The galactic disk is ~1000 light-years thick but intergalactic targets can be millions of light-years away in any direction, so for most target galaxies there's no frontier colony meaningfully closer than the home system, or at least the “core”.
5. Model 7:??? Excited to hear other models I haven’t thought of!

I’m neither an astrophysicist nor in any other way a “real” space expert and I’ve spent less than a day thinking about the relevant dynamics, so let me know if you think I’m wrong or you have additional thoughts! Very happy to be corrected. :) 

[1] Modulo other reasons for going faster, like worries about single-system x-risk, stagnation, meme wars etc. There are also other reasons to go slower, for example worries about interstellar x-risks/ vulnerable universe, wanting more value certainty and fear of value drift, being scared of aliens, etc.

[2] + relativistic effects and other cosmological effects that I don't understand. I never studied relativity but I'd be surprised if it changes the OOM calculus.

[3] where we predict additional research time yields diminishing returns relative to acting on current knowledge

[4] See also earlier work by Kennedy. Kennedy 2006's 'wait calculation' formalizes a version of this tradeoff for nearby stars and gets centuries-scale optimal delays, though his model doesn't consider the intergalactic case and has additional assumptions about transportation speeds that I’m unsure about.

Know Your Meme says it started off as video game jargon; my impression is that it's pretty common online outside of that.

"The only reason we're still talking to these people is we need them and we need them now. The problem for these guys is they are that good," a Defense official told Axios ahead of the meeting.

I'm confused. Why would you ever say this before a negotiation :O

How much of a post are you comfortable for AI to write?

One day the AIs can be much better at writing but that day is not today

This seems like a pretty unlikely fallacy, but I agree it's theoretically possible (and ocassionally happens in practice).

The difference between 0 and 1 is significant! And it's very valuable to figure out when the transition point happens, if you can.

PSA: regression to the mean/mean reversion is a statistical artifact, not a causal mechanism.

So mean regression says that children of tall parents are likely to be shorter than their parents, but it also says parents of tall children are likely to be shorter than their children.

Put in a different way, mean regression goes in both directions. 

This is well-understood enough here in principle, but imo enough people get this wrong in practice that the PSA is worthwhile nonetheless.

I think something a lot of people miss about the “short-term chartist position” (these trends have continued until time t, so I should expect it to continue to time t+1) for an exponential that’s actually a sigmoid is that if you keep holding it, you’ll eventually be wrong exactly once.

Whereas if someone is “short-term chartist hater” (these trends always break, so I predict it’s going to break at time t+1) for an exponential that’s actually a sigmoid is that if you keep holding it, you’ll eventually be correct exactly once.

Now of course most chartists (myself included) want to be able to make stronger claims than just t+1, and people in general would love to know more about the world than just these trends. And if you're really good at analysis and wise and careful and lucky you might be able to time the kink in the sigmoid and successfully be wrong 0 times, which is for sure a huge improvement over being wrong once! But this is very hard.

And people who ignore trends as a baseline are missing an important piece of information, and people who completely reject these trends are essentially insane.

Also seems a bit misleading to count something like "one afternoon in Vietnam" or "first day at a new job" as a single data point when it's hundreds of them bundled together?

From a information-theoretic perspective, people almost never refer to a single data point as strictly as just one bit, so whether you are counting only one float in a database or a whole row in a structured database, or also a whole conversation, we're sort of negotiating price. 

I think the "alien seeing a car" makes the case somewhat clearer. If you already have a deep model of cars (or even a shallow one), seeing another instance of a Ford Focus tells you relatively little, but an alien coming across one will get many bits about it, perhaps more than a human spending an afternoon in Vietnam.

EDIT: I noticed that in my examples I primed Claude a little, and when unprimed Claude does not reliably (or usually) get to the answer. However Claude 4.xs are still noticeable in how little handholding they need for this class of conceptual errors, Geminis often takes like 5 hints where Claude usually gets it with one. And my impression was that Claude 3.xs were kinda hopeless (they often don't get it even with short explanations by me, and when they do, I'm not confident they actually got it vs just wanted to agree).