Working (6-15 years of experience)

Physician - general internal medicine. Interested in progress, existential risk and epistemology. Did some earning to give, but now have my doubts. Heavier on ideas than execution.

3

27

Refuting longtermism with Fermat's Last Theorem

Making an estimate about something you're unaware of is like guessing the likelihood of the discovery of nuclear energy in 1850.

I can put a number on the likelihood of discovering something totally novel, but applying a number doesn't mean it's meaningful. A psychic could make quantified guesses and tell us about the factors involved in that assessment, but that doesn't make it meaningful.

Refuting longtermism with Fermat's Last Theorem

I'm saying the opposite - you can't rank the difficulty of unsolved problems if you don't know what's required to solve them. That's what yet-to-be-discovered means, you don't know the missing bit, so you can't compare.

Refuting longtermism with Fermat's Last Theorem

It's not that "it happened this one time with Wiles, where he really knew a topic and was also way off in his estimate, and so that's how it goes." It's that the Wiles example shows us that we are always in his shoes when contemplating the yet-to-be-discovered, we are completely in the dark. It's not that he didn't know, it's that he COULDN'T know, and neither could anyone else who hadn't made the discovery.

Refuting longtermism with Fermat's Last Theorem

But such work would undoubtedly produce unanticipated and destabilizing discoveries. You can't grow knowledge in foreseeable ways, with only foreseeable consequences.

Refuting longtermism with Fermat's Last Theorem

I'd take the bet, but the feeling I have that inclines me toward choosing the affirmative says nothing about the actual state of the science/engineering. Even if I research for many hours on the current state of research, this will only affect the feeling I have in my mind. I can assign that feeling a probability, and tell others that the feeling I have is "roughly informed," and I can enroll in Phil Tetlock's forecasting challenge. But all of this learns nothing about the currently unknown discoveries that need to be made in order to bring about cold fusion.

Imagine asking Andrew Wiles the morning of his discovery if he wanted to bet that a solution would be found that afternoon. Given his despair, he might take 100x against. And this subjective sense of things would indeed be well-formed, he could talk to us for hours about why his approach doesn't work. And we'd come away convinced - it's hopeless. But that feeling of hopelessness, unlikelihood, despair - they have nothing to do with the math.

Estimating what remains to be discovered for a breakthrough is like trying to measure a gap but not knowing where to place the other end of the ruler.

Refuting longtermism with Fermat's Last Theorem

He was incentivized to decide whether to quit or to persevere (at the cost of other opportunities.) For accuracy, all he needed was "likely enough to be worth it." And yet, at the moment when it should have been most evident what this likelihood was, he was so far off in his estimate that he almost quit.

Imagine if a good EA stopped him in his moment of despair and encouraged him, with all the tools available, to create the most accurate estimate, I bet he'd still consider quitting. He might even be more convinced that it's hopeless.

AMA: Jason Crawford, The Roots of Progress

Yes, but what I’m getting at is How do we know there’s a limited number of low hanging fruit? Or, as we make progress, don’t previously high fruit come into reach? AND, more progress opens more markets/fields.

It seems to me low hanging fruit is a bad analogy because there’s not way to know the number of undiscovered fruit out there. And perhaps it’s infinite. Or, it INCREASES the more we figure out.

My two cents - stagnation isn’t due to supply of good ideas waiting to be discovered, it’s stifling of free and open exploration by our norms that promote institutionalization of discovery.

AMA: Jason Crawford, The Roots of Progress

How could it be that *ideas are progressively harder to find* AND *we waited so long for the bicycle*? How can we know how many undiscovered bicycles, ie low hanging fruit, are out there?

Seems as progress progresses and the adjacent possible expands, the number of undiscovered bicycles within easy reach expands.

How to promote widespread usage of high quality, reusable masks

I think the idea of effective mask use has withstood sufficient criticism to warrant spreading aggressively, both to the public as well as experts in the field. It may be a mistake, but compared to no mask at all (risk of infection, barriers to reentering society) it is hard to see it being a significant mistake. The potential upside is significant. We may have a relatively cheap and safe countermeasure within reach.

A thanksgiving turkey has an excellent model that predicts the farmer wants him to be safe and happy. But an explanation of thanksgiving traditions tells us a lot more about the risks of slaughter than the number of days the turkey has been fed and protected.

With nuclear war, we have explanations for why nuclear exchange is possible, including as an outcome of a conflict.

Just like with the turkey, we should pay attention to the explanation, not just try to make predictions based on past data.

With all of this, probability terminology is baked into the language and it is hard to speak without incorporating it. With the previous post, it was co-authored, and I wanted to remove that phrase, but concessions were made.