David Thorstad

Assistant Professor of Philosophy @ Vanderbilt University
1267 karmaJoined Apr 2021www.dthorstad.com


Glad it helped! All credit to Nicholas who wrote 99% of it. If you have a minute, I uploaded a talk version of the paper last week. Would love to hear what you think, especially re accessibility: 

You folks impress me! But seriously, that's a big ask.

To be fair, this could trigger lawsuits. I hope someone is reflecting on FTX, but I wouldn't expect anyone to be keen on discussing their own involvement with FTX publicly and in great detail.

Here's a gentle introduction to the kinds of worries people have (https://spectrum.ieee.org/power-problems-might-drive-chip-specialization). Of the cited references "the chips are down for moore's law" is probably best on this issue, but a little longer/harder. There's plenty of literature on problems with heat dissipation if you search the academic literature. I can dig up references on energy if you want, but with Sam Altman saying we need a fundamental energy revolution even to get to AGI, is there really much controversy over the idea that we'll need a lot of energy to get to superintelligence? 

Ah - that comes from the discontinuity claim. If you have accelerating growth that isn't sustained for very long, you get something like population growth from 1800-2000, where the end result is impressive but hardly a discontinuity comparable to crossing the event horizon of a black hole. 

(The only way to go around the assumption of sustained growth would be to post one or a few discontinuous leaps towards superintelligence. But that's harder to defend, and it abandons what was classically taken to ground the singularity hypothesis, namely the appeal to recursive self-improvement). 

Here's the talk version for anyone who finds it easier to listen to videos: 

Thanks Peter!

I wonder if you'd be willing to be a bit more vocal about this. For example, the second most upvoted comment (27 karma right now) takes me to task for saying that "most experts are deeply skeptical of Ord’s claim"  (1/30 existential biorisk in the next 100 years).

I take that to be uncontroversial. Would you be willing to say so?

Thanks Caleb! I give reasons for skepticism about high levels of existential biorisk in Parts 9-11 of this series.

Load more