Exploring different research directions to find out where in the x-risk research ecosystem I best fit in. Part of the 2018-2020 cohort in FHI's Research Scholars Programme. Previously Executive Director of the Foundational Research Institute (now Center on Long-Term Risk), a project by the Effective Altruism Foundation (but I don't endorse that organization's 'suffering-focused' view on ethics).


Does Economic History Point Toward a Singularity?

Yes, sorry, I think I was too quick to make a comment and should have paid more attention to the context. I think the claims in my comment are correct, but as you say it's not clear what exactly they're responding to, and in particular I agree it's not relevant to Asya's point on whether to use 'explosive'.

RyanCarey's Shortform

Yes, good point! My idle speculations have also made me wonder about Indonesia at least once.

Giving and receiving feedback

Agree this is a bad property of Google docs. I wonder how much value we're losing because of this ...

EA wants to be the equivalent of the Scientific Revolution for doing good, but instead of a Republic of Letters we have a Cacophony of Comment Threads. ;)

Some extremely rough research on giving and happiness
I'm just sharing because it seems like a good norm to default towards sharing things like this, and maybe it helps someone else to do a better version.

Strongly agree that this would be a good norm. Thanks for leading by example!

Does Economic History Point Toward a Singularity?

Thanks for poking at this, it would be quite interesting to me if the "constant exponential growth" story was wrong. Which graphs in Farmer & Lafond (2016) are you referring to? To me, the graph with a summary of all trends only seems to have very few that at first glance look a bit like s-curves. But I agree one would need to go beyond eyeballing to know for sure.

I agree with your other points. My best guess is that input prices and other exogenous factors aren't that important for some of the trends, e.g. Moore's Law or agricultural productivity. And I think some of the manufacturing trends in e.g. Arrow (1971) are in terms of output quantity per hour of work rather than prices, and so also seem less dependent on exogenous factors. But I'm more uncertain about this, and agree that in principle dependence on exogenous factors complicates the interpretation.

Does Economic History Point Toward a Singularity?
it seems like most industries in the modern world are characterized by relatively continuous productivity improvements over periods of decades or centuries

This agrees with my impression. Just in case someone is looking for references for this, see e.g.:

  • Nagy et al. (2013) - several of the trends they look at, e.g. prices for certain chemical substances, show exponential growth for more than 30 years
  • Farmer & Lafond (2016) - similar to the previous paper, though fewer trends with data from more than 20 years
  • Bloom et al. (2020) - reviews trends in research productivity, most of which go back to 1975 and some to 1900
  • Some early examples from manufacturing (though not covering multiple decades) are reviewed in a famous paper by Arrow (1971), who proposed 'learning by doing' as a mechanism.
Does Economic History Point Toward a Singularity?

I agree this is puzzling, and I'd love to see more discussion of this.

However, it seems to be that at least in principle there could be a pretty boring explanation: The HGH is correct about the fundamental trend, and the literature on the Industrial Revolution has correctly identified (and maybe explained) a major instance of noise.

Note also that the phenomenon that social behavior that is individually contingent is nevertheless governed by simple macro-laws with few parameters is relatively ubiquitous. E.g. the exact timing of all major innovations since the Industrial Revolution (electricity, chemical engineering, computers, ...) seems fairly contingent, and yet overall the growth rate is remarkably close to constant. Similarly for the rest of Kaldor's facts.

Does Economic History Point Toward a Singularity?

My impression is that everyone agrees that the Industrial Revolution led to an increase in the growth rate, but that the Hyperbolic Growth Hypothesis (HGH) disagrees with the Series Of Exponentials Hypothesis on whether that increase in the growth rate was trend-breaking.

Put differently, the HGH says that as far as the growth rate is concerned, the Industrial Revolution wasn't special - or if it was, then it must attribute this to noise. According to the HGH, the growth rate has been increasing all the time according to the same hyperbolic function, and the industrial revolution was just a part of this trend. I.e. only one "growth mode" for all of history, rather than the industrial revolution ushering in a new growth mode.

By contrast, on the Series Of Exponential view, the Industrial Revolution did break the previous trend - we had exponential growth both before and after, but with different doubling times.

Ben Garfinkel's Shortform
I think this depends a bit what class of safety issues we're thinking about. [...] Many other technological 'accident risks' are less social, although never entirely non-social (e.g. even in the case of bridge safety, you still need to trust some organization to do maintenance/testing properly.)

I'm not sure I agree with this. While they haven't been selected to be representative, the sense I got from the accident case studies I've read (e.g. Chernobyl, nuclear weapons accidents, and various cases from the books Flirting With Disaster and Warnings) is that the social component was quite substantial. It seems to me that usually either better engineering (though sometimes this wasn't possible) or better social management of dealing with engineering limitations (usually possible) could have avoided these accidents. It makes a lot of sense to me that some people prefer to talk of "sociotechnical systems".

Ben Garfinkel's Shortform
But you do need the right balance of conditions to hold: individual units of the technology need to offer their users large enough benefits and small enough personal safety risks, need to create large enough external safety risks, and need to have safety levels that increase slowly enough over time.
Weapons of mass destruction are sort of special in this regard. [...]
[...] I think existential safety risks become a much harder sell, though, if we're primarily imagining non-superweapon applications and distributed/gradual/what-failure-looks-like-style scenarios.

Yes, my guess is we broadly agree about all of this.

I also think it's worth noting that, on an annual basis, even nukes don't have a super high chance of producing global catastrophes through accidental use; if you have a high enough discount rate, and you buy the theory that they substantially reduce the risk of great power war, then it's even possible (maybe not likely) that their existence is currently positive EV by non-longtermist lights.

This also sounds right to me. FWIW, it's not even obvious to me if nukes are negative-EV by longtermist lights. Since nuclear winter seems unlikely to cause immediate extinction this depends on messy questions such as how the EV of trajectory changes from conventional great power war compares to the EV of trajectory changes from nuclear winter scenarios.

Load More