BLUF:
* To determine whether AI is ‘improving exponentially’, ‘hitting the wall’, or any other claim which involves a quantity or magnitude (e.g. ‘This model was a big leap/small increment’). We need a good y-axis: an interval scale of AI capability which means +1 unit always represents the same degree of ‘how much better’, in the same way +1 degree Celsius is always the same amount of ‘how much hotter’.
* Yet there is no good y-axis for AI capability. All our...
Public service announcement
1. Applications are now open for our first ever round of the Charity Entrepreneurship Incubation Program dedicated exclusively to animal welfare. Learn more about what’s different this round here and apply...
TL;DR
* The Long-Term Future Fund is closing down, and EA Funds is launching the Transformative AI Fund with a new full-time team.
* The fund's primary focus is technical AI safety and AI governance (including post-AGI governance), as well as supporting fields such as field-building and forecasting. We'll also consider non-GCR implications of transformative AI such as flourishing futures and digital...
For me, perhaps the biggest takeaway from Aschenbrenner's manifesto is that even if we solve alignment, we still have an incredibly thorny coordination problem between the US and China, in which each is massively incentivized to race ahead and develop military power using superintelligence, putting them both and the rest of the world at immense risk. And I wonder if, after seeing this in advance, we can sit down and solve this coordination problem in ways that lead to a better outcome with a higher chance than the "race ahead" strategy and don't risk encountering a short period of incredibly volatile geopolitical instability in which both nations develop and possibly use never-seen-before weapons of mass destruction.
Edit: although I can see how attempts at intervening in any way and raising the salience of the issue risk making the situation worse.