Comments
More from the author
Curated and popular this week
50
This post presents the executive summary from Giving What We Can’s impact evaluation for 2025. At the end of this post we share links to more information, including the full report and...
Why building and backing Welfare Tech companies may be one of the most promising things we can do for billions of animals.
I used AI to assist in writing this post, but I’ve rewritten it extensively and endorse it.
* Announcing the launch of Spring Innovation Fund, a not-for-profit venture philanthropy studio and fund built specifical...
Recent opportunities to take action
I think there is a false equivalency being advanced here.
Animals did not create human beings. Human beings are by every measure, animals, with a few unique caveats just like every other species.
The potential dystopian future you reference, although possible, is unlikely. I believe that it is more likely that an individual, an institution or a group of people/institutions will use such tech as an extension of their power and coercive force.
In the relevant sense of "create," humans will not create the AIs that disempower humanity. Let me explain.
If we built AI using ordinary software ("Good Old Fashioned AI") then it would be a big file of code, every line of which would have been put there on purpose by someone & probably annotated with explanation for why it was there and how it works. And at higher levels of abstrction, the system would also be interpretable/transparent, because the higher-level structure of the system would also have been deliberately chosen by some human or group of humans.
But instead we're going to use deep learning / artificial neural networks. These are basically meta-programs that perform a search process over possible object-level programs (circuits), until they find one that performs perfectly on the training environment. So the trained neural net -- the circuit that pops out at the end of the training process -- is a tangled spaghetti mess of complex, uber-efficient structure, that is very good at scoring highly in the training environment. But that's all we know about it; we don't know how or why it works.
If this situation is still approximately true when we get to AGI -- if we create AGI by searching for it rather than by building it -- then we really won't have much control over how it behaves once it gets powerful enough to disempower us, because we won't know what it's thinking or why it does what it does.