It seems that these are good arguments against the quick AI Doom.
I think I fall into the slow AI doom, that is we will gradually lose what is of value to us to due to too much competition in a capitalist environment. You can see the slow doom fictionalised in accelerando, in that AI doesn't kill everyone, just economically marginalises them.
Thinking about the future of uploads and brain alteration via nanotech also leads to some of the same places. Deletion of parts of current humanity by a minority leading to economic marginalisation of the people.
That an agent that can do what we currently do without the overhead of having our values can out compete us and replace us.
I think this could do with some thought before we start on any of these tracks.
My current thinking is that maybe we could form a positive cartel, that only helps and interacts with groups that are maintaining these values. You could see this as a form of attempted "value fixing", like "price fixing" in current cartels, in positive cartels they would seek to maintain the values we currently have and punish those that defect and lose values. This obviates the need for a singleton organisation, although a singleton core rule book on how the cartel works might be needed . Getting that core rule book right is important , as updating it on the fly should be hard.
It seems that these are good arguments against the quick AI Doom.
I think I fall into the slow AI doom, that is we will gradually lose what is of value to us to due to too much competition in a capitalist environment. You can see the slow doom fictionalised in accelerando, in that AI doesn't kill everyone, just economically marginalises them.
Thinking about the future of uploads and brain alteration via nanotech also leads to some of the same places. Deletion of parts of current humanity by a minority leading to economic marginalisation of the people.
That an agent that can do what we currently do without the overhead of having our values can out compete us and replace us.
I think this could do with some thought before we start on any of these tracks.
My current thinking is that maybe we could form a positive cartel, that only helps and interacts with groups that are maintaining these values. You could see this as a form of attempted "value fixing", like "price fixing" in current cartels, in positive cartels they would seek to maintain the values we currently have and punish those that defect and lose values. This obviates the need for a singleton organisation, although a singleton core rule book on how the cartel works might be needed . Getting that core rule book right is important , as updating it on the fly should be hard.