A single AGI with its utility function seems almost impossible to make safe. What happens if you have a population of AGIs each with its own utility function? Probably dangerous to make biological analogies ... but biological systems are often kept stable by the interplay between agonist and antagonist processes. For instance, one AGI in the population wants to collect stamps and another wants to keep that from happening.
A single AGI with its utility function seems almost impossible to make safe. What happens if you have a population of AGIs each with its own utility function? Probably dangerous to make biological analogies ... but biological systems are often kept stable by the interplay between agonist and antagonist processes. For instance, one AGI in the population wants to collect stamps and another wants to keep that from happening.