Hi Jeremy

I think an AGI would be much better at creating arguments for why humanity should not be eliminated. If an agi is incapable of creating these arguments itself I wonder if it capable enough to destroy humanity.

I think the thing to worry about more is that an agi correctly determines that humanity, or most of humanity needs to be destroyed (ex. Agi cares about all life and all humans murder there face mites so they must be stopped). But is that really all that bad?

Jeremy - I think it will be important to develop compelling arguments about why it's worth preserving humanity -- not so much to convince an ASI not to kill us all, but to remind ordinary citizens why AI X risk is bad, and to persuade transhumanists/posthumanists/Singularity-enthusiasts that replacing humanity ASAP with machine intelligences is a ghoulish, psychopathic, & repulsive ideology.

The trouble is, the standard 20th century arguments for preserving humanity in the face of AI are wearing a bit thin. A lot of those arguments involved sentimental appeals to human creativity, e.g. 'We should preserve humans because humans make wonderful art, music, literature, science, and humor that AI won't be able to match'. Well, AI is getting increasingly better than average humans at all of these domains, so appeals to human creativity won't be very compelling. 

What other arguments can we offer?