I've been working on an "ethics calculator" that I believe could be used to help align an AGI to act ethically. I've posted on this on lesswrong.com. Got mostly downvoted. (If you have thoughts on why, please share them.) I'm working on an update on my progress - would this be something of interest to people on this site? Should I post/crosspost the update here? Note that my version of ethics is not the "standard" EA ethics. Here's a snippit from the update post that's still in progress:
In its first iteration, the calculator uses a utilitarian framework that measures “utility” in terms of value defined as net “positive” experiences, including the expected effects of raising people’s self-esteem levels on how they’d tend to rate more experiences as “positive,” and explicitly including the value of rights. In some sense, this framework combines utilitarianism (through adding up value), moderate deontology (through valuing rights), and virtue ethics (through self-esteem).
My goals are to get people thinking about the importance of having an "ethics module" ready to go when the first AGI comes online, and also to secure funding for future work. Thanks for any feedback you have!