[ Question ]

What are the challenges and problems with programming law-breaking constraints into AGI?

by MichaelStJules1 min read2nd Feb 202034 comments


AI GovernanceAI Alignment

I'm thinking the objective function could have constraints on the expected number of times the AI breaks the law, or the probability that it breaks the law, e.g.

  • only actions with a probability of breaking any law < 0.0001 are permissible, or
  • only actions for which the expected number of broken laws is < 0.001 are permissible.

There could also be separate constraints for individual laws or groups of laws, and these could depend on the severity of the penalties.

Looser constraints like this seem like they could avoid issues of lexicality and prioritizing avoidance of breaking the law over everything we want the AI to actually do, since the surest way to avoid breaking the law completely would be to never do anything (although we could also have a separate constraint for this).

Of course, the constraints should depend on breaking the law, not just being caught breaking the law, so the AI should predict whether or not it will break the law, not merely whether or not it will be caught breaking the law.

The AI could also predict whether or not it will break laws that don't exist now but will in the future (possibly even in response to its actions).

What are the challenges and problems with such an approach? Would it be too difficult to capture such constraints? Are laws too imprecise or ambiguous for this? Can we just have the AI consider multiple interpretations of the laws or try to predict how a human (or human judge) would interpret the law and apply it to its actions given the information the AI has?

How much work should the AI spend on estimating the probabilities that it will break laws?

What kinds of cases would it miss, say, given current laws?

New Answer
Ask Related Question
New Comment

1 Answers

You're focusing on the issue that current laws don't capture everything we care about, which is definitely a problem.

However, the bigger problem is that there isn't a clear definition of what does and doesn't break the law that you can write down in a program.

You might say that we could train an AI system to learn what is and isn't breaking the law; but then you might as well train an AI system to learn what is and isn't the thing you want it to do. It's not clear why training to follow laws would be easier than training it to do what you want; the latter would be a much more useful AI system.