Thomas Larsen

Working (0-5 years experience)
43Berkeley, CA, USAJoined Jul 2021



Former UMich EA club co-president. Currently doing research to prevent x-risk from AI. 


I'm  confused why almost all of the comments seem to be from people donating to many charities. For small amounts that an individual would donate, I don't imagine that diminishing marginal returns would kick in, so shouldn't one donate entirely to the charity that has the highest EV on the current margin?  

AI operates in the single-minded pursuit of a goal that humans provide it. This goal is specified in something called the reward function.

It turns out the problem is a lot worse than this -- even if we knew of a safe goal to give AI, we would have no idea how to build an AI that pursues that goal! 

See this post for more detail. Another way of saying this using the inner/outer alignment framework: reward is the outer optimization target, but this  does  automatically induce inner optimization in the same direction.