... This seems to be saying that because we are aligning AI, they will be more utilitarian. But I thought we were discussing unaligned AI? I agree that the fact we are aligning AI should make one more optimistic. Could you define what you mean by "unaligned AI"? It seems quite plausible that I will agree with your position, and think it amounts to something like "we were pretty successful with alignment". I agree with theses like "it tentatively appears that the normative value of alignment work is very uncertain, and plausibly approximately neutral, from a total utilitarian perspective", and would go further and say that alignment work is plausibly negative from a total utilitarian perspective. I disagree with the implied theses in statements like "I'm not very sympathetic to pausing or slowing down AI as a policy proposal." If you wrote a post that just said "look, we're super uncertain about things, here's your reminder that there are worlds in which alignment work is negative", I'd be on board with it. But it feels like a motte-and-bailey to write a post that is clearly trying to cause the reader to feel a particular way about some policy, and then retreat to "well my main thesis was very weak and unobjectionable". Some more minor comments: Well, I can believe it's weak in some absolute sense. My claim is that it's much stronger than all of the arguments you make put together. This is a pretty good example of something I'd call different! You even use the adjective "inhumanly"! To the extent your argument is that this is strong evidence that the AIs will continue to be altruistic and kind, I think I disagree, though I've now learned that you are imagining lots of alignment work happening when making the unaligned AIs, so maybe I'd agree depending on the specific scenario you're imagining. Sorry, I was being sloppy there. My actual claim is that your arguments either: * Don't seem to bear on the question of whether AIs are more utilitarian than humans,

Hi Toby, thank you so much for writing a comprehensive review for this time sensitive opportunity. For anyone interested in a different version of the suggested answers that Ben and Haven made that is more aimed at encouraging the adoption of on-farm monitoring systems in order to lay the groundwork for AI analysis for welfare metrics, please see this guide that I just wrote. 

I believe Deborah edited the title, and we can't blame voters for voting on the title as it stood at the time of their vote. I didn't vote, but I think it's reasonable to expect someone posting a link to say something of moderate substance about why they think it might be of value to the reader. I don't think "this is bad criticism of FHI" is enough. Mentioning that this person has a wide audience of 1.25MM would be just enough, while summarizing the critique would be better.