Since I started PauseAI, I’ve encountered a wall of paranoid fear from EAs and rationalists that the slightest amount of wrongthink or willingness to use persuasive speech as an intervention will taint the person’s mind for life with self-deception-- that “politics” will kill their mind. I saw people shake in fear to join a protest of an industry they thought would destroy the world if unchecked because they didn’t want to be photographed next to an “unnuanced” sign. They were afraid of sinning by saying something wrong. They were afraid of sinning by even trying to talk persuasively!
The worry about destroying one’s objectivity was often phrased to me as “being a scout/not being a soldier”, referring to Julia Galef’s book Scout Mindset. I think we have all the info we need to contradict the fear of not being a scout in her metaphor. Scouts are important for success in battle because accurate information is important to draw up a good battle plan. But those battle plans are worthless without soldiers to fight the battle! “Everyone Should be a Mapmaker and Fear that Using the Map to Actually Do Something Could Make Them a Worse Mapmaker” would be a much less rousing title, but this is how many EAs and rationalists have chosen to interpret the book.
Even a scout can’t be only a scout. If a scout reports what they found to a superior officer, and the officer wants to pretend they didn’t hear it, a good scout doesn’t just stay curious about the situation or note that the superior officer has chosen a narrative. They fight to be heard! Because the truth of what they saw matters to the war effort. The success of the scout and the officer and the soldier is all ultimately measured in the outcome of the war. Accurate intel is important for something larger than the map— for the battle.
Imagine if the insecticide-treated bednets hemmed and hawed about the slight chance of harm from their use in anti-malaria interventions. Would that help one bit? No! What helps is working through foreseeable issues ahead of time at the war table, then actually trying the intervention with each component fully committed. Bednets are soldiers, and all our thinking about the best interventions would be useless if there were no soldiers to actually carry the interventions out. Advocating for the PauseAI proposal and opposing companies who are building AGI through protests is an intervention, much like spreading insecticide-treated bednets, but instead of bednets the soldiers are people armed with facts and arguments that we hope will persuade the public and government officials.
Interventions that involve talking, thinking, persuasion, and winning hearts and minds require commitment to the intervention and not simply to the accuracy of your map or your reputation for accurate predictions. To be a soldier in this intervention, you have to be willing to be part of the action itself and not just part of the zoomed out thinking. This is very scary for a contingent of EAs and rationalists today who treat thinking and talking as sacred activities that must follow the rules of science or lesswrong and not be used for anything else. Some of them would like to entirely forbid "politics" (by which they generally mean trying to persuade people of your position and get them on your side) or "being a [rhetorical] soldier" out of the fear that people cannot compartmentalize persuasive speech acts from scout thinking and will lose their ability to earnestly truth-seek.
I think these concerns are wildly overblown. What are the chances that amplifying the message of an org you trust in a way the public will understand undermines your ability to think critically? That's just contamination thinking. I developed the PauseAI US interventions with my scout hat on. When planning a protest, I'm an officer. At the protest, I'm a soldier. Lo and behold, I am not mindkilled. In fact, it's illuminating to serve in all of those roles-- I feel I have a better and more accurate map because of it. Even if I didn't, a highly accurate map simply isn't necessary for all interventions. Advocating for more time for technical safety work and for regulations to be established is kind of a no-brainer.
It's noble to serve as a soldier when we need humans as bednets to carry out the interventions that scouts have identified and officers have chosen to execute. Soldiers win wars. The most accurate map made by the most virtuous scout is worth nothing without soldiers to do something with it.
i'm not bothered by your comments.
your first reply seemed to be about how i worded the point (you wrote "obnoxiously posed", and reworded it) rather than pedanticness/irrelevance. i mentally replaced "this is obnoxious" with "this makes me feel annoyed", which i think is okay to say. i also considered letting you know i'm autistic, which makes me word things differently or more literally[1] or in ways that can seem to have unintended emotional content. (i wonder if that's what made it feel like "marking it up in red pen")
onto object-level: what i wrote actually seemed substantive to me, i.e. it really did seem to me that the quote in point 2 was strongly misrepresenting the position the post intended to argue against, so i wouldn't consider it pedantic. (it could separately be false, of course)
it did not occur to me that you might endorse the scout/soldier metaphor, and just be using the existence of scout/soldier in 'scout/soldier mindset' to bring it up; so yes, if that's actually the case, it would have been better to notice that and then either not comment on it or probe it as you say. using a metaphor is not invalid.
here's how i perceived it at the time: 'scout mindset' and 'soldier mindset' have particular meanings, so whether traditional soldiers are necessary for traditional scouts is a different topic. writing about them instead seemed 'opportunistic' in some sense, as if the text was using the terminological overlap to sneak through an argument about one as about the other.
i wonder if this thread could have been mitigated if i were more clear about that in my initial comment. if anyone has advice it is welcome.
maybe 'more structured like the thought is structured internally'