Thank you for the labor of writing this post, which was extremely helpful to me in clarifying my own thinking and concerns. I plan to share it widely.
"I think it would be tempting to assume that the best of these people will already have intuited the importance of scope sensitivity and existential risk, and that they’ll therefore know to give EA a chance, but that’s not how it works." This made my heart sing. EA would be so much better if more people understood this.
Happy to see this being discussed :) I may come back and write more later, but a couple quick points:
Hi all -- Cate Hall from Alvea here. Just wanted to drop in to emphasize the "we're hiring" part at the end there. We are still rapidly expanding and well funded. If in doubt, send us a CV.
Thanks so much for your detailed comment, and sorry for not seeing it earlier!
I'm a bit unclear what's going on in the Thermo-Fischer example: The second question from the initial letter makes it sound like TF had been granted a license to export under the EAR, but I don't see a claim that the technology was covered by the Commerce Control List, and the response from Ross seems to suggest otherwise (from what I can tell, I'm behind the WSJ paywall).
In any event, I think this is just the same issue that comes up generally with regulation of dual-use technologies. There's a question of whether technology with dual-use potential can be restricted from export under the CCL, and I think the answer to that is clearly yes (see, e.g., the restriction on software for DNA synthesizers). Then there's the separate question of whether it should be restricted, and that's going to require a context-dependent analysis of each case, with consideration of the balance of offensive and defensive uses of the tech. This is often a difficult question, but I think the analysis from a GCBR/advocacy perspective is going to be the same as it is for, say, differential development of technologies.
The concern about multilateral controls is a good one in general, though I think unilateral controls still pack a lot of punch when it comes to, e.g., publication of research by researchers at American universities.
Hiya -- EA lawyer here. While the US legal system is generally a mess and you can find examples of people suing for all sorts of stuff, I think the risk of giving honest feedback (especially when presented with ordinary sensitivity to people you believe to be average-or-better-intentioned) is minimal. I'd be very surprised if it contributed significantly to the bottom-line evaluation here, and would be interested to speak to any lawyer who disagreed about their reasons for doing so.
I just totally missed that the info was in the job ads -- so thank you very much for providing that information, it's really great to see. Sorry for missing it the first time around!
Just a quick note in favor of putting more specific information about compensation ranges in recruitment posts. Pay is by necessity an important factor for many people, and it feels like a matter of respect for applicants that they not spend time on the application process without having that information. I suspect having publicly available data points on compensation also helps ensure pay equity and levels some of the inherent knowledge imbalance between employers and job-seekers, reducing variance in the job search process. This all feels particularly true for EA, which is too young to have standardized roles and compensation across a lot of organizations.
I’ve been on the EA periphery for a number of years but have been engaging with it more deeply for about 6 months. My half-in, half-out perspective, which might be the product of missing knowledge, missing arguments, all the usual caveats but stronger:
Motivated reasoning feels like a huge concern for longtermism.
First, a story: I eagerly adopted consequentialism when I first encountered it for the usual reasons; it seemed, and seems, obviously correct. At some point, however, I began to see the ways I was using consequentialism to let myself off the hook, ethically. I started eating animal products more, and told myself it was the right decision because not doing so depleted my willpower and left me with less energy to do higher impact stuff. Instead, I decided, I’d offset through donations. Similar thing when I was asked, face to face, to donate to some non-EA cause: I wanted to save my money for more effective giving. I was shorter with people because I had important work I could be doing, etc., etc.
What I realized when I looked harder at my behavior was that I had never thought critically about most of these “trade-offs,” not even to check whether they were actually trade-offs! I was using consequentialism as a license to do whatever I wanted to do anyway, and it was easy to do that because it’s harder for every day consequentialist decisions to be obviously incorrect, the way deontological ones can be. Hand-wavey, “directionally correct” answers were just fine. It just so happened that nearly all of my rough cost-benefit analyses turned up the answers I wanted to hear.
I see a similar issue taking root in the longtermist community: It’s so easy to collapse into the arms of “if there’s even a small chance X will make a very good future more likely …” As with consequentialism, I totally buy the logic of this! The issue is that it’s incredibly easy to hide motivated reasoning in this framework. Figuring out what’s best to do is really hard, and this line of thinking conveniently ends the inquiry (for people who want that). My perception is that “a small chance X helps” is being invoked not infrequently to justify doing whatever work the invoker wanted to do anyway, and to excuse them internally from trying to figure out impact relative to other available options.
Longtermism puts an arbitrarily heavy weight on one side of the scales, so things look pretty similar no matter what you’re comparing it to. (Speaking loosely here: longtermism isn’t one thing, not all people are doing this, etc. etc.) Having the load-bearing component of a cost-benefit analysis be effectively impossible to calculate is a huge downside if you’re concerned about “motivational creep,” even if there isn’t a better way to do that kind of work.
I see this as an even bigger issue because, as I perceive it, the leading proponents of longtermism are also sort of the patron saints of EA generally: Will MacAskill, Toby Ord, etc. Again, the issue isn’t that those people are wrong about the merits of longtermism — I don’t think that — it’s that motivated reasoning is that much easier when your argument pattern-matches to one they’ve endorsed. I’m not sure if the model of EA as having a “culture of dissent” is accurate in the first place, but if so it seems to break down around certain people and certain fashionable arguments/topics.