Quick takes

Aimed at an audience newer to EA/AI safety It seems like many people have been confused lately about why 1) AI safety orgs are in desperate need of good hires and 2) org hiring rates are very low. My understanding is that there are many reasons for this, but a 2021 post states it pretty well: The best career advice seems to be to focus on developing a rare and valuable skill that orgs currently need. In AI safety, this skill is a combination of specific context, agency, mission-alignment, and strategic vision. You can develop it by reading a lot, writing out your thoughts, talking to people at events, and completing real projects. It's hard to do really impactful work. You need a plausible explanation for why your efforts are the best way your salary can be spent. This takes time, effort, and careful strategizing. Most of the world has high schools, colleges, and internships to train people for jobs, but a similar system is only recently becoming legible for AI safety and EA. The rare skills are rare for a reason. Still, I think the broader EA community could do a better job of communicating what it means for these "talent-gaps" to exist.

Anthropic just released its S-1 form, which contains the official financial information it must report to the SEC in order to go public. From the S-1, we now know that Anthropic's revenue grew more than 100x (specifically, 112x) from 2023 to 2025. Revenue was: * $41 million in 2023 * $386 million in 2024 * $4.59 billion in 2025 A few people in the EA community apparently believe this revenue growth rate will continue for the next several years at least. However, I am confident we won't see Anthropic bring in $514 billion in revenue in 2027, which is what extrapolation would imply. Indeed, I'm confident that Anthropic won't hit half of what extrapolation would imply, which is $257 billion in 2027. I'm willing to bet as much and would be delighted to wager $20 on it, to be donated to the charity of the winner's choice. I also have an open offer for a more general AI bubble bet.

The "Desperate" AI Safety Talent Bottleneck is deeply misleading. Please.stop. I worked at Google. Google does not tell applicants that there is a talent bottleneck and they are desperately hiring. They say "we're cool, join us!"  I didn't get into Harvard.  Harvard does not tell applicants that they are desperately seeking students. I am not misled.  I've gotten rejected from countless AI Safety organizations. AI Safety tells applicants - we are in desperate need of talent (operators/generalists)! Please join us!  I've coached 70+ aspiring career pivoters on navigating the ecosystem. A common theme is the dejection & disappointment from all this rejection. Because of this false marketing, I am the one picking up the pieces to calibrate professionals on how difficult, competitive, and picky organizations are, how to build context, and how to endure the marathon, which is like any other job hunt.  Please.stop. Some alternatives: * We are an exciting, growing organization and you should apply for our roles! * Many people are excited to work for us - here are qualities of candidates we're especially excited about. * EA roles are competitive, and we would still love to see your application. Relevant posts: * https://forum.effectivealtruism.org/posts/B6d8Wzk4gNzHsXvdi/ai-safety-is-extremely-bottlenecked-on-grantmakers?commentId=n2Rd7RR4y9EnKP42F * https://forum.effectivealtruism.org/posts/b82SLXwEHRCs3TFJA/why-experienced-professionals-fail-to-land-high-impact-roles * https://forum.effectivealtruism.org/posts/jmbP9rwXncfa32seH/after-one-year-of-applying-for-ea-jobs-it-is-really-really 

Some quick observations on the attacks on METR, EA, and other AI safety orgs (from my perspective as a political campaigner and former Communications Director for a union and mayor during COVID): 1. If you're explaining, you're losing: Every rebuttal repeats the accusation to people who haven't heard it yet. Answer the accusation once, link to it, and stop explaining. This is the "illusory truth" effect (information is true when it's repeated). 2. Silence doesn't work either: Not responding on the record suggests something is being hidden. Make one clear statement rather than amplifying in multiple twitter threads. 3. Concede what you can: A concept I know from political comms is the "admission against interest". This buys trust, disarms cynical voters, and establishes credibility. Saying "yes, AND here's what we're doing about it" will take some air out of the critique. 4. Messenger > message: Ideally people who know the organization being attacked, are from outside the AI safety world, have independent standing, or better yet, have disagreed with the organization publicly will land the defense better. 5. Going after people's personal lives or spouses is overreach: Ordinary people will obviously find this unfair and it's where their attacks will backfire. 6. Don't quote tweet and rebut: You're just distributing their message for them. I want to write something longer on this but don't have time right now. Lots of opportunity for good crisis comms here. Let me know what's wrong about this take and if you'd like to hear more!

With all the recent media attention on EA, I think what's really unclear to outsiders is the relationship between EA and Anthropic (rightfully so!). To most people it doesn't make sense that a community so focused on AI safety can also be linked to one of the leading companies. I'd personally be very interested to see a survey of what EAs think of Anthropic, but maybe that's a minefield.. Even a good explainer meant for the public could go a long way.

MATS Residency applications close this Friday, Oct 31. The Residency is built for experienced researchers with an established track record, ready to do independent, full-time AI safety work. Residents get: • $155k–$285k compensation • Abundant compute • Support from leading AI safety researchers If you know someone with a strong research background who's been thinking about AI safety, please send this their way. Apply: matsprogram.org/residency

Tentative thesis: China is unlikely to accept a subordinate position in AI capabilities to the US, just as the US is unlikely to accept a subordinate position in AI capabilities to China or anyone else.  This thesis suggests that there are two likely paths for the future international AI (non)regulation: 1) a continuing race between the US and China, perhaps joined by some late entrants, possibly with some rules established (e.g. mutual ban on autonomous weapons or something like that), or 2) a “pause or stop deal” that will establish some sort of ceiling on capabilities, which will be equal for China and the US. What I think is far less likely is a deal in which China would accept lower AI capabilities than the US. I simply don’t see a good reason why Chinese leaders would accept subordinate position. They know that China has the ability to catch-up to the US in various technological domains, as evidenced by the fact that in, like, 1990, China was behind in more or less everything, and now they are pushing the technological frontier in many areas.  Some estimates floating around the web suggest that, if the US would stop AI development now, China could reach current US capabilities within months. That is maybe overly optimistic/pessimistic depending on where you stand, but I very much doubt that catch-up would take more than a decade. And Chinese government, being patriotic about the abilities of the Chinese nation, probably will not have an absurdly pessimistic estimate. Moreover, the Chinese regime is in many ways oriented around this idea of catching up (this is a big difference between China and EU). It is a central plank of the official historical narrative of the People’s Republic of China (see for example preamble to their constitution: https://english.www.gov.cn/archive/lawsregulations/201911/20/content_WS5ed8856ec6d0b3f0e9499913.html) that the century between First Opium War and the Communist victory in the Chinese civil war in 1949 was the century

AI safety needs people everywhere but quickly stated, current talent bottlenecks to me look like: -- Founders -- Grantmakers -- (technical) Research leads  -- Policy entrepreneurs and implementors (which includes a lot of technical work) -- bets in international coordination and/or cooperation -- All manner of supporting talent -- program leads, ops proper, public outreach, content creators, comms   Most sought-after qualities for talent are: -- context, mission alignment, domain understanding, sophisticated views on AI strategy and threat modelling etc. -- "good judgement", "sound epistemics", "reasoning transparency" and other similar ideas/meta-skills from the EA/rationalist cannon -- a willingness to get shit done/bias for action (rather than be in learning mode, or people who need a lot of management and oversight, or folks with too many preferences/constraints) -- low ego, similar to above -- ambitious folks, since they would be really trying to be their own managers, take on bigger projects, grow themselves and their teams etc.    Finally, even having these, it's not enough to just claim to have these; job-seekers mainly trip up in being able to demonstrate and be legible about having them.