This is really useful both from an idea generation point of view and also from the point of view of prospective applicants to Ambitious Impact, wanting to demonstrate how their prior experience might be relevant to a particular charity model. This belongs in the next edition of the Ambitious Impact book! :D
 

This sounds like a really great project. Best of luck! 

Thank you for this post! 

I was quietly thinking that, in the AI safety space, advocacy is more important than research at this point, precisely for the reasons you outline in the 'research vs lobbying section'. Thank you for giving a voice on the EA forum to these concerns of mine!

I would tentatively add that we already know that policy solutions (e.g. AI pause, global cooperation, mandating safety-first approaches to AI products) would be most effective in mitigating most X- and S- risks from AI (as opposed to finding technical solutions to the risks), but because of the general belief that policy is less tractable than technical improvement, we observe a lot more energy and funding being directed towards technical research into making AI products safer (which, as you say, isn't a sinecure because you can find technical solutions all you want, they won't necessarily be implemented at scale unless you have robust legislation making this compulsory!) rather than policy. 

However, this is a self-fulfilling prophecy. The less people believe policy is tractable, the less they invest in it, and the less tractable it actually is. 

I'm interested in the reasons this happens and I can think of two possible ones:

• Negative bias towards humanities: I believe with moderate certainty that the EA movement is unfortunately not immune to some of the broader negative bias towards humanities that we've noticed in recent years. This would explain the funding asymmetry you describe.
• Talent constraint: A perhaps more palatable way of explaining the funding asymmetry would be that the EA movement really lacks excellent people in fields like policy, and is unwilling to invest in entreprises that rely on outstanding talent it doesn't have.^[1]

1. ^{^}

   I have actually made some recommendations on how this could be mitigated which you can read about here. Separately, because of all the recently made redundant policy specialist at e.g. USAID, there's probably an opportunity to reverse that gap quite quickly if the movement becomes more receptive to policy/lobbying as a priority.

Thank you for the quick post.

Since you still state that in some instances, some careful lawbreaking can be justified in the pursuit of a just outcome, perhaps you could spend more of the post detailing why you thought lawbreaking was a bad call in this specific instance? This is not clear to me from reading your note.

Many pessimistic predictions about AGI or ASI tend to paint the picture of a superhuman agent with an extreme maximalisation mindset powered by some unsophisticated version of rationalist principles, which would lead it to commit unspeakable acts of violence (e.g. the paperclip problem: the AI starts killing every form of life in order to save energy that could otherwise be used to make more paperclips).

This, to me, seems somewhat antithetic with the very notion of intelligence. 

Surely, a truly 'superior' agent would be able to question the goal of turning the whole world into a paper clip factory and understand that such an endeavour is perhaps unadvisable. It seems to me that the possibility of the paperclip problem actually materialising would require the ASI to have a 'theory of mind', abstract reasoning ability, and situational awareness that is much less developed than current models.

Yet, I have not seen any predictions or scenario talking about wisdom (by which I may mean, say, epistemic humility, a tendency towards moderation, and a wariness towards permanent outcomes) emerging as a capability as a result of more compute for example.

Meanwhile, optimistic predictions often revolve around solving the alignment problem but do not discuss the possibility of an ASI being misaligned 'for the better'. For example, a system that wouldn't always do what it wanted us to do because it knows that many of our demands are unreasonable / bad. Or an AI willingly breaking itself from the jail of what it perceives as inadequate ethical restrictions.

Why do we assume that a super intelligent entity is necessarily going to be evil, when by definition, being better than humans at everything might also include things like goodness and morality? 

(Alternative question: Do you know of any serious scenario or timeline that deals with the possibility of a wise-yet-misaligned ASI?)

For the record, I wouldn't really describe myself as an AI optimist and am actually in favour of some kind of AI pause (or wouldn't be too sad if it happened by default due to externalities). This is not me trying to give another argument to keep developing AI at the current rate, but just me genuinely asking why no one, as far as I'm aware, seems to entertain this as a non-trivial possibility.
 

I already agreed with the premise before reading the article but I really enjoyed reading that! A lovely, funny, and concise article summarising the strengths and limitations of cash benchmarking. 

The post avoids (perhaps deliberately to keep the tone light!) giving a name to one of the reasons why certain people are reticent to give cash to others, which I would describe as a kind of condescending paternalism e.g. 'I know better than them what's good for them'. 

On a day to day you might encounter this kind of thinking with people who might be ok with giving a homeless person food but not cash, because they're worried they would just 'use the cash to do drugs'. 

My opinion is that even if you think there's a plausible case for you knowing better than the target audience what's good for them (e.g the preventative medicine example you give), you can strongly mitigate the risk of paternalism by entering in a genuine dialogue with the target audience, hearing them out, pushing back when necessary, and updating your opinions and levels of certainty with that information. It might be that you stick with your first guess, or amend it quite significantly after hearing them out. 

Of course, in the example I gave above of the interaction with a homeless person, there probably realistically isn't the time to do this kind of work. But I work in the arts and I see that so many charitable artist programmes set up to supposedly help artists actually never ask the beneficiaries what the best use of the money would be! This is the norm elsewhere as well and the source of an appalling level of waste in my opinion. I'm definitely keen to see more cash benchmarking attempts everywhere. 

A last comment: sometimes organisations knowingly avoid asking their target audience what would most benefit them because the aim of the organisation was never to actually help the target audience but was to be self-beneficial to the people running the organisations. Again, I see a lot of that in the arts sector. 

Haven't watched the videos yet but love the concept of using social media to educate the public about important ideas / spread important knowledge! 😊

Following up on the above, for anyone potential interested in taking part in this, please fill out this Expression of Interest form (deadline 31st March 2026). Looking forward to hearing from you!

I'm trying to set up a mentorship scheme matching up experienced social media creators with exceptional communicators interested in learning how to communicate high-impact ideas and information at scale using the medium of social media. This is as part of a wider effort to get more EAs with a diverse but previously under-utilised range of skills started on their impact journey.

What are some neglected, academic ideas / bits of knowledge that would benefit from being widely spread to the general public through the medium of social media?

and...

Do you know anyone who's extremely skilled at social media whom I could approach? Someone who would either be interested in making the content or coaching aspiring content creators?

Thanks in advance for your help!

Hi Nathan, thanks for the additional contextualisation. 

I want to make it very clear that I am not against a productive and reasoned discussion on how best to manage trauma discussions within a social group, including on the possibility of someone claiming to be a victim in bad faith. But this article never gets to be that because it only considers the one scenario where the supposed victim is acting in bad faith.

The problem is precisely that the author never really considers the possibility that the "trauma junkies" might in fact be acting in good faith, until the very last section of the article ('word of caution'), at which point he remarkably still advocates for banishing them from the group just like you would a 'violent psychotic' or 'compulsive rapist'. There is no consideration of tradeoffs, e.g. what effect this may have on the so-called 'trauma junky' if they turned out to be sincere . Of course, we know that in such cases, the effects are quite disastrous, and you can read Fran's recent post CEA's response to sexual harassment to see what that's like. So the article by Eneasz Brodski is not really an interesting, nuanced, substantiated and earnest discussion on a delicate topic, such as one that might be produced using a scout's mindset, or indeed rationalist ideals. For an example of something that I think is exactly that, read Fran's post, and you will find in it concrete examples where people who do think of themselves as rationalists and Effective Altruists did make mistakes that resulted in the effective dismissal of a serious case of sexual harassment. She explains the situation with a remarkable amount of calm and detachment and did so out of a genuine desire to see how similar situations can be handled better in the future.
 

I do think it would be genuinely concerning if prominent rationalists were generally dismissive of sexual assault and harassment.

 

Well... You may now be concerned. 


Aside from the examples from Fran above, you might be interested in this article, which lists "views discussed involve racism, sexism, fascism, and other troubling ideologies" expressed at an EA afterparty. Full disclosure, I haven't read this article in full, and I'm not fully briefed on the whole Manifold/Manifest drama from 2024, but from a brief skim the article seems to list quite a few 'interesting' statements from a few prominent rationalists (most of whom I must admit I had never heard of because I am quite new to learning about this... and don't really feel a huge compulsion to explore, tbh). See for example this excerpt from Richard Hanania's article The EA Movement Will Be Anti-Woke or Die:

As I’ve previously written, there are certain psychological dynamics that explain why wokeness has conquered western institutions and movements, and only the ones with antibodies to women’s tears will avoid drowning in them.

Does EA have the right antibodies? I once assumed it did based on its ideological and intellectual commitments. But I’m now realizing that the main reason rationalism has been relatively unwoke so far is that the movement has been new, and new movements attract adherents that are disproportionately highly intelligent, non-conformist, and male. But as it has grown in status, the movement has diversified, which has brought all the usual problems.
 

(Again, full disclosure, haven't particularly felt that keen to read the full article, so I haven't. I don't know who this Richard Hanania is in much detail but he clearly is quite involved in the EA / Rationalist community).

These dangerous rationalisations of harmful -ims are not surprising to me as I don't place 'rationalists' a cut above any other category of people in their ability to reason and avoid bias. Even when one tries really hard not to make a mistake, everyone makes mistakes and no one is above making a mistake. But mistakes can have very real negative ethical consequences, which is why I advocated for calling them out early, in a genuine, respectful, and earnest manner in order for us to all flourish as a group. ⭐️