I'm not sure what to think. It feels like EA's are cool with being seen as weird. And I'm not cool with that, because IMHO it dramatically reduces our potential for impact. This being an example, where arguments which make sense about AI safety get rejected for no reason other than that they come from "Rationalists."
Ultimately do we want to feel good about ourselves, reassured that we're right even as everyone else is wrong, or do we actually want to make sure that the right decisions get taken to prevent AI catastrophe?
My problem is that it seems like ALL the coverage that EA's get is like this.
I do take your point that EA's may seem weird, but most of the EA's I know are mostly weird in a good way - more caring, more focused on helping, more concerned with the victims, ... - but we allow all this to be overshadowed by the perceived weirdness of using logic and data to help people more, or by stereotypes based on more controversial ideas like longtermism taken to its logical extreme.
And I'm not worried about its effect on our delicate egos .. I'm worried by how this limits our potential to impact the vital debates of our time. It frustrates me that EA arguments can be dismissed by ad hominem arguments. This article basically says :`"we can ignore those concerns, they only come from those weird EA's."
Absolutely. I agree. I just think it's sad - for us but also for them. My impression is that for every "successful" academic, there are 10 frustrated academics who feel they have been undervalued, disrespected, unlucky, ... and even the successful ones frequently have very high stress levels - whereas those who focus on the subject rather than the prestige tend to be happier. Of course some achieve both and that is the ideal.
I hope you're right. To be crystal clear, I fully support giving staff a good living wage. And I have no doubt that the aim team are extremely self-motivated. But I just fear that you are betting against decades of experience and research about motivation. I would love to be wrong.
I'm curious why you think this was AI/LLM reasoning.
I'm not sure what precise objection you have to the article. There is a clear, tangible problem: what we are doing now is not working. The majority of people are not engaged in AI Safety. When people are not engaged in a critical current topic, that is almost by definition a communication problem.
You didn't ask me (I think), but yes, I believe that if there is no alternative funding available, then we need to take funding from other critical EA work and use it to get the general public engaged in AI. I believe that if we do the calculation accurately, the impact per $ of getting the American and EU voters actively considering AI when they vote is enormous, as it will cause politicians to take the issue, and safety concerns, far more seriously than they are currently, and lead to sensible regulation.
One data-point to put this in context: multinational companies (Unilever, J&J, Anheiser-Busch, etc) regularly spend more money on advertising (short-form communication aimed at the general public) than they spend on producing the products they sell. This is because it is proven to make a difference, and they can measure the impact per dollar precisely.
It sounds like you either do not believe this is valuable, or do not believe it can be done. These companies, and the AI labs and the climate-deniers would not be spending so much money on communication if it wasn't effective.
The AI Safety movement does not need to spend a lot of money on this, but we do need to do the bare minimum, or we're just letting the other side dominate the public debate.
100% spot on. I have a draft post I'd been meaning to share about this:
I received 9 AI-Safety related mails in my inbox this week. The average length is 2699 words, or about 10 pages of a Word document. Most of these are full of links to much longer articles for anyone who is serious about the topic. One includes a “very strong recommendation” to listen to a 6-hour Dwarkesh podcast. And this is the short-content stuff, not the in-depth predictions like AI 2027 and Europe 2031. The content is outstanding, thoughtful, thorough and deeply researched, and I absolutely would not change it.
But in parallel to this we need to start creating sharp, CONCISE, messaging that will be read by the 99% of people who might only have a few minutes to learn about the topic and may not see why they should spend even that. These people vote too.
This requires a totally different skillset (that I do not have either). The vital soundbite. The one line, one data point that shocks people - delivered in isolation, not cluttered with caveats or even justification.
We can learn from advertisers whose job is to communicate a complex idea in 30 seconds in an engaging way. There is a science and an art to this, and those of us who enjoy long, complex arguments usually do not even realise this, let alone possess it ...
Or we can learn from our enemies. Donald Trump is a master of this - his statement that "Canada has been ripping us off for decades" is a great example. Even though we know he's a shameless liar, we're compelled to think about it - it engages us, it enables otherwise rational people to ask "but has Canada beeb screwing us all these years?" ...
In the field of AI Safety, the statement "If anyone builds it, everybody dies!" was a great example. It could have changed the game. It was the kind of idea that got people's attention.
But instead of doubling down and getting an active public debate on AI Safety on the front pages, we (the self-styled AI Safety community!!) killed all the momentum with our desire to be precise and calibrated and not to exaggerate. We watered it down. We said it wasn't as simple as that. We mentioned multiple scenarios, many of which turn out OK. We presented complex scenarios and analyses that served only to tell 99% of people that this was a very complex topic and probably not as worrying as it had seemed, and that they would never understand the arguments well enough.
We have to choose. Engaging content tends to shock. It exaggerates. It oversimplifies. But it gets people thinking. We need to realise that this is a separate but necessary step to having the AI Safety discussion that the world needs to have.
Meanwhile Donald Trump invents a lie that has 60 million people believing they understand international tariffs, which he doesn't even understand himself. We re right, but he is a far more effective communicator.
It sickens me to se this again and again - on the climate, on science, on AI, ... ; the good guys are right but the bad guys win the public debate because they are waaaaaaaay better at communicating to the general public.
When IABIED came out, we needed everyone to be joining in the chorus that this was massively scary, that yes, it was likely we'd all die, that stopping AI was the single most urgent thing that humanity needed to do. We needed to make it go viral. Because that's how you get the topic into the public eye. That's how you get voters and influencers and even politicians engaged. Once we get the engagement, once everything from pausing AI to shutting down AI labs globally is in the Overton window, then we can start hedging a bit, sharing more complex ideas. Because then people will realise that they need to engage.
But it's a 2-stage process. The first stage is to get Joe and Jane Public to get a visceral feeling that this is scary, that they need to do something about it. There are people out there who could do this, but almost all of them are being paid by the AI labs to reassure the public that there's nothing to worry about ...
I don't know if you're connected into the global EG ecosystem, if not, talk to Luke Moore, he knows everyone and everything.
Some of your challenges have been solved or are being worked on in different countries, in different ways. E.g. many European EGI's had to translate material and adapt for local consumption. Several (including us in Ireland) have struggled with tax-policies which discourage foreign donations.
For funding, have you talked to Meta-Charity Funders?
I work with a lot of PhD's and post-docs in academia and industry. My sense is that, while the incentives are nominally what they are, there is a deeper problem with using them as an excuse. If you're in an academic field to "succeed", you are in the wrong field. Academia, PhD's, post-docs and all that are about spending irrational amounts of energy and going to great depth in a subject because you want to, driven by intellectual curiosity. Some people, a small percentage, turn this into a highly paid career, but these are the exceptions, and there is a lot of luck involved - the theory you guess turns out to be right, you advisor picks a topic that turns out to be hot 10 years later, etc. You cannot control these things. But you can have a very satisfying, if not super well-paid, life. It feels like getting paid for stuff you'd happily do in your free time, arguing about to[pics you're passionate about. Once you deviate from the path of seeking the truth, you lose this. You are no longer being true to yourself. You stress about perception and luck and so much that you can't control. You envy rather than celebrate other researchers and new discoveries, It becomes psychologically stressful, you will not be happy. If you're smart enough to do a PhD but more interested in getting ahead, just become a CEO.
Thanks, this is some great perspective. It is true that I'm based in Europe and most of my direct interventions with EAs have been with Effective Giving and Animal Rights people, maybe not typical, at least in style. I do love reading EA's like Yudkowsky and MacAskill too, and often I agree with them, but I can see why many might not.
However where I would push back is on the idea that EA does not belong in the centre of the Overton window. IMHO there is hope for a post-fascist world where truth-seeking becomes again a virtue in itself. EA's themselves disagree on many things, so I'm not pushing one particular political position, but rather the idea that, when faced with challenging problems, EA-type analysis would be part of the discussion, part of the truth-seeking, with people like Scott Alexander or Elizier Yudkowsky or Will MacAskill playing the role that French intellectuals used to play - helping people to tease through the details, popularising ideas even if they can't fit in a 3-word slogan, ...
... and this becomes much easier if we can bring the concept of EA into mainstream dialogue in a positive context. Maybe we could get Billie Eilish to talk about factory farms ...
I'm not sure what to think. It feels like EA's are cool with being seen as weird. And I'm not cool with that, because IMHO it dramatically reduces our potential for impact. This being an example, where arguments which make sense about AI safety get rejected for no reason other than that they come from "Rationalists."
Ultimately do we want to feel good about ourselves, reassured that we're right even as everyone else is wrong, or do we actually want to make sure that the right decisions get taken to prevent AI catastrophe?
My problem is that it seems like ALL the coverage that EA's get is like this.
I do take your point that EA's may seem weird, but most of the EA's I know are mostly weird in a good way - more caring, more focused on helping, more concerned with the victims, ... - but we allow all this to be overshadowed by the perceived weirdness of using logic and data to help people more, or by stereotypes based on more controversial ideas like longtermism taken to its logical extreme.
And I'm not worried about its effect on our delicate egos .. I'm worried by how this limits our potential to impact the vital debates of our time. It frustrates me that EA arguments can be dismissed by ad hominem arguments. This article basically says :`"we can ignore those concerns, they only come from those weird EA's."
Absolutely. I agree. I just think it's sad - for us but also for them. My impression is that for every "successful" academic, there are 10 frustrated academics who feel they have been undervalued, disrespected, unlucky, ... and even the successful ones frequently have very high stress levels - whereas those who focus on the subject rather than the prestige tend to be happier. Of course some achieve both and that is the ideal.
Fair push-back, indeed there is no need to exaggerate when the facts are already so strong!
I hope you're right. To be crystal clear, I fully support giving staff a good living wage. And I have no doubt that the aim team are extremely self-motivated. But I just fear that you are betting against decades of experience and research about motivation. I would love to be wrong.
I'm curious why you think this was AI/LLM reasoning.
I'm not sure what precise objection you have to the article. There is a clear, tangible problem: what we are doing now is not working. The majority of people are not engaged in AI Safety. When people are not engaged in a critical current topic, that is almost by definition a communication problem.
You didn't ask me (I think), but yes, I believe that if there is no alternative funding available, then we need to take funding from other critical EA work and use it to get the general public engaged in AI. I believe that if we do the calculation accurately, the impact per $ of getting the American and EU voters actively considering AI when they vote is enormous, as it will cause politicians to take the issue, and safety concerns, far more seriously than they are currently, and lead to sensible regulation.
One data-point to put this in context: multinational companies (Unilever, J&J, Anheiser-Busch, etc) regularly spend more money on advertising (short-form communication aimed at the general public) than they spend on producing the products they sell. This is because it is proven to make a difference, and they can measure the impact per dollar precisely.
It sounds like you either do not believe this is valuable, or do not believe it can be done. These companies, and the AI labs and the climate-deniers would not be spending so much money on communication if it wasn't effective.
The AI Safety movement does not need to spend a lot of money on this, but we do need to do the bare minimum, or we're just letting the other side dominate the public debate.
100% spot on. I have a draft post I'd been meaning to share about this:
I received 9 AI-Safety related mails in my inbox this week. The average length is 2699 words, or about 10 pages of a Word document. Most of these are full of links to much longer articles for anyone who is serious about the topic. One includes a “very strong recommendation” to listen to a 6-hour Dwarkesh podcast. And this is the short-content stuff, not the in-depth predictions like AI 2027 and Europe 2031. The content is outstanding, thoughtful, thorough and deeply researched, and I absolutely would not change it.
But in parallel to this we need to start creating sharp, CONCISE, messaging that will be read by the 99% of people who might only have a few minutes to learn about the topic and may not see why they should spend even that. These people vote too.
This requires a totally different skillset (that I do not have either). The vital soundbite. The one line, one data point that shocks people - delivered in isolation, not cluttered with caveats or even justification.
We can learn from advertisers whose job is to communicate a complex idea in 30 seconds in an engaging way. There is a science and an art to this, and those of us who enjoy long, complex arguments usually do not even realise this, let alone possess it ...
Or we can learn from our enemies. Donald Trump is a master of this - his statement that "Canada has been ripping us off for decades" is a great example. Even though we know he's a shameless liar, we're compelled to think about it - it engages us, it enables otherwise rational people to ask "but has Canada beeb screwing us all these years?" ...
In the field of AI Safety, the statement "If anyone builds it, everybody dies!" was a great example. It could have changed the game. It was the kind of idea that got people's attention.
But instead of doubling down and getting an active public debate on AI Safety on the front pages, we (the self-styled AI Safety community!!) killed all the momentum with our desire to be precise and calibrated and not to exaggerate. We watered it down. We said it wasn't as simple as that. We mentioned multiple scenarios, many of which turn out OK. We presented complex scenarios and analyses that served only to tell 99% of people that this was a very complex topic and probably not as worrying as it had seemed, and that they would never understand the arguments well enough.
We have to choose. Engaging content tends to shock. It exaggerates. It oversimplifies. But it gets people thinking. We need to realise that this is a separate but necessary step to having the AI Safety discussion that the world needs to have.
Meanwhile Donald Trump invents a lie that has 60 million people believing they understand international tariffs, which he doesn't even understand himself. We re right, but he is a far more effective communicator.
It sickens me to se this again and again - on the climate, on science, on AI, ... ; the good guys are right but the bad guys win the public debate because they are waaaaaaaay better at communicating to the general public.
When IABIED came out, we needed everyone to be joining in the chorus that this was massively scary, that yes, it was likely we'd all die, that stopping AI was the single most urgent thing that humanity needed to do. We needed to make it go viral. Because that's how you get the topic into the public eye. That's how you get voters and influencers and even politicians engaged. Once we get the engagement, once everything from pausing AI to shutting down AI labs globally is in the Overton window, then we can start hedging a bit, sharing more complex ideas. Because then people will realise that they need to engage.
But it's a 2-stage process. The first stage is to get Joe and Jane Public to get a visceral feeling that this is scary, that they need to do something about it. There are people out there who could do this, but almost all of them are being paid by the AI labs to reassure the public that there's nothing to worry about ...
Fantastic! I love this idea and your plans.
I don't know if you're connected into the global EG ecosystem, if not, talk to Luke Moore, he knows everyone and everything.
Some of your challenges have been solved or are being worked on in different countries, in different ways. E.g. many European EGI's had to translate material and adapt for local consumption. Several (including us in Ireland) have struggled with tax-policies which discourage foreign donations.
For funding, have you talked to Meta-Charity Funders?
Great post, just seeing it now.
I work with a lot of PhD's and post-docs in academia and industry. My sense is that, while the incentives are nominally what they are, there is a deeper problem with using them as an excuse. If you're in an academic field to "succeed", you are in the wrong field. Academia, PhD's, post-docs and all that are about spending irrational amounts of energy and going to great depth in a subject because you want to, driven by intellectual curiosity. Some people, a small percentage, turn this into a highly paid career, but these are the exceptions, and there is a lot of luck involved - the theory you guess turns out to be right, you advisor picks a topic that turns out to be hot 10 years later, etc. You cannot control these things. But you can have a very satisfying, if not super well-paid, life. It feels like getting paid for stuff you'd happily do in your free time, arguing about to[pics you're passionate about. Once you deviate from the path of seeking the truth, you lose this. You are no longer being true to yourself. You stress about perception and luck and so much that you can't control. You envy rather than celebrate other researchers and new discoveries, It becomes psychologically stressful, you will not be happy. If you're smart enough to do a PhD but more interested in getting ahead, just become a CEO.
Thanks, this is some great perspective. It is true that I'm based in Europe and most of my direct interventions with EAs have been with Effective Giving and Animal Rights people, maybe not typical, at least in style. I do love reading EA's like Yudkowsky and MacAskill too, and often I agree with them, but I can see why many might not.
However where I would push back is on the idea that EA does not belong in the centre of the Overton window. IMHO there is hope for a post-fascist world where truth-seeking becomes again a virtue in itself. EA's themselves disagree on many things, so I'm not pushing one particular political position, but rather the idea that, when faced with challenging problems, EA-type analysis would be part of the discussion, part of the truth-seeking, with people like Scott Alexander or Elizier Yudkowsky or Will MacAskill playing the role that French intellectuals used to play - helping people to tease through the details, popularising ideas even if they can't fit in a 3-word slogan, ...
... and this becomes much easier if we can bring the concept of EA into mainstream dialogue in a positive context. Maybe we could get Billie Eilish to talk about factory farms ...