This is a special post for quick takes by joshcmorrison. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Comment Permalink
I haven't had time to read all the discourse about Manifest (which I attended), but it does highlight a broader issue about EA that I think is poorly understood, which is that different EAs will necessarily have ideological convictions that are inconsistent with one another.
That is, some people will feel their effective altruist convictions motivate them to work to build artificial intelligence at OpenAI or Anthropic; others will think those companies are destroying the world. Some will try to save lives by distributing medicines; others will think the people those medicines save eat enough tortured animals to generally make the world worse off. Some will think liberal communities should exclude people who champion the existence of racial differences in intelligence; others will think excluding people for their views is profoundly harmful and illiberal.
I'd argue that the early history of effective altruism (i.e. the last 10-15 years) has generally been one of centralization around purist goals -- i.e. there're central institutions that effective altruism revolves around and specific causes and ideas that are the most correct form of effective altruism. I'm personally much more a proponent of liberal, member-first effective altruism than purist, cause-first EA. I'm not sure which of those options the Manifest example supports, but I do think it's indicative of the broader reality that for a number of issues, people on each side can believe the most effective altruist thing to do is to defeat the other.
If anyone is based in the Toronto area and wants to support challenge studies, there's a chance you could be quite helpful to 1Day Sooner's hepatitis c work. Please email me (josh@1daysooner.org) if you want to learn more.
I run an advocacy nonprofit, 1Day Sooner. When good things happen that we have advocated for, it raises the obvious question, "were we the but-for cause?"
A recent experience in our malaria advocacy work (W.H.O. prequalification of the R21 vaccine, a key advocacy target of ours) is exemplary. Prequalification was on the critical path for malaria vaccine deployment. Based on analysis of public sources and conversations with insiders, we came to the view that there was friction and possibly political pressure delaying prequalification from occurring as quickly as would be ideal. We decided to focus public pressure on a faster process (by calling for a prequalification timeline, asking Peter Singer to include the request in his op-ed on the subject, discussing the issue with relevant stakeholders, and asking journalists to inquire about it). We thought it would take at least till January and probably longer. Then a few days before Christmas, a journalist we were talking to sent us a W.H.O. press release -- that morning prequalification had been announced. Did it happen sooner because of us?
The short answer is we don't know. The reason I'm writing about it is that it highlights a type of causal uncertainty that I think is common (though not universal) in advocacy campaigns and should be relevant to EA thinking.
In some campaigns, you find yourself on the inside of a decision-maker's process in a way that can give you some amount of certainty as to your role.[1] For my kidney donor reimbursement campaign at Waitlist Zero (pre-1Day Sooner), I saw some text related to some Trump administration actions before they happened, had good transparency into the decision-making behind the Advancing American Kidney Health Initiative that my policy was a part of, and had decent confidence that my work was a but-for cause.
But for others, like the W.H.O. prequalification above or the Biden Admnistration's announcement of Project NextGen, things are much fuzzier. Something you advocated for happens without your getting advance notice. You've made the case for it publicly and perhaps exercised some levers to pressure decision-makers. Did you influence the outcome? How can you know?
I'm highlighting this experience because when it happened with NextGen I didn't really understand how to think about it, and now with prequalification I'm at least noticing the common pattern. (To be clear, I think the case for our causal influence on prequalification is stronger than for NextGen).
I think it's of note for EA advocates because it raises the challenge of evaluating advocacy through a consequentialist framework. To me the strongest theoretical challenge to consequentialism is uncertainty and the unknowability of the future. The uncertainty of advocacy impact is a very practical example of this broader challenge.
One thought about this I'd sent to a funder who asked about the NextGen campaign is the below:
So like ideally what you want as an advocacy group is to be inside-track on relevant decisions but maybe second-best is to be creating a narrative out of which the decision you are seeking manifests. That is to say we put on the performance of the need/desirability of a "Warp Speed 2" that the relevant decision-makers witnessed and participated in (even if it is unknowable whether it was causal or they were reactive to it) and this performative scaffold is sort of prima facie valuable for goals that are valuable (and the nature of advocacy/consequentialism is that oftentimes that's all that will be knowable even during a successful campaign).
To be clear, the prequalification advocacy story was different than this performative scaffolding concept -- the most obvious way we may have been causally relevant is that the comms department of the relevant entities were likely getting journalistic inquiries about the issue from some major outlets, which very possibly scared the bejesus out of them and increased the desirability of hurrying up.
I raise these case studies because I hope they can provoke further thought, discussion, and insights from EAs involved in advocacy work.
- ^
Even this is complicated because decision-makers often have an incentive to flatter you and for many issues, even if you're on the inside of the process you don't know if the process would have happened without you
As someone who works on comms stuff, I struggle with this a lot too! One thing I've found helpful is just asking decision makers, or people close to decision makers, why they did something. It's imperfect, but often helpful — e.g. when I've asked DC people what catalysed the increased political interest in AI safety, they overwhelmingly cited the CAIS letter, which seems like a fairly good sign that it worked. (Similarly, I've heard from people that Ian Hogarth's FT article may have achieved a similar effect in the UK.)
There are also proxies that can be kind of useful — if an article is incredibly widely read, and is the main topic on certain corners of Twitter for the day, and then the policy suggestions from that article end up happening, it's probably at least in part because of the article. If readership/discussion was low, you're probably not the cause.
Great to see more advocacy and advocacy evaluation-related content on the EA Forum! Sharing a few things that might be of interest to you / others
- Founders Pledge has a great document on evaluating policy organisations that puts forward some interesting considerations on evaluating the counterfactual impact of an org e.g.
- "We gather evidence from independent referees and written sources to confirm or disconfirm the charity’s own account of their impact. Below is a hierarchy of testimony evidence, ranked from the most to least desirable.
- 1. Well-informed people with incentives to downplay the role played by the organisation (e.g. a rival organisation)
- 2. Well-informed people with no incentive to mislead about the role played by the organisation (e.g. Government bureaucrats or politicians who were directly involved in the policy change"
- 3. Well-informed people who have incentives to mislead about the role played by the organisation (e.g. An organisation’s campaign partners.)
- 4. People with less information on the role played by the organisation" [I made small edits to make this shorter]
- "We gather evidence from independent referees and written sources to confirm or disconfirm the charity’s own account of their impact. Below is a hierarchy of testimony evidence, ranked from the most to least desirable.
- I also recommend Hear This Idea's podcast with Steven Teles, a political scientist who wrote a great book about advocacy within the conservative legal movement and an article about the difficulty of evaluating advocacy.
For a while I've been thinking about an idea I call Artificial Environment risk, but I haven't had the time to develop the concept in detail (or come up with a better name). The idea is roughly that the natural environment is relatively robust (since it's been around for a long time, and average species extinction rates and reasons are somewhat predictable), but as a higher proportion of humanity's environment is artificially created we voyage into an unknown without a track record of safety or stability. So the risk of dangerous phenomena increase dramatically -- as we go to a paradigm where we depend on an environment that has been roughly stable for millions of years to one for which exists evidence measured maybe in decades (or less!). Global warming and ozone degradation are obvious examples of this. But I also think that AI risk, biosecurity, nuclear war etc. fall under this (perhaps overly large) umbrella -- as humans gain more and more ability to manipulate our environment, we accumulate more and more opportunities for systemic destruction.
Some risks, like global warming, are fairly observable and relatively straightforward epistemically. But then things like the attention environment and influence of social media are more complicated -- as tools for attracting attention become more and more powerful, it creates unforeseen (and hard-to-specify or verify) effects (e.g. perhaps education polarization, weakening of elite mediation of information, harm to public discourse) that may be quite influential (and may interact with or compound other effects) but will be hard to plan for, understand, or mitigate.
One reason I find the idea worth trying to sketch out is, assuming technological development continues to progress, as humanity's control over our environment increases, risk will generally continue to rise, and fewer technologies will be realistically considered riskless. (So we will have more tradeoffs about things like whether to develop technologies that can end infectious disease but also enable better weapons).
The idea of differential technological acceleration is aimed at this problem, but I am not sure how predictable offense/defense will be or how to effectively make political decisions about which fields and industries to nurture or cull. Part of the implication I draw from categorizing this broad set of risks together is that the space for new scientific and technological development will become more crowded -- with fewer value-neutral or obviously positive opportunities for growth over time.
I think this may also tend to manifest in more of a clear division between what you might call left-EAs (progress-focused) and right-EAs (security-focused) (in some sense this corresponds to global health focused EAs vs. existential risk focused EAs currently, but the division is less clear). But that also goes to a separate view I have that EAs will have to accept more internal ideological diversity over time and recognize that the goals of different effective altruists will conflict with one another (e.g. extending human lifespan may be bad for animal welfare; synthetic biology advances to cure infectious disease may increase risks to biohazards etc.).
Very possible these ideas aren't original -- as I said they're very thinly sketched at the moment, but have been thinking about them for a while so figured I should write them out.
The hygiene hypothesis (especially the autoimmune disease variant, brief 2-paragraph summary here if you Ctrl+F "Before we go") could be another example.
On a somewhat related note, Section V of this SlateStarCodex post goes through some similar examples where humans departing from long-lived tradition has negative effects that don't become visible for a long time.
This paper starts with a simple model that formalizes a tradeoff between technological progress increasing growth/wellbeing/consumption, and having a small chance of a massive disaster that kills off a lot of people. When do we choose to keep growing? The intuitive idea is that we should keep growing if the growth rate is higher than (the odds ratio of death) times (the dollar value of a statistical life). If the value of life in dollar terms is low - because everyone is desperately poor, so dollars are worth a lot - then growth is worth it. But under very mild assumptions about preferences, where the value of life relative to money grows with income, we will eventually choose to stop growing.
However, the question becomes different is the value of new technologies is saving lives rather than increasing prosperity. Money has diminishing marginal utility; life does not. So technology that saves lives with certainty, but destroys a lot of lives with some probability, is just a gamble. We decide to keep progressing if it saves more lives in expectation than stopping, but unfortunately that's not a very helpful answer.
Curated and popular this week
·
"Part one of our challenge is to solve the technical alignment problem, and that’s what everybody focuses on, but part two is: to whose values do you align the system once you’re capable of doing that, and that may turn out to be an even harder problem", Sam Altman, OpenAI CEO (Link).
In this post, I argue that:
1. "To whose values do you align the system" is a critically neglected space I termed “Moral Alignment.” Only a few organizations work for non-humans in this field, with a total budget of 4-5 million USD (not accounting for academic work). The scale of this space couldn’t be any bigger - the intersection between the most revolutionary technology ever and all sentient beings. While tractability remains uncertain, there is some promising positive evidence (See “The Tractability Open Question” section).
2. Given the first point, our movement must attract more resources, talent, and funding to address it. The goal is to value align AI with caring about all sentient beings: humans, animals, and potential future digital minds. In other words, I argue we should invest much more in promoting a sentient-centric AI.
The problem
What is Moral Alignment?
AI alignment focuses on ensuring AI systems act according to human intentions, emphasizing controllability and corrigibility (adaptability to changing human preferences). However, traditional alignment often ignores the ethical implications for all sentient beings. Moral Alignment, as part of the broader AI alignment and AI safety spaces, is a field focused on the values we aim to instill in AI. I argue that our goal should be to ensure AI is a positive force for all sentient beings.
Currently, as far as I know, no overarching organization, terms, or community unifies Moral Alignment (MA) as a field with a clear umbrella identity. While specific groups focus individually on animals, humans, or digital minds, such as AI for Animals, which does excellent community-building work around AI and animal welfare while
·
Many thanks to Constance Li, Rachel Mason, Ronen Bar, Sam Tucker-Davis, and Yip Fai Tse for providing valuable feedback. This post does not necessarily reflect the views of my employer.
Artificial General Intelligence (basically, ‘AI that is as good as, or better than, humans at most intellectual tasks’) seems increasingly likely to be developed in the next 5-10 years. As others have written, this has major implications for EA priorities, including animal advocacy, but it’s hard to know how this should shape our strategy. This post sets out a few starting points and I’m really interested in hearing others’ ideas, even if they’re very uncertain and half-baked.
Is AGI coming in the next 5-10 years?
This is very well covered elsewhere but basically it looks increasingly likely, e.g.:
* The Metaculus and Manifold forecasting platforms predict we’ll see AGI in 2030 and 2031, respectively.
* The heads of Anthropic and OpenAI think we’ll see it by 2027 and 2035, respectively.
* A 2024 survey of AI researchers put a 50% chance of AGI by 2047, but this is 13 years earlier than predicted in the 2023 version of the survey.
* These predictions seem feasible given the explosive rate of change we’ve been seeing in computing power available to models, algorithmic efficiencies, and actual model performance (e.g., look at how far Large Language Models and AI image generators have come just in the last three years).
* Based on this, organisations (both new ones, like Forethought, and existing ones, like 80,000 Hours) are taking the prospect of near-term AGI increasingly seriously.
What could AGI mean for animals?
AGI’s implications for animals depend heavily on who controls the AGI models. For example:
* AGI might be controlled by a handful of AI companies and/or governments, either in alliance or in competition.
* For example, maybe two government-owned companies separately develop AGI then restrict others from developing it.
* These actors’ use of AGI might be dr
·
Last week, I participated in Animal Advocacy Careers’ Impactful Policy Careers programme. Below I’m sharing some reflections on what was a really interesting week in Brussels!
Please note I spent just one week there, so take it all with a grain of (CAP-subsidized) salt. Posts like this and this one are probably much more informative (and assume less context). I mainly wrote this to reflect on my time in Brussels (and I capped it at 2 hours, so it’s not a super polished draft). I’ll focus mostly on EU careers generally, less on (EU) animal welfare-related careers.
Before I jump in, just a quick note about how I think AAC did something really cool here: they identified a relatively underexplored area where it’s relatively easy for animal advocates to find impactful roles, and then designed a programme to help these people better understand that area, meet stakeholders, and learn how to find roles. I also think the participants developed meaningful bonds, which could prove valuable over time. Thank you to the AAC team for hosting this!
On EU careers generally
* The EU has a surprisingly big influence over its citizens and the wider world for how neglected it came across to me. There’s many areas where countries have basically given a bunch (if not all) of their decision making power to the EU. And despite that, the EU policy making / politics bubble comes across as relatively neglected, with relatively little media coverage and a relatively small bureaucracy.
* There’s quite a lot of pathways into the Brussels bubble, but all have different ToCs, demand different skill sets, and prefer different backgrounds. Dissecting these is hard, and time-intensive
* For context, I have always been interested in “a career in policy/politics” – I now realize that’s kind of ridiculously broad. I’m happy to have gained some clarity on the differences between roles in Parliament, work at the Commission, the Council, lobbying, consultancy work, and think tanks.
* The absorbe
For a while I've been thinking about an idea I call Artificial Environment risk, but I haven't had the time to develop the concept in detail (or come up with a better name). The idea is roughly that the natural environment is relatively robust (since it's been around for a long time, and average species extinction rates and reasons are somewhat predictable), but as a higher proportion of humanity's environment is artificially created we voyage into an unknown without a track record of safety or stability. So the risk of dangerous phenomena increase dramatically -- as we go to a paradigm where we depend on an environment that has been roughly stable for millions of years to one for which exists evidence measured maybe in decades (or less!). Global warming and ozone degradation are obvious examples of this. But I also think that AI risk, biosecurity, nuclear war etc. fall under this (perhaps overly large) umbrella -- as humans gain more and more ability to manipulate our environment, we accumulate more and more opportunities for systemic destruction.
Some risks, like global warming, are fairly observable and relatively straightforward epistemically. But then things like the attention environment and influence of social media are more complicated -- as tools for attracting attention become more and more powerful, it creates unforeseen (and hard-to-specify or verify) effects (e.g. perhaps education polarization, weakening of elite mediation of information, harm to public discourse) that may be quite influential (and may interact with or compound other effects) but will be hard to plan for, understand, or mitigate.
One reason I find the idea worth trying to sketch out is, assuming technological development continues to progress, as humanity's control over our environment increases, risk will generally continue to rise, and fewer technologies will be realistically considered riskless. (So we will have more tradeoffs about things like whether to develop technologies that can end infectious disease but also enable better weapons).
The idea of differential technological acceleration is aimed at this problem, but I am not sure how predictable offense/defense will be or how to effectively make political decisions about which fields and industries to nurture or cull. Part of the implication I draw from categorizing this broad set of risks together is that the space for new scientific and technological development will become more crowded -- with fewer value-neutral or obviously positive opportunities for growth over time.
I think this may also tend to manifest in more of a clear division between what you might call left-EAs (progress-focused) and right-EAs (security-focused) (in some sense this corresponds to global health focused EAs vs. existential risk focused EAs currently, but the division is less clear). But that also goes to a separate view I have that EAs will have to accept more internal ideological diversity over time and recognize that the goals of different effective altruists will conflict with one another (e.g. extending human lifespan may be bad for animal welfare; synthetic biology advances to cure infectious disease may increase risks to biohazards etc.).
Very possible these ideas aren't original -- as I said they're very thinly sketched at the moment, but have been thinking about them for a while so figured I should write them out.
The hygiene hypothesis (especially the autoimmune disease variant, brief 2-paragraph summary here if you Ctrl+F "Before we go") could be another example.
On a somewhat related note, Section V of this SlateStarCodex post goes through some similar examples where humans departing from long-lived tradition has negative effects that don't become visible for a long time.