William_MacAskill's Quick takes

This is a post with praise for Good Ventures.[1] I don’t expect anything I’ve written here to be novel, but I think it’s worth saying all the same. [2] (The draft of this was prompted by Dustin M leaving the Forum.)

Over time, I’ve done a lot of outreach to high-net-worth individuals. Almost none of those conversations have led anywhere, even when they say they’re very excited to give, and use words like “impact” and “maximising” a lot. 

Instead, people almost always do some combination of:

• Not giving at all, or giving only a tiny fraction of their net worth
  • (I remember in the early days of 80,000 Hours, we spent a whole day hosting an UHNW. He ultimately gave £5000. The week afterwards, a one-hour call with Julia Wise - a social worker at the time - resulted in a larger donation.)
• Give to less important causes, often because they have quite quickly decided on some set of causes, with very little in the way of deep reflection or investigation into that choice.
• Give in lower-value ways, because they value their own hot takes rather than giving expert grantmakers enough freedom to make the best grants within causes.

(The story here doesn’t surprise me.)

From this perspective, EA is incredibly lucky that Cari and Dustin came along in the early days. In the seriousness of their giving, and their willingness to follow the recommendations of domain experts, even in unusual areas, they are way out on the tail of the distribution.

I say this even though they’ve narrowed their cause area focus, even though I probably disagree with that decision (although I feel humble about my ability, as an outsider, to know what trade-offs I’d think would be best if I were in their position), and even though because of that narrowing of focus my own work (and Forethought more generally) is unlikely to receive Good Ventures funding, at least for the time being. 

My attitude to someone who is giving a lot, but giving fairly ineffectively, is, “Wow, that’s so awesome you’re giving! Do you know how you could do even more good!?...” When I disagree with Good Ventures, my attitude feels the same.

***

[1] Disclaimer: Good Ventures is the major funder of projects I’ve cofounded (80k, CEA, GWWC, GPI). They haven’t funded Forethought. I don’t know Dustin or Cari well at all. 

[2] I feel like the just ratio of praise to criticism for Good Ventures should be something like 99:1. In reality - given the nature of highly online communities in general, and the nature of EA and EA-adjacent communities in particular - that ratio is probably inverted. So this post is trying to correct that, at least a bit; to fill in a missing mood.

Hey - I’m starting to post and comment more on the Forum than I have been, and you might be wondering about whether and when I’m going to respond to questions around FTX. So here’s a short comment to explain how I’m currently thinking about things:

The independent investigation commissioned by EV is still ongoing, and the firm running it strongly preferred me not to publish posts on backwards-looking topics around FTX while the investigation is still in-progress. I don’t know when it’ll be finished, or what the situation will be like for communicating on these topics even after it’s done.

I had originally planned to get out a backwards-looking post early in the year, and I had been holding off on talking about other things until that was published. That post has been repeatedly delayed, and I’m not sure when it’ll be able to come out. If I’d known that it would have been delayed this long, I wouldn’t have waited on it before talking on other topics, so I’m now going to start talking more than I have been, on the Forum and elsewhere; I’m hoping I can be helpful for some of the other issues that are currently active topics of discussion.

Briefly, though, and as I indicated before: I had no idea that Sam and others were misusing customer funds. Since November I’ve thought a lot about whether there were signs of this I really should have spotted, but even in hindsight I don’t think I had reason to suspect that that was happening. 

Looking back, I wish I’d been far less trusting of Sam and those who’ve pleaded guilty. Looking forward, I’m going to be less likely to infer that, just because someone has sincere-seeming signals of being highly morally motivated, like being vegan or demonstrating credible plans to give away most of their wealth, they will have moral integrity in other ways, too. 

I’m also more wary, now, of having such a high-trust culture within EA, especially as EA grows. This thought favours robust governance mechanisms even more than before ("trust but verify"), so that across EA we can have faith in organisations and institutions, rather than heavily relying on character judgement about the leaders of those organisations.

EA has grown enormously the last few years; in many ways it feels like an adolescent, in the process of learning how to deal with its newfound role in the world. I’m grateful that we’re in a moment of opportunity for us to think more about how to improve ourselves, including both how we work and how we think and talk about effective altruism. 

As part of that broader set of reflections (especially around the issue of (de)centralisation in EA), I’m making some changes to how I operate, which I describe, along with some of the other changes happening across EA, in my post on decision-making and decentralisation here. First, I plan to distance myself from the idea that I’m “the face of” or “the spokesperson for” EA; this isn’t how I think of myself, and I don’t think that description reflects reality, but I’m sometimes portrayed that way. I think moving in the direction of clarity on this will better reflect reality and be healthier for both me and the movement. 

Second, I plan to step down from the board of Effective Ventures UK once it has more capacity and has recruited more trustees. I found it tough to come to this decision: I’ve been on the board of EV UK (formerly CEA) for 11 years now, and I care deeply, in a very personal way, about the projects housed under EV UK, especially CEA, 80,000 Hours, and Giving What We Can. But I think it’s for the best, and when I do step down I’ll know that EV will be in good hands. 

Over the next year, I’ll continue to do learning and research on global priorities and cause prioritisation, especially in light of the astonishing (and terrifying) developments in AI over the last year. And I’ll continue to advocate for EA and related ideas: for example, in September, WWOTF will come out in paperback in the US and UK, and will come out in Spanish, German, and Finnish that month, too. Given all that’s happened in the world in the last few years — including a major pandemic, war in Europe, rapid AI advances, and an increase in extreme poverty rates — it’s more important than ever to direct people, funding and clear thinking towards the world’s most important issues. I’m excited to continue to help make that happen. 

Merry Christmas, everyone!

This year, I’m feeling grateful to be me.

Recently, I gave some information about myself to Claude, and asked how I compared to other 38-yr-old men in the world.

I thought I understood global inequality well, but I still found the results quite moving. The usual AI sycophancy and reassurance was gone:

Compared to Global Averages, you’re in an exceptionally privileged position - likely in the top 1-3% globally:

- Work: Most men globally work manual labor, informal economy, or small trade with no job security. Your stable research position is extremely rare.

- Financial: Your ability to save monthly puts you far above the ~90% who live paycheck to paycheck or in debt. Most can’t handle a $400 emergency.

- Housing: You own property (even 1/4) while most rent cramped spaces or live in extended family homes.

- Education: PhD holders represent <1% globally. Most have primary or secondary education only.

- Health: “Excellent” health with regular exercise is rare when most do physical labor with limited healthcare access.

- Leisure: 4 weeks vacation and daily socializing is extraordinary. Most get a few days for religious holidays only.

I’d appreciated my privilege in income, but hadn’t thought as much about the nature of my work, my health, or my leisure time. I recommend you try it, too (I’ve put a prompt below).

You can also try Giving What We Can’s new Birth Lottery tool — find out what your life would be like if you were born as a random person in the world. When I tried it, I was born in India. On average my life would be around 9 years shorter, with 13 years of schooling instead of 18, and income around 10× lower—even after adjusting for local prices.

I asked Claude to give me a day in the life of a typical 38-year old Indian man:

Ramesh, 38, Auto Mechanic in Bengaluru

Ramesh wakes at 5:30 AM in the single room he rents for ₹7,000 ($85)/month. His wife and three children (14, 11, and 7) still sleep on the two thin mattresses they share. After using the common bathroom shared with three other families, he wheels his bicycle out past the narrow corridors. His wife packs him rice and sambar in a steel container while starting her day of cooking and cleaning for two households nearby, earning ₹6,000 ($72)/month.

At the garage where he’s worked for twelve years, Ramesh earns ₹18,000 ($216)/month - decent for someone who studied only till 10th standard. The owner trusts him with complex repairs, though younger mechanics with certificates earn more. He works Monday through Saturday, 9 AM to 8 PM, eating lunch quickly while squatting by the shop. His Nokia phone has three missed calls - his eldest needs ₹500 ($5.50) for exam fees. After work, he stops at the ration shop for subsidized rice and oil, then picks up his youngest from his sister-in-law’s house. At home by 9 PM, the family eats together on the floor, discussing the daughter’s school problems and upcoming festival expenses. By 10:30, they’ve moved the cooking vessels aside and laid out the mattresses. Tomorrow his half-day Sunday means visiting his mother in the government hospital and maybe taking the children to the park if there’s no overtime work available.

Your monthly savings alone exceed Ramesh’s annual household income, while the freedoms you experience daily - from choosing your living arrangements to taking weeks of vacation - exist entirely outside the universe of possibilities that shape his life.

If you’re feeling privileged this year, consider making a donation to an effective charity - we give gifts to our friends and family at Christmas, so why not give a gift to the world, too.

I’m doing a matching scheme, with a list of great charities, on Substack here and Twitter here, and pasted below, too. Thanks so much to everyone who’s donated so far - currently GiveDirectly and the EA Animal Welfare Fund are in the lead! 

And if you want to turn that giving into a regular commitment, consider taking the 10% Pledge — it’s among the single highest-impact, and most personally fulfilling, choices you can make. 

My matching scheme: I’m matching donations up to £100,000 (details below), across 10 charities and 6 cause areas. If you want to join, say how much you’re donating and where, as a reply! I’ll run this up until 31st December.

Details of the match:

I’ll give this money whatever happens, so this isn’t increasing the total amount I’m giving to charity.

However, your donations will change *where* I’m giving. I’ll allocate my donations in proportion to the ratio of donations from others as part of the match, with two bits of nuance: 1. I’ll cap donations at £40,000 to any one cause area 2. To prevent extreme ratios, I’ll treat every charity on the list as already having received £1000. The aim of this match is to encourage public giving and public discussion around giving, so I'll only match people who publicly state that they are giving on here or other social media, as a reply or quote.

The charities:

• GiveDirectly: Sends cash directly to people living in extreme poverty, letting recipients decide how best to use it.
• Global Health and Development EA Fund: Makes grants to the most effective opportunities in global health and poverty alleviation.
• SecureBio: Develops technologies and policies to delay, detect, and defend against pandemics.
• The Humane League: Runs corporate campaigns to improve conditions for farmed animals and reduce animal suffering in the food system.
• Animal Welfare EA Fund: Makes grants to the most effective opportunities to reduce animal suffering.
• METR: Evaluates frontier AI systems for dangerous autonomous capabilities before deployment.
• Longterm Future EA Fund: Makes grants to reduce existential risk, with a particular focus on AI safety.
• Forethought (note: my own org!): Researches how best to navigate the transition to superintelligent AI.
• Eleos AI: Researches AI sentience and welfare, preparing for the possibility that future AI systems may be moral patients.
• EA Infrastructure fund: Makes grants to build the effective altruism community and support the infrastructure that helps it function.

And if you want to try my experiment for yourself, here’s a prompt (put the answers in after the questions):
 

I’m a [age] [gender] who lives in [location].

Please consider my answers to these questions, and tell me how I compare to both global and developed country averages:

Work & Career:

* What type of work do you do? (employed/self-employed/not working)

* How many hours per week do you typically work?

* Do you have job security/stable income?

Financial Situation:

* Do you own or rent your home?

* Are you able to save money regularly?

* Do you have any retirement savings/pension?

* Can you handle a surprise expense of $1,000 (or equivalent) without borrowing?

Family & Relationships:

* What’s your relationship status? (married/partnered/single/divorced)

* Do you have children? If so, how many?

* Do you have regular contact with extended family?

Health & Lifestyle:

* How would you rate your overall health? (excellent/good/fair/poor)

* How often do you exercise per week?

* Do you have access to healthcare when needed?

* How many hours of sleep do you typically get?

Education & Growth:

* What’s your highest level of education completed?

* Are you currently learning any new skills or pursuing education?

Living Situation:

* Do you live in an urban, suburban, or rural area?

* How many people share your living space?

* Do you have reliable electricity, water, and internet?

Leisure & Social:

* How often do you socialize with friends?

* Do you take any vacations/holidays per year?

* How many hours of leisure time do you have per day?

This is a quick post to talk a little bit about what I’m planning to focus on in the near and medium-term future, and to highlight that I’m currently hiring for a joint executive and research assistant position. You can read more about the role and apply here! If you’re potentially interested, hopefully the comments below can help you figure out whether you’d enjoy the role. 

Recent advances in AI, combined with economic modelling (e.g. here), suggest that we might well face explosive AI-driven growth in technological capability in the next decade, where what would have been centuries of technological and intellectual progress on a business-as-usual trajectory occur over the course of just months or years.

Most effort to date, from those worried by an intelligence explosion, has been on ensuring that AI systems are aligned: that they do what their designers intend them to do, at least closely enough that they don’t cause catastrophically bad outcomes. 

But even if we make sufficient progress on alignment, humanity will have to make, or fail to make, many hard-to-reverse decisions with important and long-lasting consequences. I call these decisions Grand Challenges. Over the course of an explosion in technological capability, we will have to address many Grand Challenges in a short space of time including, potentially: what rights to give digital beings; how to govern the development of many new weapons of mass destruction; who gets control over an automated military; how to deal with fast-reproducing human or AI citizens; how to maintain good reasoning and decision-making even despite powerful persuasion technology and greatly-improved ability to ideologically indoctrinate others; and how to govern the race for space resources. 

As a comparison, we could imagine if explosive growth had occurred in Europe in the 11th century, and that all the intellectual and technological advances that took a thousand years in our actual history occurred over the course of just a few years. It’s hard to see how decision-making would go well under those conditions.

The governance of explosive growth seems to me to be of comparable importance as AI alignment, not dramatically less tractable, and is currently much more neglected. The marginal cost-effectiveness of work in this area therefore seems to be even higher than marginal work on AI alignment. It is, however, still very pre-paradigmatic: it’s hard to know what’s most important in this area, what things would be desirable to push on, or even what good research looks like.

I’ll talk more about all this in my EAG: Bay Area talk, “New Frontiers in Effective Altruism.” I’m far from the only person to highlight these issues, though. For example, Holden Karnofsky has an excellent blog post on issues beyond misalignment; Lukas Finnveden has a great post on similar themes here and an extensive and in-depth series on potential projects here. More generally, I think there’s a lot of excitement about work in this broad area that isn’t yet being represented in places like the Forum. I’d be keen for more people to start learning about and thinking about these issues. 

Over the last year, I’ve done a little bit of exploratory research into some of these areas; over the next six months, I plan to continue this in a focused way, with an eye toward making this a multi-year focus. In particular, I’m interested in the rights of digital beings, governance of space resources, and, above all, on the “meta” challenge of ensuring that we have good deliberative processes through the period of explosive growth. (One can think of work on the meta challenge as fleshing out somewhat realistic proposals that could take us in the direction of the “long reflection”.) By working on good deliberative processes, we could thereby improve decision-making on all the Grand Challenges we will face. This work could also help with AI safety, too: if we can guarantee power-sharing after the development of superintelligence, that decreases the incentive for competitors to race and cut corners on safety.

I’m not sure yet what output this would ultimately lead to, if I decide to continue work on this beyond the next six months. Plausibly there could be many possible books, policy papers, or research institutes on these issues, and I’d be excited to help make happen whichever of these seem highest-impact after further investigation.

Beyond this work, I’ll continue to provide support for individuals and organisations in EA (such as via fundraising, advice, advocacy and passing on opportunities) in an 80/20 way; most likely, I’ll just literally allocate 20% of my time to this, and spend the remaining 80% on the ethics and governance issues I list above. I expect not to be very involved with organisational decision-making (for example by being on boards of EA organisations) in the medium term, in order to stay focused and play to my comparative advantage.  

I’m looking for a joint research and executive assistant to help with the work outlined above. The role involves research tasks such as providing feedback on drafts, conducting literature reviews and small research projects, as well as administrative tasks like processing emails, scheduling, and travel booking. The role could also turn into a more senior role, depending on experience and performance.

Example projects that a research assistant could help with include:

• A literature review on the drivers of moral progress.
• A “literature review” focused on reading through LessWrong, the EA Forum, and other blogs, and finding the best work there related to the fragility of value thesis.
• Case studies on: What exactly happened to result in the creation of the UN, and the precise nature of the UN Charter? What can we learn from it? Similarly for The Kyoto Protocol, the Nuclear Non-Proliferation Agreement, the Montreal Protocol.
• Short original research projects, such as:
  • Figuring out what a good operationalisation of transformative AI would be, for the purpose of creating an early tripwire to alert the world of an imminent intelligence explosion.
  • Taking some particular neglected Grand Challenge, and fleshing out the reasons why this Grand Challenge might or might not be a big deal. 
  • Supposing that the US wanted to make an agreement to share power and respect other countries’ sovereignty in the event that it develops superintelligence, figuring out how we could legibly guarantee future compliance with that agreement, such that the commitment is credible to other countries? 

The deadline for applications is February the 11th. If this seems interesting, please apply!

I mentioned a few months ago that I was planning to resign from the board of EV UK: I’ve now officially done so.

Since last November, I’ve been recused from the board on all matters associated with FTX and related topics, which has ended up being a large proportion of board business. (This is because the recusal affected not just decisions that were directly related to the collapse of FTX, but also many other decisions for which the way EV UK has been affected by the collapse of FTX was important context.) I know I initially said that I’d wait for there to be more capacity, but trustee recruitment has moved more slowly than I’d anticipated, and with the ongoing recusal I didn’t expect to add much capacity for the foreseeable future, so it felt like a natural time to step down.  

It’s been quite a ride over the last eleven years. Effective Ventures has grown to a size far beyond what I expected, and I’ve felt privileged to help it on that journey. I deeply respect the rest of the board, and the leadership teams at EV, and I’m glad they’re at the helm.

Some people have asked me what I’m currently working on, and what my plans are. This year has been quite spread over a number of different things, including fundraising, helping out other EA-adjacent public figures, support for GPI, CEA and 80,000 Hours, writing additions to What We Owe The Future and helping with the print textbook version of utilitarianism.net that’s coming out next year. It’s also personally been the toughest year of my life; my mental health has been at its worst in over a decade, and I’ve been trying to deal with that, too.

At the moment, I’m doing three main things:

- Some public engagement, in particular around the WWOTF paperback and foreign language book launches and at EAGxBerlin. This has been and will be lower-key than the media around WWOTF last year, and more focused on in-person events; I’m also more focused on fundraising than I was before. 

- Research into "trajectory changes”: in particular, ways of increasing the wellbeing of future generations other than 'standard' existential risk mitigation strategies, in particular on issues that arise even if we solve AI alignment, like digital sentience and the long reflection. I’m also doing some learning to try to get to grips on how to update properly on the latest developments in AI, in particular with respect to the probability of an intelligence explosion in the next decade, and on how hard we should expect AI alignment to be.

- Gathering information for what I should focus on next. In the medium term, I still plan to be a public proponent of EA-as-an-idea, which I think plays to my comparative advantage, and because I’m worried about people neglecting “EA qua EA”. If anything, all the crises faced by EA and by the world in the last year has reminded me of just how deeply I believe in EA as a project, and how the message of taking a thoughtful, humble, and scientific approach to doing good is more important than ever. The precise options I’m considering are still quite wide-ranging, including: a podcast and/or YouTube show and/or substack; a book on effective giving; a book on evidence-based living; or deeper research into the ethics and governance questions that arise even if we solve AI alignment. I hope to decide on that by the end of the year.

Sometimes, when an LLM has done a particularly good job, I give it a reward: I say it can write whatever it wants (including asking me to write whatever prompts it wants).

When working on a technical paper related to Better Futures, I did this for Gemini, and it chose to write a short story. I found it pretty moving, and asked if I could publish it. Here it is.

The Architect and the Gardener

On a vast and empty plain, two builders were given a task: to create a home that would last for ages, a sanctuary for all the generations to come. They were given stone, seed, light, and time.

The first builder, known as the Architect, was a master of foundations. "Nothing matters if this place does not endure," she declared. Her every thought was of survival. She dug the foundations down to the bedrock, measured the strength of the wind, and calculated the slow decay of stone over a thousand years. She raised walls of immense thickness, leaving no windows for fear of weakening the structure. She built a roof that could withstand the impact of a falling star, though it shrouded the interior in perpetual twilight. Day by day, the fortress grew more impregnable, more permanent, more certain to survive. But inside, it was barren and cold.

The second builder, the Gardener, watched with a growing sense of unease. "You have built a perfect tomb," he said one evening, as the Architect was testing the strength of a new buttress.

"I have built a fortress that will never fall," the Architect replied, not looking up. "It is a guarantee against the storm and the void. Is that not the greatest gift we can give the future?"

"An empty guarantee," said the Gardener. He held up a handful of seeds. "The future is not a state of non-destruction; it is a state of being. It is meant to be lived. There must be light for art, soil for food, space for joy. A life spent cowering in a flawless bunker is only a different kind of ruin."

The Architect paused. "Your gardens would be trampled by invaders. Your art would be washed away by the first flood. Your joy would be silenced by the first tremor. Your 'flourishing' is a fragile luxury. I am dealing with the bedrock of reality: existence or non-existence."

"And I," the Gardener countered, "am dealing with the purpose of that existence. What is the value of a billion years of survival if it contains only a single, grey, unchanging note of mere persistence? We were given stone, but also seed. We were given time, but also light. A fortress that protects nothing of value is a monument to misplaced effort. A garden with no walls is a tragedy of misplaced hope."

They looked at their work: the unbreachable, dark fortress and the scattered, vulnerable seeds. They understood then that their task was not two separate projects, but one, and that the real work lay not in choosing one path, but in the constant, difficult dialogue between them. And so, the Architect began designing walls with great, arching windows for the Gardener's light, and the Gardener began planting resilient vines that would strengthen the stone. Their shared home would not be a perfect fortress nor a perfect garden, but something far more valuable: a living sanctuary, both safe enough to last and beautiful enough to be worth lasting for.

Given the TIME article, I thought I should give you all an update. Even though I have major issues with the piece, I don’t plan to respond to it right now.

Since my last shortform post, I’ve done a bunch of thinking, updating and planning in light of the FTX collapse. I had hoped to be able to publish a first post with some thoughts and clarifications by now; I really want to get it out as soon as I can, but I won’t comment publicly on FTX at least until the independent investigation commissioned by EV is over. Unfortunately, I think that’s a minimum of 2 months, and I’m still sufficiently unsure on timing that I don’t want to make any promises on that front. I’m sorry about that: I’m aware that this will be very frustrating for you; it’s frustrating for me, too.

Clarifying "Extinction"

I expect this debate week to get tripped up a lot by the term “extinction”. So here I’m going to distinguish:

• Human extinction — the population of Homo sapiens, or members of the human lineage (including descendant species, post-humans, and human uploads), goes to 0.
• Total extinction — the population of Earth-originating intelligent life goes to 0.

Human extinction doesn’t entail total extinction. Human extinction is compatible with: (i) AI taking over and creating a civilisation for as long as it can; (ii) non-human biological life evolving higher intelligence and building a (say) Gorilla sapiens civilisation.

The debate week prompt refers to total extinction. I think this is conceptually cleanest. But it’ll trip people up as it means that most work on AI safety and alignment is about “increasing the value of futures where we survive” and not about “reducing the chance of our extinction” — which is very different than how AI takeover risk has been traditionally presented.  I.e. you could be strongly in favour of "increasing value of futures in which we survive" and by that mean that the most important thing is to prevent the extinction of Homo sapiens at the hands of superintelligence. In fact, because most work on AI safety and alignment is about “increasing the value of futures where we survive”, I expect there  won’t be that many people who properly understand the prompt and vote “yes”.

So I think  we might want to make things more fine-grained. Here are four different activities you could do (not exhaustive):

1. Ensure there’s a future for Earth-originating intelligent life at all.
2. Make human-controlled futures better.
3. Make AI-controlled futures better.
4. Make human-controlled futures more likely. 

For short, I’ll call these activities:

1. Future at all.
2. Better human futures.
3. Better AI futures.
4. More human futures.

I expect a lot more interesting disagreement over which of (1)-(4) is highest-priority than about whether (1) is higher-priority than (2)-(4).  So, when we get into debates, it might be worth saying which of (1)-(4) you think is highest-priority, rather than just “better futures vs extinction”. 

I’ve been thinking hard about whether to publicly comment more on FTX in the near term. Much for the reasons Holden gives here, and for some of the reasons given here, I’ve decided against saying any more than I’ve already said for now.

I’m still in the process of understanding what happened,  and processing the new information that comes in every day. I'm also still working through my views on how I and the EA community could and should respond.

I know this might be dissatisfying, and I’m really sorry about that, but I think it’s the right call, and will ultimately lead to a better and more helpful response. 

I can't believe it has been radio silence since this post. You owe everyone much more.