This is a quick post to talk a little bit about what I’m planning to focus on in the near and medium-term future, and to highlight that I’m currently hiring for a joint executive and research assistant position. You can read more about the role and apply here! If you’re potentially interested, hopefully the comments below can help you figure out whether you’d enjoy the role.  Recent advances in AI, combined with economic modelling (e.g. here), suggest that we might well face explosive AI-driven growth in technological capability in the next decade, where what would have been centuries of technological and intellectual progress on a business-as-usual trajectory occur over the course of just months or years. Most effort to date, from those worried by an intelligence explosion, has been on ensuring that AI systems are aligned: that they do what their designers intend them to do, at least closely enough that they don’t cause catastrophically bad outcomes.  But even if we make sufficient progress on alignment, humanity will have to make, or fail to make, many hard-to-reverse decisions with important and long-lasting consequences. I call these decisions Grand Challenges. Over the course of an explosion in technological capability, we will have to address many Grand Challenges in a short space of time including, potentially: what rights to give digital beings; how to govern the development of many new weapons of mass destruction; who gets control over an automated military; how to deal with fast-reproducing human or AI citizens; how to maintain good reasoning and decision-making even despite powerful persuasion technology and greatly-improved ability to ideologically indoctrinate others; and how to govern the race for space resources.  As a comparison, we could imagine if explosive growth had occurred in Europe in the 11th century, and that all the intellectual and technological advances that took a thousand years in our actual history occurred over the course of just a few years. It’s hard to see how decision-making would go well under those conditions. The governance of explosive growth seems to me to be of comparable importance as AI alignment, not dramatically less tractable, and is currently much more neglected. The marginal cost-effectiveness of work in this area therefore seems to be even higher than marginal work on AI alignment. It is, however, still very pre-paradigmatic: it’s hard to know what’s most important in this area, what things would be desirable to push on, or even what good research looks like. I’ll talk more about all this in my EAG: Bay Area talk, “New Frontiers in Effective Altruism.” I’m far from the only person to highlight these issues, though. For example, Holden Karnofsky has an excellent blog post on issues beyond misalignment; Lukas Finnveden has a great post on similar themes here and an extensive and in-depth series on potential projects here. More generally, I think there’s a lot of excitement about work in this broad area that isn’t yet being represented in places like the Forum. I’d be keen for more people to start learning about and thinking about these issues.  Over the last year, I’ve done a little bit of exploratory research into some of these areas; over the next six months, I plan to continue this in a focused way, with an eye toward making this a multi-year focus. In particular, I’m interested in the rights of digital beings, governance of space resources, and, above all, on the “meta” challenge of ensuring that we have good deliberative processes through the period of explosive growth. (One can think of work on the meta challenge as fleshing out somewhat realistic proposals that could take us in the direction of the “long reflection”.) By working on good deliberative processes, we could thereby improve decision-making on all the Grand Challenges we will face. This work could also help with AI safety, too: if we can guarantee power-sharing after the development of superintelligence, that decreases the incentive for competitors to race and cut corners on safety. I’m not sure yet what output this would ultimately lead to, if I decide to continue work on this beyond the next six months. Plausibly there could be many possible books, policy papers, or research institutes on these issues, and I’d be excited to help make happen whichever of these seem highest-impact after further investigation. Beyond this work, I’ll continue to provide support for individuals and organisations in EA (such as via fundraising, advice, advocacy and passing on opportunities) in an 80/20 way; most likely, I’ll just literally allocate 20% of my time to this, and spend the remaining 80% on the ethics and governance issues I list above. I expect not to be very involved with organisational decision-making (for example by being on boards of EA organisations) in the medium term, in order to stay focused and play to my comparative advantage.   I’m looking for a joint research and executive assistant to help with the work outlined above. The role involves research tasks such as providing feedback on drafts, conducting literature reviews and small research projects, as well as administrative tasks like processing emails, scheduling, and travel booking. The role could also turn into a more senior role, depending on experience and performance. Example projects that a research assistant could help with include: * A literature review on the drivers of moral progress. * A “literature review” focused on reading through LessWrong, the EA Forum, and other blogs, and finding the best work there related to the fragility of value thesis. * Case studies on: What exactly happened to result in the creation of the UN, and the precise nature of the UN Charter? What can we learn from it? Similarly for The Kyoto Protocol, the Nuclear Non-Proliferation Agreement, the Montreal Protocol. * Short original research projects, such as: * Figuring out what a good operationalisation of transformative AI would be, for the purpose of creating an early tripwire to alert the world of an imminent intelligence explosion. * Taking some particular neglected Grand Challenge, and fleshing out the reasons why this Grand Challenge might or might not be a big deal.  * Supposing that the US wanted to make an agreement to share power and respect other countries’ sovereignty in the event that it develops superintelligence, figuring out how we could legibly guarantee future compliance with that agreement, such that the commitment is credible to other countries?  The deadline for applications is February the 11th. If this seems interesting, please apply!

In this "quick take", I want to summarize some my idiosyncratic views on AI risk.  My goal here is to list just a few ideas that cause me to approach the subject differently from how I perceive most other EAs view the topic. These ideas largely push me in the direction of making me more optimistic about AI, and less likely to support heavy regulations on AI. (Note that I won't spend a lot of time justifying each of these views here. I'm mostly stating these points without lengthy justifications, in case anyone is curious. These ideas can perhaps inform why I spend significant amounts of my time pushing back against AI risk arguments. Not all of these ideas are rare, and some of them may indeed be popular among EAs.) 1. Skepticism of the treacherous turn: The treacherous turn is the idea that (1) at some point there will be a very smart unaligned AI, (2) when weak, this AI will pretend to be nice, but (3) when sufficiently strong, this AI will turn on humanity by taking over the world by surprise, and then (4) optimize the universe without constraint, which would be very bad for humans. By comparison, I find it more likely that no individual AI will ever be strong enough to take over the world, in the sense of overthrowing the world's existing institutions and governments by surprise. Instead, I broadly expect unaligned AIs will integrate into society and try to accomplish their goals by advocating for their legal rights, rather than trying to overthrow our institutions by force. Upon attaining legal personhood, unaligned AIs can utilize their legal rights to achieve their objectives, for example by getting a job and trading their labor for property, within the already-existing institutions. Because the world is not zero sum, and there are economic benefits to scale and specialization, this argument implies that unaligned AIs may well have a net-positive effect on humans, as they could trade with us, producing value in exchange for our own property and services. Note that my claim here is not that AIs will never become smarter than humans. One way of seeing how these two claims are distinguished is to compare my scenario to the case of genetically engineered humans. By assumption, if we genetically engineered humans, they would presumably eventually surpass ordinary humans in intelligence (along with social persuasion ability, and ability to deceive etc.). However, by itself, the fact that genetically engineered humans will become smarter than non-engineered humans does not imply that genetically engineered humans would try to overthrow the government. Instead, as in the case of AIs, I expect genetically engineered humans would largely try to work within existing institutions, rather than violently overthrow them. 2. AI alignment will probably be somewhat easy: The most direct and strongest current empirical evidence we have about the difficulty of AI alignment, in my view, comes from existing frontier LLMs, such as GPT-4. Having spent dozens of hours testing GPT-4's abilities and moral reasoning, I think the system is already substantially more law-abiding, thoughtful and ethical than a large fraction of humans. Most importantly, this ethical reasoning extends (in my experience) to highly unusual thought experiments that almost certainly did not appear in its training data, demonstrating a fair degree of ethical generalization, beyond mere memorization. It is conceivable that GPT-4's apparently ethical nature is fake. Perhaps GPT-4 is lying about its motives to me and in fact desires something completely different than what it professes to care about. Maybe GPT-4 merely "understands" or "predicts" human morality without actually "caring" about human morality. But while these scenarios are logically possible, they seem less plausible to me than the simple alternative explanation that alignment—like many other properties of ML models—generalizes well, in the natural way that you might similarly expect from a human. Of course, the fact that GPT-4 is easily alignable does not immediately imply that smarter-than-human AIs will be easy to align. However, I think this current evidence is still significant, and aligns well with prior theoretical arguments that alignment would be easy. In particular, I am persuaded by the argument that, because evaluation is usually easier than generation, it should be feasible to accurately evaluate whether a slightly-smarter-than-human AI is taking bad actions, allowing us to shape its rewards during training accordingly. After we've aligned a model that's merely slightly smarter than humans, we can use it to help us align even smarter AIs, and so on, plausibly implying that alignment will scale to indefinitely higher levels of intelligence, without necessarily breaking down at any physically realistic point. 3. The default social response to AI will likely be strong: One reason to support heavy regulations on AI right now is if you think the natural "default" social response to AI will lean too heavily on the side of laissez faire than optimal, i.e., by default, we will have too little regulation rather than too much. In this case, you could believe that, by advocating for regulations now, you're making it more likely that we regulate AI a bit more than we otherwise would have, pushing us closer to the optimal level of regulation. I'm quite skeptical of this argument because I think that the default response to AI (in the absence of intervention from the EA community) will already be quite strong. My view here is informed by the base rate of technologies being overregulated, which I think is quite high. In fact, it is difficult for me to name even a single technology that I think is currently clearly underregulated by society. By pushing for more regulation on AI, I think it's likely that we will overshoot and over-constrain AI relative to the optimal level. In other words, my personal bias is towards thinking that society will regulate technologies too heavily, rather than too loosely. And I don't see a strong reason to think that AI will be any different from this general historical pattern. This makes me hesitant to push for more regulation on AI, since on my view, the marginal impact of my advocacy would likely be to push us even further in the direction of "too much regulation", overshooting the optimal level by even more than what I'd expect in the absence of my advocacy. 4. I view unaligned AIs as having comparable moral value to humans: This idea was explored in one of my most recent posts. The basic idea is that, under various physicalist views of consciousness, you should expect AIs to be conscious, even if they do not share human preferences. Moreover, it seems likely that AIs — even ones that don't share human preferences — will be pretrained on human data, and therefore largely share our social and moral concepts. Since unaligned AIs will likely be both conscious and share human social and moral concepts, I don't see much reason to think of them as less "deserving" of life and liberty, from a cosmopolitan moral perspective. They will likely think similarly to the way we do across a variety of relevant axes, even if their neural structures are quite different from our own. As a consequence, I am pretty happy to incorporate unaligned AIs into the legal system and grant them some control of the future, just as I'd be happy to grant some control of the future to human children, even if they don't share my exact values. Put another way, I view (what I perceive as) the EA attempt to privilege "human values" over "AI values" as being largely arbitrary and baseless, from an impartial moral perspective. There are many humans whose values I vehemently disagree with, but I nonetheless respect their autonomy, and do not wish to deny these humans their legal rights. Likewise, even if I strongly disagreed with the values of an advanced AI, I would still see value in their preferences being satisfied for their own sake, and I would try to respect the AI's autonomy and legal rights. I don't have a lot of faith in the inherent kindness of human nature relative to a "default unaligned" AI alternative. 5. I'm not fully committed to longtermism: I think AI has an enormous potential to benefit the lives of people who currently exist. I predict that AIs can eventually substitute for human researchers, and thereby accelerate technological progress, including in medicine. In combination with my other beliefs (such as my belief that AI alignment will probably be somewhat easy), this view leads me to think that AI development will likely be net-positive for people who exist at the time of alignment. In other words, if we allow AI development, it is likely that we can use AI to reduce human mortality, and dramatically raise human well-being for the people who already exist. I think these benefits are large and important, and commensurate with the downside potential of existential risks. While a fully committed strong longtermist might scoff at the idea that curing aging might be important — as it would largely only have short-term effects, rather than long-term effects that reverberate for billions of years — by contrast, I think it's really important to try to improve the lives of people who currently exist. Many people view this perspective as a form of moral partiality that we should discard for being arbitrary. However, I think morality is itself arbitrary: it can be anything we want it to be. And I choose to value currently existing humans, to a substantial (though not overwhelming) degree. This doesn't mean I'm a fully committed near-termist. I sympathize with many of the intuitions behind longtermism. For example, if curing aging required raising the probability of human extinction by 40 percentage points, or something like that, I don't think I'd do it. But in more realistic scenarios that we are likely to actually encounter, I think it's plausibly a lot better to accelerate AI, rather than delay AI, on current margins. This view simply makes sense to me given the enormously positive effects I expect AI will likely have on the people I currently know and love, if we allow development to continue.

You can now import posts directly from Google docs Plus, internal links to headers[1] will now be mapped over correctly. To import a doc, make sure it is public or shared with "eaforum.posts@gmail.com"[2], then use the widget on the new/edit post page: Importing a doc will create a new (permanently saved) version of the post, but will not publish it, so it's safe to import updates into posts that are already published. You will need to click the "Publish Changes" button to update the live post. Everything that previously worked on copy-paste[3] will also work when importing, with the addition of internal links to headers (which only work when importing). There are still a few things that are known not to work: * Nested bullet points (these are working now) * Cropped images get uncropped * Bullet points in footnotes (these will become separate un-bulleted lines) * Blockquotes (there isn't a direct analog of this in Google docs unfortunately) There might be other issues that we don't know about. Please report any bugs or give any other feedback by replying to this quick take, you can also contact us in the usual ways. Appendix: Version history There are some minor improvements to the version history editor[4] that come along with this update: * You can load a version into the post editor without updating the live post, previously you could only hard-restore versions * The version that is live[5] on the post is shown in bold Here's what it would look like just after you import a Google doc, but before you publish the changes. Note that the latest version isn't bold, indicating that it is not showing publicly: 1. ^ Previously the link would take you back to the original doc, now it will take you to the header within the Forum post as you would expect. Internal links to bookmarks (where you link to a specific text selection) are also partially supported, although the link will only go to the paragraph the text selection is in 2. ^ Sharing with this email address means that anyone can access the contents of your doc if they have the url, because they could go to the new post page and import it. It does mean they can't access the comments at least 3. ^ I'm not sure how widespread this knowledge is, but previously the best way to copy from a Google doc was to first "Publish to the web" and then copy-paste from this published version. In particular this handles footnotes and tables, whereas pasting directly from a regular doc doesn't. The new importing feature should be equal to this publish-to-web copy-pasting, so will handle footnotes, tables, images etc. And then it additionally supports internal links 4. ^ Accessed via the "Version history" button in the post editor 5. ^ For most intents and purposes you can think of "live" as meaning "showing publicly". There is a bit of a sharp corner in this definition, in that the post as a whole can still be a draft. To spell this out: There can be many different versions of a post body, only one of these is attached to the post, this is the "live" version. This live version is what shows on the non-editing view of the post. Independently of this, the post as a whole can be a draft or published.

I met Australia's Assistant Minister for Defence last Friday. I asked him to write an email to the Minister in charge of AI, asking him to establish an AI Safety Institute. He said he would. He also seemed on board with not having fully autonomous AI weaponry. All because I sent one email asking for a meeting + had said meeting.  Advocacy might be the lowest hanging fruit in AI Safety.

Animal Justice Appreciation Note Animal Justice et al. v A.G of Ontario 2024 was recently decided and struck down large portions of Ontario's ag-gag law. A blog post is here. The suit was partially funded by ACE, which presumably means that many of the people reading this deserve partial credit for donating to support it. Thanks to Animal Justice (Andrea Gonsalves, Fredrick Schumann, Kaitlyn Mitchell, Scott Tinney), co-applicants Jessica Scott-Reid and Louise Jorgensen, and everyone who supported this work!

I'm going to be leaving 80,000 Hours and joining Charity Entrepreneurship's incubator programme this summer! The summer 2023 incubator round is focused on biosecurity and scalable global health charities and I'm really excited to see what's the best fit for me and hopefully launch a new charity. The ideas that the research team have written up look really exciting and I'm trepidatious about the challenge of being a founder but psyched for getting started. Watch this space! <3 I've been at 80,000 Hours for the last 3 years. I'm very proud of the 800+ advising calls I did and feel very privileged I got to talk to so many people and try and help them along their careers! I've learned so much during my time at 80k. And the team at 80k has been wonderful to work with - so thoughtful, committed to working out what is the right thing to do, kind, and fun - I'll for sure be sad to leave them. There are a few main reasons why I'm leaving now: 1. New career challenge - I want to try out something that stretches my skills beyond what I've done before. I think I could be a good fit for being a founder and running something big and complicated and valuable that wouldn't exist without me - I'd like to give it a try sooner rather than later. 2. Post-EA crises stepping away from EA community building a bit - Events over the last few months in EA made me re-evaluate how valuable I think the EA community and EA community building are as well as re-evaluate my personal relationship with EA. I haven't gone to the last few EAGs and switched my work away from doing advising calls for the last few months, while processing all this. I have been somewhat sad that there hasn't been more discussion and changes by now though I have been glad to see more EA leaders share things more recently (e.g. this from Ben Todd). I do still believe there are some really important ideas that EA prioritises but I'm more circumspect about some of the things I think we're not doing as well as we could (e.g. Toby's thoughts here and Holden's caution about maximising here and things I've posted about myself). Overall, I'm personally keen to take a step away from EA meta at least for a bit and try and do something that helps people where the route to impact is more direct and doesn't go via the EA community. 3. Less convinced of working on AI risk - Over the last year I've also become relatively less convinced about x-risk from AI - especially the case that agentic deceptive strategically-aware power-seeking AI is likely. I'm fairly convinced by the counterarguments e.g. this and this and I'm worried at the meta level about the quality of reasoning and discourse e.g. this. Though I'm still worried about a whole host of non-x-risk dangers from advanced AI. That makes me much more excited to work on something bio or global health related. So overall it seems like it was good to move on to something new and it took me a little while to find something I was as excited about as CE's incubator programme! I'll be at EAG London this weekend! And will hopefully you'll hear more from me later this year about the new thing I'm working on - so keep an eye out as no doubt I'll be fundraising and/or hiring at some point! :)

Hey - I’m starting to post and comment more on the Forum than I have been, and you might be wondering about whether and when I’m going to respond to questions around FTX. So here’s a short comment to explain how I’m currently thinking about things: The independent investigation commissioned by EV is still ongoing, and the firm running it strongly preferred me not to publish posts on backwards-looking topics around FTX while the investigation is still in-progress. I don’t know when it’ll be finished, or what the situation will be like for communicating on these topics even after it’s done. I had originally planned to get out a backwards-looking post early in the year, and I had been holding off on talking about other things until that was published. That post has been repeatedly delayed, and I’m not sure when it’ll be able to come out. If I’d known that it would have been delayed this long, I wouldn’t have waited on it before talking on other topics, so I’m now going to start talking more than I have been, on the Forum and elsewhere; I’m hoping I can be helpful for some of the other issues that are currently active topics of discussion. Briefly, though, and as I indicated before: I had no idea that Sam and others were misusing customer funds. Since November I’ve thought a lot about whether there were signs of this I really should have spotted, but even in hindsight I don’t think I had reason to suspect that that was happening.  Looking back, I wish I’d been far less trusting of Sam and those who’ve pleaded guilty. Looking forward, I’m going to be less likely to infer that, just because someone has sincere-seeming signals of being highly morally motivated, like being vegan or demonstrating credible plans to give away most of their wealth, they will have moral integrity in other ways, too.  I’m also more wary, now, of having such a high-trust culture within EA, especially as EA grows. This thought favours robust governance mechanisms even more than before ("trust but verify"), so that across EA we can have faith in organisations and institutions, rather than heavily relying on character judgement about the leaders of those organisations. EA has grown enormously the last few years; in many ways it feels like an adolescent, in the process of learning how to deal with its newfound role in the world. I’m grateful that we’re in a moment of opportunity for us to think more about how to improve ourselves, including both how we work and how we think and talk about effective altruism.  As part of that broader set of reflections (especially around the issue of (de)centralisation in EA), I’m making some changes to how I operate, which I describe, along with some of the other changes happening across EA, in my post on decision-making and decentralisation here. First, I plan to distance myself from the idea that I’m “the face of” or “the spokesperson for” EA; this isn’t how I think of myself, and I don’t think that description reflects reality, but I’m sometimes portrayed that way. I think moving in the direction of clarity on this will better reflect reality and be healthier for both me and the movement.  Second, I plan to step down from the board of Effective Ventures UK once it has more capacity and has recruited more trustees. I found it tough to come to this decision: I’ve been on the board of EV UK (formerly CEA) for 11 years now, and I care deeply, in a very personal way, about the projects housed under EV UK, especially CEA, 80,000 Hours, and Giving What We Can. But I think it’s for the best, and when I do step down I’ll know that EV will be in good hands.  Over the next year, I’ll continue to do learning and research on global priorities and cause prioritisation, especially in light of the astonishing (and terrifying) developments in AI over the last year. And I’ll continue to advocate for EA and related ideas: for example, in September, WWOTF will come out in paperback in the US and UK, and will come out in Spanish, German, and Finnish that month, too. Given all that’s happened in the world in the last few years — including a major pandemic, war in Europe, rapid AI advances, and an increase in extreme poverty rates — it’s more important than ever to direct people, funding and clear thinking towards the world’s most important issues. I’m excited to continue to help make that happen. 

Mildly against the Longtermism --> GCR shift Epistemic status: Pretty uncertain, somewhat rambly TL;DR replacing longtermism with GCRs might get more resources to longtermist causes, but at the expense of non-GCR longtermist interventions and broader community epistemics Over the last ~6 months I've noticed a general shift amongst EA orgs to focus less on reducing risks from AI, Bio, nukes, etc based on the logic of longtermism, and more based on Global Catastrophic Risks (GCRs) directly. Some data points on this: * Open Phil renaming it's EA Community Growth (Longtermism) Team to GCR Capacity Building * This post from Claire Zabel (OP) * Giving What We Can's new Cause Area Fund being named "Risk and Resilience," with the goal of "Reducing Global Catastrophic Risks" * Longview-GWWC's Longtermism Fund being renamed the "Emerging Challenges Fund" * Anecdotal data from conversations with people working on GCRs / X-risk / Longtermist causes My guess is these changes are (almost entirely) driven by PR concerns about longtermism. I would also guess these changes increase the number of people donation / working on GCRs, which is (by longtermist lights) a positive thing. After all, no-one wants a GCR, even if only thinking about people alive today. Yet, I can't help but feel something is off about this framing. Some concerns (no particular ordering): 1. From a longtermist (~totalist classical utilitarian) perspective, there's a huge difference between ~99% and 100% of the population dying, if humanity recovers in the former case, but not the latter. Just looking at GCRs on their own mostly misses this nuance. * (see Parfit Reasons and Persons for the full thought experiment) 2. From a longtermist (~totalist classical utilitarian) perspective, preventing a GCR doesn't differentiate between "humanity prevents GCRs and realises 1% of it's potential" and "humanity prevents GCRs realises 99% of its potential" * Preventing an extinction-level GCR might move us from 0% to 1% of future potential, but there's 99x more value in focusing on going from the "okay (1%)" to "great (100%)" future. * See Aird 2020 for more nuances on this point 3. From a longtermist (~suffering focused) perspective, reducing GCRs might be net-negative if the future is (in expectation) net-negative * E.g. if factory farming continues indefinitely, or due to increasing the chance of an S-Risk * See Melchin 2021 or DiGiovanni 2021  for more * (Note this isn't just a concern for suffering-focused ethics people) 4. From a longtermist perspective, a focus on GCRs neglects non-GCR longtermist interventions (e.g. trajectory changes, broad longtermism, patient altruism/philanthropy, global priorities research, institutional reform, ) 5. From a "current generations" perspective, reducing GCRs is probably not more cost-effective than directly improving the welfare of people / animals alive today * I'm pretty uncertain about this, but my guess is that alleviating farmed animal suffering is more welfare-increasing than e.g. working to prevent an AI catastrophe, given the latter is pretty intractable (But I haven't done the numbers) * See discussion here * If GCRs actually are more cost-effective under a "current generations" worldview, then I question why EAs would donate to global health / animal charities (since this is no longer a question of "worldview diversification", just raw cost-effectiveness) More meta points 1. From a community-building perspective, pushing people straight into GCR-oriented careers might work short-term to get resources to GCRs, but could lose the long-run benefits of EA / Longtermist ideas. I worry this might worsen community epistemics about the motivation behind working on GCRs: * If GCRs only go through on longtermist grounds, but longtermism is false, then impartial altruists should rationally switch towards current-generations opportunities. Without a grounding in cause impartiality, however, people won't actually make that switch 2. From a general virtue ethics / integrity perspective, making this change on PR / marketing reasons alone - without an underlying change in longtermist motivation - feels somewhat deceptive. * As a general rule about integrity, I think it's probably bad to sell people on doing something for reason X, when actually you want them to do it for Y, and you're not transparent about that 3. There's something fairly disorienting about the community switching so quickly from [quite aggressive] "yay longtermism!" (e.g. much hype around launch of WWOTF) to essentially disowning the word longtermism, with very little mention / admission that this happened or why

The FTX and Alameda estates have filed an adversary complaint against the FTX Foundation, SBF, Ross Rheingans-Yoo, Nick Beckstead, and some biosciences firms, available here. I should emphasize that anyone can sue over anything, and allege anything in a complaint (although I take complaints signed by Sullivan & Cromwell attorneys significantly more seriously than I take the median complaint). I would caution against drawing any adverse inferences from a defendant's silence in response to the complaint. The complaint concerns a $3.25MM "philantrophic gift" made to a biosciences firm (PLS), and almost $70MM in non-donation payments (investments, advance royalties, etc.) -- most of which were also to PLS. The only count against Beckstead relates to the donation. The non-donation payments were associated with Latona, which according to the complaint "purports to be a non-profit, limited liability company organized under the laws of the Bahamas[,] incorporated in May 2022 for the purported purpose of investing in life sciences companies [which] held itself out as being part of the FTX Foundation." The complaint does not allege that either Beckstead or Rheingans-Yoo knew of the fraud at the core of FTX and Alameda. It does, however, allege (para. 46) that the "transfers were nominally made on behalf of the  FTX Foundation and Latona, but actually were made for the benefit of Bankman-Fried." This is a weak spot in the complaint for me. There's a quote from SBF about needing to do some biosecurity work for PR and political reasons (para. 47), but corporations do charitable stuff for PR and political reasons all the time. There are quotations about wanting to de-emphasize the profit motive / potential in public communications (para. 49-50), but if the profits flowed to a non-profit it's unclear how that would personally enrich SBF. Quoting Beckstead, the complaint alleges that the investments were "ill-advised and not well-evaluated" (para. 51). It further alleges that there was no, or very little due dilligence (para. 51), such as a lack of any valuation analysis and in most cases lack of access to a data room. As a result, Latona "often paid far more than fair or reasonably equivalent value for the  investments" (para. 52). As for the $3.25MM gift, it was "was made in a similarly slapdash fashion" as the Foundation agreed to it without even knowing whether the recipient was a non-profit (para. 53). There are also allegations about how that gift came to be (para. 53-64). Paragraph 68 is troubling in terms of Latona's lack of internal controls: * On May 23, 2022, Rheingans-Yoo sent a Slack message to the Head of  Operations for FTX Ventures, asking her to wire $50 million to PLS on behalf of Latona,  because “Latona doesn’t have a bank account yet, and we’d like to move these funds as soon as possible.” After the Head of Operations inquired why they were wiring $50 million when the SAFE agreement was only for $35 million, Rheingans-Yoo said there was a “separate purchase agreement [that] had a $15mln cash advance.” When the Head of Operations asked for the purchase agreement for $15 million, Rheingans-Yoo replied, “I have an email, no formal agreement,” but “if that’s not sufficient, we can send the 35 first and get the purchase more formally papered. There apparently wasn't an attempt to formally paper the $15MM until September, and that paper was woefully vague and inadequate if the complaint is credible (para. 70). Likewise, Rheingans-Yoo allegedly offered to send over $3MM to another firm, Lumen, "on a handshake basis" without any paperwork "while we hammer out the full details" (para. 75). However, the funding was not actually sent until an agreement was signed (para. 78-79). A third investment was made on Rheingans-Yoo's recommendation despite Beckstead describing it as "unattractive" for various reasons (para. 82-84). The "donate, then invest" approach seemed to also be in play with a fourth firm called Riboscience. Paragraph 93 doesn't sound great: "On June 29, 2022, Glenn emailed Rheingans-Yoo that “sufficient time ha[d] passed since the (most generous) donation to the Glenn Labs, that we can now proceed with your desired investment in Riboscience.” Counts One to Five are similar to what I would expect most clawback complaints would look like. Note that they do not allege any misconduct by the recipients as part of the cause of action, as no such misconduct is necessary for a fraudulent conveyance action. You can also see the bankruptcy power to reach beyond the initial transferee to subsequent transferees under 11 USC 550 at play here. Most of the prefactory material is doubtless there in an attempt to cut off a defense from the defendant biosciences firms that they gave something of reasonably equivalent value in exchange for the investments. In Count Eleven, the complaint alleges that "Rheingans-Yoo knew that the transactions with the Lifesciences Defendants did not provide and had virtually no prospect of providing Alameda with reasonably equivalent value, and that Bankman-Fried personally benefited from the transactions. Rheingans-Yoo thus knowingly assisted in and/or failed to prevent Bankman-Fried’s breaches of fiduciary duty to Alameda." (para. 169). This allegedly harmed Alameda to the tune of $68.3MM. Elsewhere, the complaint alleges that " [u]pon information and belief, Bankman-Fried and Rheingans-Yoo intended to benefit personally from any profits generated by any of  these companies if they turned out to be successful and/or developed a successful product." (para. 5). However, "upon information and belief" is lawyer-speak for "we're speculating, or at least don't have a clear factual basis for this allegation yet." In Count Twelve, the complaint alleges that "Beckstead and Rheingans-Yoo knew that the transfer to PLS funded by FTX did not provide and had virtually no prospect of providing FTX with reasonably equivalent value, and that Bankman-Fried personally benefited from the transaction. Beckstead and Rheingans-Yoo thus aided and abetted Bankman-Fried’s breaches of fiduciary duty to FTX." (para. 174). This allegedly harmed FTX to the tune of $3.25MM. In the end, the complaint doesn't exactly make me think highly of anyone involved with FTX or the FTX Foundation. However, from my non-specialist eyes, I'm not seeing a slam dunk case for critical assertions about Beckstead and Rheingans-Yoo's knowledge in paras. 169 and 174.

Should we fund people for more years at a time? I've heard that various EA organisations and individuals with substantial track records still need to apply for funding one year at a time, because they either are refused longer-term funding, or they perceive they will be. For example, the LTFF page asks for applications to be "as few as possible", but clarifies that this means "established organizations once a year unless there is a significant reason for submitting multiple applications". Even the largest organisations seem to only receive OpenPhil funding every 2-4 years. For individuals, even if they are highly capable, ~12 months seems to be the norm. Offering longer (2-5 year) grants would have some obvious benefits: * Grantees spend less time writing grant applications * Evaluators spend less time reviewing grant applications * Grantees plan their activities longer-term The biggest benefit, though, I think, is that: * Grantees would have greater career security. Job security is something people value immensely. This is especially true as you get older (something I've noticed tbh), and would be even moreso for someone trying to raise kids. In the EA economy, many people get by on short-term grants and contracts, and even if they are employed, their organisation might itself not have a very steady stream of income. Overall, I would say that although EA has made significant progress in offering good salaries and great offices, the job stability is still not great. Moreover, career security is a potential blind spot for grantmakers, who generally do have ~permanent employment from a stable employer. What's more, I think that offering stable income may in many cases be cheaper than improving one's salary and office. Because some people have, for years, never been refused a grant, and would likely return any funds that turn out not to be needed. And despite the low chance of funding being "wasted", they still have to apply annually. In such cases, it seems especially clear that the time savings and talent retention benefits would outweigh any small losses.

EA forum content might be declining in quality. Here are some possible mechanisms: 1. Newer EAs have worse takes on average, because the current processes of recruitment and outreach produce a worse distribution than the old ones 2. Newer EAs are too junior to have good takes yet. It's just that the growth rate has increased so there's a higher proportion of them. 3. People who have better thoughts get hired at EA orgs and are too busy to post. There is anticorrelation between the amount of time people have to post on EA Forum and the quality of person. 4. Controversial content, rather than good content, gets the most engagement. 5. Although we want more object-level discussion, everyone can weigh in on meta/community stuff, whereas they only know about their own cause areas. Therefore community content, especially shallow criticism, gets upvoted more. There could be a similar effect for posts by well-known EA figures. 6. Contests like the criticism contest decrease average quality, because the type of person who would enter a contest to win money on average has worse takes than the type of person who has genuine deep criticism. There were 232 posts for the criticism contest, and 158 for the Cause Exploration Prizes, which combined is more top-level posts than the entire forum in any month except August 2022. 7. EA Forum is turning into a place primarily optimized for people to feel welcome and talk about EA, rather than impact. 8. All of this is exacerbated as the most careful and rational thinkers flee somewhere else, expecting that they won't get good quality engagement on EA Forum

The EA Mindset   This is an unfair caricature/ lampoon of parts of the 'EA mindset' or maybe in particular, my mindset towards EA.    Importance: Literally everything is at stake, the whole future lightcone astronomical utility suffering and happiness. Imagine the most important thing you can think of, then times that by a really large number with billions of zeros on the end. That's a fraction of a fraction of what's at stake.    Special: You are in a special time upon which the whole of everything depends. You are also one of the special chosen few who understands how important everything is. Also you understand the importance of rationality and evidence which everyone else fails to get (you even have the suspicion that some of the people within the chosen few don't actually 'really get it').    Heroic responsiblity: "You could call it heroic responsibility, maybe,” Harry Potter said. “Not like the usual sort. It means that whatever happens, no matter what, it’s always your fault. Even if you tell Professor McGonagall, she’s not responsible for what happens, you are. Following the school rules isn’t an excuse, someone else being in charge isn’t an excuse, even trying your best isn’t an excuse. There just aren’t any excuses, you’ve got to get the job done no matter what.”    Fortunately, you're in a group of chosen few. Unfortunately, there's actually only one player character, that's you, and everyone else is basically a robot. Relying on a robot is not an excuse for failing to ensure that everything ever goes well (specifically, goes in the best possible way).    Deference: The thing is though, a lot of this business seems really complicated. Like, maximising the whole of the impact universe long term happiness... where do you even start? Luckily some of the chosen few have been thinking about this for a while, and it turns out the answer is AI safety. Obviously you wouldn't trust just anyone on this, everything is at stake after all. But the chosen few have concluded this based on reason and evidence and they also drink huel like you. And someone knows someone who knows Elon Musk, and we have $10 trillion now so we can't be wrong. (You'd quite like to have some of that $10 trillion so you can stop eating supernoodles, but probably it's being used on more important stuff...) (also remember, there isn't really a 'we', everyone else is a NPC, so if it turns out the answer was actually animals and not AI safety after all, that's your fault for not doing enough independent thinking).    Position to do good: You still feel kinda confused about what's going on and how to effectively maximise everything. But the people at EA orgs seem to know what's going on and some of them go to conferences like the leaderscone forum. So if you can just get into an EA org then probably they'll let you know all the secrets and give you access to the private google docs and stuff.  Also, everyone listens to people at EA orgs, and so you'll be in a much better position to do good afterwards. You might even get to influence some of that $10 trillion dollars that everyone talks about. Maybe Elon Musk will let you have a go on one of his rockets.    Career capital: EA orgs are looking for talented, impressive ambitious, high potential, promising people. You think you might be one of those, but sometimes you have your doubts, as you sometimes fail at basic things like having enough clean clothes. If you had enough career capital, you could prove to yourself and others that you in fact did have high potential, and would get a job at an EA org. You're considering getting enough career capital by starting a gigaproject or independently solving AI safety.  This things seem kind of challenging, but you can just use self improvement to make yourself the kind of person that could do these things. 

tl;dr: In the context of interpersonal harm: 1. I think we should be more willing than we currently are to ban or softban people. 2. I think we should not assume that CEA's Community Health team "has everything covered" 3. I think more people should feel empowered to tell CEA CH about their concerns, even (especially?) if other people appear to not pay attention or do not think it's a major concern. 4. I think the community is responsible for helping the CEA CH team with having  a stronger mandate to deal with interpersonal harm, including some degree of acceptance of mistakes of overzealous moderation. (all views my own) I want to publicly register what I've said privately for a while: For people (usually but not always men) who we have considerable suspicion that they've been responsible for significant direct harm within the community, we should be significantly more willing than we currently are to take on more actions and the associated tradeoffs of limiting their ability to cause more harm in the community. Some of these actions may look pretty informal/unofficial (gossip, explicitly warning newcomers against specific people, keep an unofficial eye out for some people during parties, etc). However, in the context of a highly distributed community with many people (including newcomers) that's also embedded within a professional network, we should be willing to take more explicit and formal actions as well.  This means I broadly think we should increase our willingness to a) ban potentially harmful people from events, b) reduce grants we make to people in ways that increase harmful people's power, c) warn organizational leaders about hiring people in positions of power/contact with potentially vulnerable people.  I expect taking this seriously to involve taking on nontrivial costs. However, I think this is probably worth it. I'm not sure why my opinion here is different from others[1]', however I will try to share some generators of my opinion, in case it's helpful: A. We should aim to be a community that's empowered to do the most good. This likely entails appropriately navigating the tradeoff of both attempting to reducing the harms of a) contributors feeling or being unwelcome due to sexual harassment or other harms and b) contributors feeling or being unwelcome due to false accusations or overly zealous response. B. I think some of this is fundamentally a sensitivity vs specificity tradeoff. If we have a detection system that's too tuned to reduce the risk of false positives (wrong accusations being acted on), we will overlook too many false negatives (people being too slow to be banned/censured, or not at all), and vice versa. Consider the first section of "Difficult Tradeoffs"  Avoid false negatives: take action if there’s reason to think someone is causing problemsAvoid false positives: don’t unfairly harm someone’s reputation / ability to participate in EA  In the world we live in, I've yet to hear of a single incidence where, in full context, I strongly suspect CEA CH (or for that matter, other prominent EA organizations) was overzealous in recommending bans due to interpersonal harm. If our institutions are designed to only reduce first-order harm (both from direct interpersonal harm and from accusations), I'd expect to see people err in both directions.  Given the (apparent) lack of false positives, I broadly expect we accept too high a rate of false negatives. More precisely, I do not think CEA CH's current work on interpersonal harm will lead to a conclusion like "We've evaluated all the evidence available for the accusations against X. We currently think there's only a ~45% chance that X has actually committed such harms, but given the magnitude of the potential harm, and our inability to get further clarity with more investigation, we've pre-emptively decided to ban X from all EA Globals pending further evidence."  Instead, I get the impression that substantially more certainty is deemed necessary to take action. This differentially advantages conservatism, and increases the probability and allowance of predatory behavior. C. I expect an environment with more enforcement is more pleasant than an environment with less enforcement. I expect an environment where there's a default expectation of enforcement for interpersonal harm is more pleasant for both men and women. Most directly in reducing the first-order harm itself, but secondarily an environment where people are less "on edge" for potential violence is generally more pleasant. As a man, I at least will find it more pleasant to interact with women in a professional context if I'm not worried that they're worried I'll harm them. I expect this to be true for most men, and the loud worries online about men being worried about false accusations to be heavily exaggerated and selection-skewed[2].  Additionally, I note that I expect someone who exhibit traits like reduced empathy, willingness to push past boundaries, sociopathy, etc, to also exhibit similar traits in other domains. So someone who is harmful in (e.g.) sexual matters is likely to also be harmful in friendly and professional matters. For example, in the more prominent cases I'm aware of where people accused of sexual assault were eventually banned, they also appeared to have done other harmful activities like  a systematic history of deliberate deception, being very nasty to men, cheating on rent, harassing people online, etc. So I expect more bans to broadly be better for our community. D. I expect people who are involved in EA for longer to be systematically biased in both which harms we see, and which things are the relevant warning signals. The negative framing here is "normalization of deviance". The more neutral framing here is that people (including women) who have been around EA for longer a) may be systematically less likely to be targeted (as they have more institutional power and cachet) and b) are selection-biased to be less likely to be harmed within our community (since the people who have received the most harm are more likely to have bounced off). E. I broadly trust the judgement of CEA CH in general, and Julia Wise in particular. EDIT 2023/02: I tentatively withhold my endorsement until this allegation is cleared up. I think their judgement is broadly reasonable, and they act well within the constraints that they've been given. If I did not trust them (e.g. if I was worried that they'd pursue political vendettas in the guise of harm-reduction), I'd be significantly more worried about given them more leeway to make mistakes with banning people.[3] F. Nonetheless, the CEA CH team is just one group of individuals, and does a lot of work that's not just on interpersonal harm. We should expect them to a) only have a limited amount of information to act on, and b) for the rest of EA to need to pick up some of the slack where they've left off. For a), I think an appropriate action is for people to be significantly more willing to report issues to them, as well as make sure new members know about the existence of the CEA CH team and Julia Wise's work within it. For b), my understanding is that CEA CH sees themself as having what I call a "limited purview": e.g. they only have the authority to ban people from official CEA and maybe CEA-sponsored events, and not e.g. events hosted by local groups. So I think EA community-builders in a group organizing capacity should probably make it one of their priorities to be aware of the potential broken stairs in their community, and be willing to take decisive actions to reduce interpersonal harms.  Remember: EA is not a legal system. Our objective is to do the most good, not to wait to be absolutely certain of harm before taking steps to further limit harm. One thing my post does not cover is opportunity cost. I mostly framed things as changing the decision-boundary. However, in practice I can see how having more bans is more costly in time and maybe money than the status quo. I don't have good calculations here, however my intuition is strongly in the direction that having a safer and more cohesive is worth the relevant opportunity costs. 1. ^ fwiw my guess is that the average person in EA leadership wishes the CEA CH team does more (is currently insufficiently punitive), rather than wish that they did less (is currently overzealous). I expect there's significant variance in this opinion however. 2. ^ This is a potential crux. 3. ^ I can imagine this being a crux for people who oppose greater action. If so, I'd like to a) see this argument explicitly being presented and debated, and b) see people propose alternatives for reducing interpersonal harm that routes around CEA CH.

EA Forum discourse tracks actual stakes very poorly Examples: 1. There have been many posts about EA spending lots of money, but to my knowledge no posts about the failure to hedge crypto exposure against the crypto crash of the last year, or the failure to hedge Meta/Asana stock, or EA’s failure to produce more billion-dollar start-ups. EA spending norms seem responsible for $1m–$30m of 2022 expenses, but failures to preserve/increase EA assets seem responsible for $1b–$30b of 2022 financial losses, a ~1000x difference. 2. People are demanding transparency about the purchase of Wytham Abbey (£15m), but they’re not discussing whether it was a good idea to invest $580m in Anthropic (HT to someone else for this example). The financial difference is ~30x, the potential impact difference seems much greater still. Basically I think EA Forum discourse, Karma voting, and the inflation-adjusted overview of top posts completely fails to correctly track the importance of the ideas presented there. Karma seems to be useful to decide which comments to read, but otherwise its use seems fairly limited. (Here's a related post.)

Comments on Jacy Reese Anthis' Some Early History of EA (archived version). Summary: The piece could give the reader the impression that Jacy, Felicifia and THINK played a comparably important role to the Oxford community, Will, and Toby, which is not the case. I'll follow the chronological structure of Jacy's post, focusing first on 2008-2012, then 2012-2021. Finally, I'll discuss "founders" of EA, and sum up. 2008-2012 Jacy says that EA started as the confluence of four proto-communities: 1) SingInst/rationality, 2) Givewell/OpenPhil, 3) Felicifia, and 4) GWWC/80k (or the broader Oxford community). He also gives honorable mentions to randomistas and other Peter Singer fans. Great - so far I agree. What is important to note, however, is the contributions that these various groups made. For the first decade of EA, most key community institutions of EA came from (4) - the Oxford community, including GWWC, 80k, and CEA, and secondly from (2), although Givewell seems to me to have been more of a grantmaking entity than a community hub. Although the rationality community provided many key ideas and introduced many key individuals to EA, the institutions that it ran, such as CFAR, were mostly oriented toward its own "rationality" community.  Finally, Felicifia is discussed at greatest length in the piece, and Jacy clearly has a special affinity to it, based on his history there, as do I. He goes as far as to describe the 2008-12 period as a history of "Felicifia and other proto-EA communities". Although I would love to take credit for the development of EA in this period, I consider Felicifia to have had the third- or fourth-largest role in "founding EA" of groups on this list. I understand its role as roughly analogous to the one currently played (in 2022) by the EA Forum, as compared to those of CEA and OpenPhil: it provides a loose social scaffolding that extends to parts of the world that lack any other EA organisation. It therefore provides some interesting ideas and leads to the discovery of some interesting people, but it is not where most of the work gets done.  Jacy largely discusses the Felicifia Forum as a key component, rather than the Felicifia group-blog. However, once again, this is not quite what I would focus on. I agree that the Forum contributed a useful social-networking function to EA. However, I suspect we will find that more of the important ideas originated on Seth Baum's Felicifia group-blog and more of the big contributors started there. Overall, I think the emphasis on the blog should be at least as great as that of the forum. 2012 onwards Jacy describes how he co-founded THINK in 2012 as the first student network explicitly focused on this emergent community. What he neglects to discuss at this time is that the GWWC and 80k Hours student networks already existed, focusing on effective giving and impactful careers. He also mentions that a forum post dated to 2014 discussed the naming of CEA but fails to note that the events described in the post occurred in 2011, culminating in the name "effective altruism" being selected for that community in December 2011. So steps had already been taken toward having an "EA" moniker and an EA organisation before THINK began. Co-founders of EA To wrap things up, let's get to the question of how this history connects to the "co-founding" of EA. > Some people including me have described themselves as “co-founders” of EA. I hesitate to use this term for anyone because this has been a diverse, diffuse convergence of many communities. However, I think insofar as anyone does speak of founders or founding members, it should be acknowledged that dozens of people worked full-time on EA community-building and research since before 2012, and very few ideas in EA have been the responsibility of one or even a small number of thinkers. We should be consistent in the recognition of these contributions. There may have been more, but only three people come to mind, who have described themselves as co-founders of EA: Will, Toby, and Jacy. For Will and Toby, this makes absolute sense: they were the main ringleaders of the main group (the Oxford community) that started EA, and they founded the main institutions there. The basis for considering Jacy among the founders, however, is that he was around in the early days (as were a couple of hundred others), and that he started one of the three main student groups - the latest, and least-important among them. In my view, it's not a reasonable claim to have made. Having said that, I agree that it is good to emphasise that as the "founders" of EA, Will and Toby only did a minority - perhaps 20% - of the actual work involved in founding it. Moreover, I think there is a related, interesting question: if Will and Toby had not founded EA, would it have happened otherwise? The groundswell of interest that Jacy describes suggests to me an affirmative answer: a large group of people were already becoming increasingly interested in areas relating to applied utilitarianism, and increasingly connected with one another, via GiveWell, academic utilitarian research, Felicifia, utilitarian Facebook groups, and other mechanisms. I lean toward thinking that something like an EA movement would have happened one way or another, although it's characteristics might have been different.

Red teaming papers as an EA training exercise? I think a plausibly good training exercise for EAs wanting to be better at empirical/conceptual research is to deep dive into seminal papers/blog posts and attempt to identify all the empirical and conceptual errors in past work, especially writings by either a) other respected EAs or b) other stuff that we otherwise think of as especially important.  I'm not sure how knowledgeable you have to be to do this well, but I suspect it's approachable for smart people who finish high school, and certainly by the time they finish undergrad^ with  a decent science or social science degree. I think this is good career building for various reasons: * you can develop a healthy skepticism of the existing EA orthodoxy * I mean skepticism that's grounded in specific beliefs about why things ought to be different, rather than just vague "weirdness heuristics" or feeling like the goals of EA conflict with other tribal goals. * (I personally  have not found high-level critiques of EA, and I have read many, to be particularly interesting or insightful, but this is just a personal take). * you actually deeply understand at least one topic well enough to point out errors * For many people and personalities, critiquing a specific paper/blog post may be a less hairy "entry point" into doing EA-research adjacent work than plausible alternatives like trying to form your own deep inside views on questions that  are really open-ended and ambiguous like "come up with a novel solution in AI alignment" or "identify a new cause X" * creates legible career capital (at least within EA) * requires relatively little training/guidance from external mentors, meaning * our movement devotes less scarce mentorship resources into this * people with worse social skills/network/geographical situation don't feel (as much) at a disadvantage for getting the relevant training * you can start forming your own opinions/intuitions of both object-level and meta-level heuristics for what things are likely to be correct vs wrong. * In some cases, the errors are actually quite big, and worth correcting  (relevant parts of ) the entire EA movement on. Main "cons" I can think of: * I'm not aware of anybody successfully  doing a really good critique for the sake of doing a really good critique. The most exciting things I'm aware of (publicly, zdgroff's critique of Ng's original paper on wild animal suffering, alexrjl's critique of Giving Green. I also have private examples) mostly comes from people trying to deeply understand a thing for themselves, and then along the way spotting errors with existing work. * It's possible that doing deliberate "red-teaming" would make one predisposed to spot trivial issues rather than serious ones, or falsely identify issues where there aren't any. * Maybe critiques are a less important skill to develop than forming your own vision/research direction and executing on it, and telling people to train for this skill might actively hinder their ability to be bold & imaginative?   ^ Of course, this depends on field. I think even relatively technical papers within EA are readable to a recent undergrad who cares enough, but  this will not be true for eg (most) papers in physics or math. 

Here's a crazy idea. I haven't run it by any EAIF people yet. I want to have a program to fund people to write book reviews and post them to the EA Forum or LessWrong. (This idea came out of a conversation with a bunch of people at a retreat; I can’t remember exactly whose idea it was.) Basic structure: * Someone picks a book they want to review. * Optionally, they email me asking how on-topic I think the book is (to reduce the probability of not getting the prize later). * They write a review, and send it to me. * If it’s the kind of review I want, I give them $500 in return for them posting the review to EA Forum or LW with a “This post sponsored by the EAIF” banner at the top. (I’d also love to set up an impact purchase thing but that’s probably too complicated). * If I don’t want to give them the money, they can do whatever with the review. What books are on topic: Anything of interest to people who want to have a massive altruistic impact on the world. More specifically: * Things directly related to traditional EA topics * Things about the world more generally. Eg macrohistory, how do governments work, The Doomsday Machine, history of science (eg Asimov’s “A Short History of Chemistry”) * I think that books about self-help, productivity, or skill-building (eg management) are dubiously on topic. Goals: * I think that these book reviews might be directly useful. There are many topics where I’d love to know the basic EA-relevant takeaways, especially when combined with basic fact-checking. * It might encourage people to practice useful skills, like writing, quickly learning about new topics, and thinking through what topics would be useful to know more about. * I think it would be healthy for EA’s culture. I worry sometimes that EAs aren’t sufficiently interested in learning facts about the world that aren’t directly related to EA stuff. I think that this might be improved both by people writing these reviews and people reading them. * Conversely, sometimes I worry that rationalists are too interested in thinking about the world by introspection or weird analogies relative to learning many facts about different aspects of the world; I think book reviews would maybe be a healthier way to direct energy towards intellectual development. * It might surface some talented writers and thinkers who weren’t otherwise known to EA. * It might produce good content on the EA Forum and LW that engages intellectually curious people. Suggested elements of a book review: * One paragraph summary of the book * How compelling you found the book’s thesis, and why * The main takeaways that relate to vastly improving the world, with emphasis on the surprising ones * Optionally, epistemic spot checks * Optionally, “book adversarial collaborations”, where you actually review two different books on the same topic.

Reflection on my time as a Visiting Fellow at Rethink Priorities this summer I was a Visiting Fellow at Rethink Priorities this summer. They’re hiring right now, and I have lots of thoughts on my time there, so I figured that I’d share some. I had some misconceptions coming in, and I think I would have benefited from a post like this, so I’m guessing other people might, too. Unfortunately, I don’t have time to write anything in depth for now, so a shortform will have to do. Fair warning: this shortform is quite personal and one-sided. In particular, when I tried to think of downsides to highlight to make this post fair, few came to mind, so the post is very upsides-heavy. (Linch’s recent post has a lot more on possible negatives about working at RP.) Another disclaimer: I changed in various ways during the summer, including in terms of my preferences and priorities. I think this is good, but there’s also a good chance of some bias (I’m happy with how working at RP went because working at RP transformed me into the kind of person who’s happy with that sort of work, etc.). (See additional disclaimer at the bottom.) First, some vague background on me, in case it’s relevant: * I finished my BA this May with a double major in mathematics and comparative literature. * I had done some undergraduate math research, had taught in a variety of contexts, and had worked at Canada/USA Mathcamp, but did not have a lot of proper non-Academia work experience. * I was introduced to EA in 2019. Working at RP was not what I had expected (it seems likely that my expectations were skewed). One example of this was how my supervisor (Linch) held me accountable. Accountability existed in such a way that helped me focus on goals (“milestones”) rather than making me feel guilty about falling behind. (Perhaps I had read too much about bad workplaces and poor incentive structures, but I was quite surprised and extremely happy about this fact.) This was a really helpful transition for me from the university context, where I often had to complete large projects with less built-in support. For instance, I would have big papers due as midterms (or final exams that accounted for 40% of a course grade), and I would often procrastinate on these because they were big, hard to break down, and potentially unpleasant to work on. (I got really good at writing a 15-page draft overnight.) In contrast, at Rethink, Linch would help me break down a project into steps (“do 3 hours of reading on X subject,” “reach out to X person,” “write a rough draft of brainstormed ideas in a long list and share it for feedback,” etc.), and we would set deadlines for those. Accomplishing each milestone felt really good, and kept me motivated to continue with the project. If I was behind the schedule, he would help me reprioritize and think through the bottlenecks, and I would move forward. (Unless I’m mistaken, managers at RP had taken a management course in order to make sure that these structures worked well — I don’t know how much that helped because I can’t guess at the counterfactual, but from my point of view, they did seem quite prepared to manage us.) Another surprise: Rethink actively helped me meet many (really cool) people (both when they did things like give feedback, and through socials or 1-1’s). I went from ~10 university EA friends to ~25 people I knew I could go to for resources or help. I had not done much EA-related work before the internship (e.g. my first EA Forum post was due to RP), but I never felt judged or less respected for that. Everyone I interacted with seemed genuinely invested in helping me grow. They sent me relevant links, introduced me to cool new people, and celebrated my successes. I also learned a lot and developed entirely new interests. My supervisor was Linch, so it might be unsurprising that I became quite interested in forecasting and related topics. Beyond this, however, I found the work really exciting, and explored a variety of topics. I read a bunch of economics papers and discovered that the field was actually really interesting (this might not be a surprise to others, but it was to me!). I also got to fine-tune my understanding of and opinions on a number of questions in EA and longtermism. I developed better work (or research) habits, gained some confidence, and began to understand myself better. Here’s what I come up with when I try to think of negatives: * I struggled to some extent with the virtual setting (e.g. due to tech or internet issues). Protip: if you find yourself with a slow computer, fix that situation asap. * There might have been too much freedom for me— I probably spent too long choosing and narrowing my next project topics. Still, this wasn’t purely negative; I think I ended up learning a lot during the exploratory interludes (where I went on deep-dives into things like x-risks from great power conflict, but they did not help me produce outputs). As far as I know, this issue is less relevant for more senior positions, and a number of more concrete projects are more straightforwardly available now. (It also seems likely that I could have mitigated this by realizing it would be an issue right away.) * I would occasionally fall behind and become stressed about that. A few tasks became ugh fields. As the summer progressed, I think I got better about immediately telling Linch when I noticed myself feeling guilty or unhappy about a project, and this helped a lot. * Opportunity cost. I don’t know exactly what I would have done during the summer if not RP, but it’s always possible it would have been better. Obviously, if I were restarting the summer, I would do some things differently. I might focus on producing outputs faster. I might be more active in trying to meet people. I would probably organize my daily routine differently. But some of the things I list here are precisely changes in my preferences or priorities that result from working at RP. :) I don’t know if anyone will have questions, but feel free to ask questions if you do have any. But I should note that I might not be able to answer many, as I’m quite low on free time (I just started a new job). Note: nobody pressured me to write this shortform, although Linch & some other people at RP did know I was doing it and were happy for it. For convenience, here’s a link to RP’s hiring page.

I think it's bad when people who've been around EA for less than a year sign the GWWC pledge. I care a lot about this. I would prefer groups to strongly discourage new people from signing it. I can imagine boycotting groups that encouraged signing the GWWC pledge (though I'd probably first want to post about why I feel so strongly about this, and warn them that I was going to do so). I regret taking the pledge, and the fact that the EA community didn't discourage me from taking it is by far my biggest complaint about how the EA movement has treated me. (EDIT: TBC, I don't think anyone senior in the movement actively encouraged we to do it, but I am annoyed at them for not actively discouraging it.) (writing this short post now because I don't have time to write the full post right now)

A case of precocious policy influence, and my pitch for more research on how to get a top policy job. Last week Lina Khan was appointed as Chair of the FTC, at age 32! How did she get such an elite role? At age 11, she moved to the US from London. In 2014, she studied antitrust topics at the New America Foundation (centre-left think tank). Got a JD from Yale in 2017, and published work relevant to the emerging Hipster Antitrust movement at the same time. In 2018, she worked as a legal fellow at the FTC. In 2020, became an associate professor of law at Columbia. This year - 2021 - she was appointed by Biden. The FTC chair role is an extraordinary level of success to reach at such a young age. But it kind-of makes sense that she should be able to get such a role: she has elite academic credentials that are highly relevant for the role, has riden the hipster antitrust wave, and has experience of and willingness to work in government. I think biosec and AI policy EAs could try to emulate this. Specifically, they could try to gather some elite academic credentials, while also engaging with regulatory issues and working for regulators, or more broadly, in the executive branch of goverment. Jason Matheny's success is arguably a related example. This also suggests a possible research agenda surrounding how people get influential jobs in general. For many talented young EAs, it would be very useful to know. Similar to how Wiblin ran some numbers in 2015 on the chances at a seat in congress given a background at Yale Law, we could ask about the whitehouse, external political appointments (such as FTC commissioner) and the judiciary. Also, this ought to be quite tractable: all the names are in public, e.g. here [Trump years] and here [Obama years], most of the CVs are in the public domain - it just needs doing.