The FTX Foundation's Future Fund is a philanthropic fund making grants and investments to ambitious projects in order to improve humanity's long-term prospects.

We have a longlist of project ideas that we’d be excited to help launch. 

We’re now announcing a prize for new project ideas to add to this longlist. If you submit an idea, and we like it enough to add to the website, we’ll pay you a prize of $5,000 (or more in exceptional cases). We’ll also attribute the idea to you on the website (unless you prefer to be anonymous). 

All submissions must be received in the next week, i.e. by Monday, March 7, 2022. 

We are excited about this prize for two main reasons:

  • We would love to add great ideas to our list of projects.
  • We are excited about experimenting with prizes to jumpstart creative ideas.

To participate, you can either

  • Add your proposal as a comment to this post (one proposal per comment, please), or
  • Fill in this form

Please write your project idea in the same format as the project ideas on our website. Here’s an example:

Early detection center

Biorisk and Recovery from Catastrophes

By the time we find out about novel pathogens, they’ve already spread far and wide, as we saw with Covid-19. Earlier detection would increase the amount of time we have to respond to biothreats. Moreover, existing systems are almost exclusively focused on known pathogens—we could do a lot better by creating pathogen-agnostic systems that can detect unknown pathogens. We’d like to see a system that collects samples from wastewater or travelers, for example, and then performs a full metagenomic scan for anything that could be dangerous

You can also provide further explanation, if you think the case for including your project idea will not be obvious to us on its face.

Some rules and fine print:

  • You may submit refinements of ideas already on our website, but these might receive only a portion of the full prize.
  • At our discretion, we will award partial prizes for submissions that are proposed by multiple people, or require additional work for us to make viable.
  • At our discretion, we will award larger prizes for submissions that we really like.
  • Prizes will be awarded at the sole discretion of the Future Fund.

We’re happy to answer questions, though it might take us a few days to respond due to other programs and content we're launching right now.

We’re excited to see what you come up with!

(Thanks to Owen Cotton-Barratt for helpful discussion and feedback.)


New Comment
735 comments, sorted by Click to highlight new comments since: Today at 5:58 AM
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

Retrospective grant evaluations

Research That Can Help Us Improve

EA funders allocate over a hundred million dollars per year to longtermist causes, but a very small fraction of this money is spent evaluating past grantmaking decisions. We are excited to fund efforts to conduct retrospective evaluations to examine which of these decisions have stood the test of time. He hope that these evaluations will help us better score a grantmaker's track record and generally make grantmaking more meritocratic and, in turn, more effective. We are interested in funding evaluations not just of our own grantmaking decisions (including decisions by regrantors in our regranting program), but also of decisions made by other grantmaking organizations in the longtermist EA community.

4Avi Lewis1y
I'd like to expand on this: a think-tank/paper that formulates a way of evaluating all grants by a set of objective, quantifiable, criteria. This in order to better inform future allocation decisions so that each dollar spent ends up making the greatest impact possible. In this respect Retrospective Grant Evaluations, is but one variable to measure grant effectiveness. I have a few more ideas that can be combined to create some kind of weighted scoring mechanism for grant evaluation: * Social return on investment (SROI). Arriving at a set of non-monetary variables to quantify social impact * Cost effective analysis. GiveWell is a leader in this. We could consider applying some of their key learnings from the non-for-profit space to EA projects * Horizon Scanning. Governmental bodies have departments that perform this kind of work. A proposal could be assessed by it's alignment with emerging technology forecasts * Backcasting. Seek out ventures that are working towards a desirable future goal * Pareto optimal.  Penalize ideas that could have potential negative impact on factors/people outside of the intended target audience. * Competence and track record. Prioritize grant allocators/judges based on previous successful grants. Prioritize grants to founder or organizations with a proven track record of competence Obviously this list could go on and  this is just a small number of possible variables. The idea is simply to build a model that can score the utility of a proposed grant. 
Is this neglecting the notion that some of the grants are to strategically develop interest by presentation appealing to different decisionmakers, since the objectives are rather already known, such as improve lives of humans and animals in the long term and prevent actors, including those who use and develop AI to reduce the wellbeing of these individuals? It can be a bit of a reputational loss risk to evaluate 'well, we started convincing the government to focus on the long term by appealing by the extent of the future so now we can start talking about the quality of life in various geographies, and if this goes well then we move onto the advancement of animal-positive systems across the spacetime?'

This list should have karma hidden and entries randomised. I guess most poeple do not read and vote all the way to the bottom. I certainly didn't the first time I read it.

I agree; something like Reddit's contest mode would be useful here. I've sorted the list by "newest first" to avoid mostly seeing the most upvoted entries.

7Stephen Clare1y
I'm (pleasantly) surprised by the number of entries! But as a result the Forum seems pretty far from optimal as a platform for this discussion. Would be helpful to have a way to filter by focus area, for example.
3Nathan Young1y
Yeah I suggest it should be done like this, with search and filters as you suggest. [] 
6Taras Morozov1y
To prove the point: ATM the most upvoted comment is also the oldest one - Pablo's Retrospective grant evaluations.
The winners have been announced []. It's interesting to note the low correlation between comment karma and awards. Of the (3 out of 6) public submissions, the winners had a mean of 20 karma [as of posting this comment []], minimum 18, and the (9 out of 15) honourable mentions a mean of 39 (suggesting perhaps these were somewhat weighted "by popular demand"), minimum 16. None of the winners were in the top 75 highest rated comments; 8/9 of the publicly posted honourable mentions were (including 4 in the top 11).  There are 6 winners and 15 honourable mentions listed in OP (21 total); the top 21 public submissions had a mean karma of 52, minimum 38; the top 50 a mean of 40, minimum 28; and the top 100 a mean of 31, minimum 18. And there are 86 public submissions not amongst the awardees with higher karma than the lowest karma award winner.  See spreadsheet [] for details. Given that half of the winners were private entries (2/3 if accounting for the fact that one was only posted publicly 2 weeks after the deadline), and 40% of the honourable mentions, one explanation could be that private entries were generally higher quality. Note karma is an imperfect measure [] (so in addition to the factor Nathan mentions, maybe the discrepancy isn't that surprising).
2Nathan Young1y
Alternatively, there could be an alternate ranking mode where you get two comments shown at once and you choose if one is better if they are about the same. Even a few people doing that would start to get a sense of if they agree with the overall ranking.

Starting EA community offices

Effective altruism

Some cities, such as Boston and New York, are home to many EAs and some EA organizations, but lack dedicated EA spaces. Small offices in these cities could greatly facilitate local EA operations. Possible uses of such offices include: serving as an EA community center, hosting talks or reading groups, providing working space for small EA organizations, reducing overhead for event hosting, etc.

(Note: I believe someone actually is looking into starting such an office in Boston. I think (?) that might already be funded, but many other cities could plausibly benefit from offices of their own.)

Here is a more ambitious version:

EA Coworking Spaces at Scale

Effective Altruism

The EA community has created several great coworking spaces, but mostly in an ad hoc way, with large overheads. Instead, a standard EA office could be created in upto 100 towns and cities. Companies,  community organisers, and individuals  working full-time on EA projects would be awarded a membership that allows them to use these offices in any city. Members gain from being able to work more flexibly, in collaboration with people with similar interests (this especially helps independent researchers with motivation). EA organisations benefit from decreased need to do office management (which can be done centrally without special EA expertise). EA community organisers gain easier access to an event space and standard resources, such as a library, and hotdesking space, and some access to the expertise of others using the office.

Here is an even more ambitious one:

Found an EA charter city

Effective Altruism

A place where EAs could live, work, and research for long periods, with an EA school for their children, an EA restaurant, and so on. Houses and a city UBI could be interesting incentives.

What would be the value add of an EA city, over and above that of an EA school and coworking space? For example, I don't see why you need to eat at an EA restaurant, rather than just a regular restaurant with tasty and ethical food. Note also that the libertarian "Free State Project" [] seems to have failed, despite there being many more libertarians than effective altruists.
2mako yass1y
Lower cost of living, meaning you can have more people working on less profitable stuff. I'm not sure 5000 free staters (out of 20k signatories) should be considered failure.
Right, but it sounds like it didn't go well afterwards? []
Mere libertarians may have failed, as anarchists did in similar attempts. But I believe that EAs can do better. An EA city would be a perfect place to apply many of the ideas and polices we are currently advocating for.
Could you elaborate on the policies? And what, roughly, are you picturing - an EA-sympathising municipal government, or a more of a Honduran special economic zone type situation?
I don't think I will elaborate on policies, given that they are the last thing to worry about. Even RP negative report [] counts new policies among the benefits of charter cities. Now we are supposed to have effective ways to improve welfare, why wouldn't we build a new city, start from scratch, do it better than everybody else, and show it to the world? While I agree that this can't be done without putting a lot of thinking into it, I believe it must be done sooner or later. From a longtermist point of view: how could we ever expect to carry out a rational colonization of other planets when nobody on earth has ever been able to successfully found at least one rational city?
1mako yass1y
Note, VR is going to get really good in the next three years, so I wouldn't personally recommend getting too invested in any physical offices, but I guess as long as we're renting it won't be our problem.
4Jeff Kaufman9mo
I think it is pretty unlikely that VR improvements on the scale of 3y make people stop caring about being actually in person. This is a really hard problem that people have been working on for decades, and while we have definitely made a lot of progress if we were 3y from "who needs offices?" I would expect to already see many early adopters pushing VR as a comfortable environment for general work (VR desktop) or meetings.
1mako yass9mo
What problem are you referring to. Face tracking and remote presence didn't have a hardware platform at all until 2016, and wasn't a desirable product until maybe this year (mostly due to covid), and wont be a strongly desirable product until hardware starts to improve dramatically next year. And due to the perversity of social software economics, it wont be profitable in proportion to its impact, so it'll come late. There are currently zero non-blurry face tracking headsets with that are light enough to wear throughout a workday, so you should expect to not see anyone using VR for work. But we know that next year there will be at least one of those (apple's headset). It will appear suddenly and without any viable intermediaries. This could be a miracle of apple, but from what I can tell, it's not. Competitors will be capable of similar feats a few years later. (I expect to see limited initial impact from applevr (limited availability and reluctance from apple to open the gates), the VR office wont come all at once, even though the technical requirements will.) (You can get headsets with adequate visual acuity (60ppd) right now, but they're heavy, which makes them less convenient to use than 4k screens. They're expensive, and they require a bigger, heavier, and possibly even more expensive computer to drive them (though this was arguably partly a software problem), which also means they wont have the portability benefits that 2025's VR headsets will have, which means they're not going to be practical for much at all, and afaik the software for face tracking isn't available for them, and even if it were, it wouldn't have a sufficiently large user network in professional realms.)
2Chris Leong1y
You think they'll get past the dizziness problem?
1mako yass1y
I think everyone will adapt. I vaguely remember hearing that there might be a relatively large contingent of people who never do adapt, I was unable to confirm this with 15 minutes of looking just now, though. Every accessibility complaint I came across seemed to be a solvable software problem rather than anything fundamental.
6Chris Leong1y
I heard that New York was starting a coworking space as well
I think Berlin has something like this
Indeed, the space was organized by Effektiv Spenden: []
1Yonatan Cale1y
I think EA Israel would have more people working remotely in international organizations if we had community offices. [We recently got an office which I'm going to check out tomorrow; Not an ideal location for me but will try!]

Investment strategies for longtermist funders

Research That Can Help Us Improve, Epistemic Institutions, Economic growth

Because of their non-standard goals, longtermist funders should arguably follow investment strategies that differ from standard best practices in investing. Longtermists place unusual value on certain scenarios and may have different views of how the future is likely to play out. 

We'd be excited to see projects that make a contribution towards producing a pipeline of actionable recommendations in this regard. We think this is mostly a matter of combining a knowledge of finance with detailed views of the future for our areas of interest (i.e. forecasts for different scenarios with a focus on how giving opportunities may change and the associated financial winners/losers). There is a huge amount of room for research on these topics. Useful contributions could be made by research that develops these views of the future in a financially-relevant way, practical analysis of existing or potential financial instruments, and work to improve coordination on these topics.

Some of the ways the strategies of altruistic funders may differ include:

  • Mission-correlated investing
... (read more)

I have had a similar idea, which I didn't submit, relating to trying to create investor access to tax-deductible longtermist/patient philanthropy funds across all major EA hubs. Ideally these would be scaled up/modelled on the existing EA long term future fund (which I recall reading about but can't find now, sorry)


Edit - found it and some ideas - see this and top level post.

Just going to note that SBF/FTX/Alameda are already setting a very high benchmark when it comes to investing!
A systemic change investment strategy [] for your review.
You may be interested in the following project I'm working for: [] . The project goal is developing a new investment philosophy & strategy (complete with new outcome metrics) aimed at achieving transformational systems change. The project leverages the Deep Transitions theoretical framework as developed within the field of Sustainability Transitions and Science, Technology and Innovation Studies to create a theory of change and subsequently enact it with a group of public and private investors. Would recommend diving into this if you're interested in the nexus of investment and transformation of current systems/shaping future trajectories. I can't say too much about future plans at this stage, except that following the completion of the current phase (developing the philosophy, strategies and metrics), there will be an extended experimentation phase in which these are applied, tested and continuously redeveloped.

Highly effective enhancement of productivity, health, and wellbeing for people in high-impact roles

Effective Altruism

When it comes to enhancement of productivity, health, and wellbeing, the EA community does not sufficiently utilise division of labour. Currently, community members need to obtain the relevant knowledge themselves and do related research, e.g. on health issues, themselves. We would like to see dedicated experts on these issues that offer optimal productivity, health, and wellbeing, as a service. As a vision, a person working in a high-impact role could book calls with highly trained nutrition specialists, exercise specialists, sleep specialists, personal coaches, mental trainers, GPs with sufficient time, and so on, increasing their work output by 50% while costing little time. This could involve innovative methods such as ML-enabled optimal experiment design to figure out which interventions work for each individual.

Note: Inspired by conversations with various people. I won't name them here because I don't want to ask for permission first, but will share the prize money with them if I win something.

I was going to write a similar comment for researching and promoting well-being and well-doing improvements for EAs as well as the general public! Since this already exists in similar form as a comment, strong upvoting instead. Relevant articles include Ben Williamson’s project ( []) and Dynomight’s article on “Effective Selfishness” ( []). I also have a forthcoming article on this. Multiple project ideas that have been submitted also echo this general sentiment. For example “ Improving ventilation,” “Reducing amount of time productive people spend doing paperwork,” and “ Studying stimulants' and anti-depressants' long-term effects on productivity and health in healthy people (e.g. Modafinil, Adderall, and Wellbutrin).” Edit: I am launching this as a project called Better []! Please get in touch if you're interested in funding, collaborating on, or using this!

Reducing gain-of-function research on potentially pandemic pathogens


Lab outbreaks and other lab accidents with infectious pathogens happen regularly. When such accidents happen in labs that work on gain-of-function research (on potentially pandemic pathogens), the outcome could be catastrophic. At the same time, the usefulness of gain-of-function research seems limited; for example, none of the major technological innovations that helped us fight COVID-19 (vaccines, testing, better treatment, infectious disease modelling) was enabled by gain-of-function research. We'd like to see projects that reduce the amount of gain-of-function research done in the world, for example by targeting coordination between journals or funding bodies, or developing safer alternatives to gain-of-function research.


Additional notes:

  • There are many stakeholders In the research system (funders, journals, scientists, hosting institutions, hosting countries). I think the concentration of power is strongest in journals: there are only a few really high profile life-science journals(*). Currently, they do publish gain-of-function research. Getting high-profile journals to coordinate against publishi
... (read more)

Putting Books in Libraries

Effective Altruism

The idea of this project is to come up with a menu of ~30 books and a list of ~10000 libraries, and to offer to buy for each library, any number of books from the menu. This would ensure that folks interested in EA-related topics, who browse a library, discover these ideas. The books would be ones that teach people to use an effective altruist mindset, similar to those on this list. The libraries could be ones that are large, or that that serve top universities or cities with large English-speaking populations.

The case for the project is that if you assume that the value of discovering one new EA contributor is $200k, and that each book is read once per year (which seems plausible based on at least one random library) then the project will deliver far greater than the financial costs, of about $20 per book. The time costs would be minimised by doing much of the correspondence with libraries over the space over a short period of weeks to months. It also can serve as a useful experiment for even larger-scale book distributions, and could be replicated in other languages.

I like this idea, but I wonder - how many  people / students actually use physical libraries still? I don't think I've used one in over 15 years. My impression is that most are in chronic decline (and many have closed over the last decade).

5Cillian Crosson1y
A way around this could be to provide e-books and audio books instead of physical copies. Would also make the distribution easier. (In the UK at least, it's possible to borrow e & audio from your local library using the Libby app)
I imagine that e-book systems (text and audio) work via access to large libraries, rather than needing people to request books be added individually? So maybe there is no action needed on this front (although someone should probably check that most EA books are available in such collections).
My understanding is that individual libraries license an ebook for a number of uses or a set period of time (say, two years).
I think print books are still preferred by more readers compared to e-books. You might as well donate the books in both the physical and digital formats and probably also as an audiobook. It looks like libraries don't generally have an official way for you to donate print books virtually or to donate e-books, so I think you would have to inquire with them about whether you can make a donation and ask them to use that to buy specific books. Note that the cost of e-book licenses to libraries is many times the consumer sale price.

I really like this project idea! It's ambitious and yet approachable, and it seems that a lot of this work could be delegated to virtual personal assistants. Before starting the project, it seems that it would be valuable to quickly get a sense of how often EA books in libraries are read. For example, you could see how many copies of Doing Good Better are currently checked out, or perhaps you could nicely ask a library if they could tell you how many times a given book has been checked out.

In terms of the cost estimates, how would targeted social media advertising compare? Say targeting people who are already interested in charity and volunteering, or technology, or veg*anism, and offering to send them a free book.

Not sure, but targeted social media advertising would also be a great project.
Added [].

Never Again: A Blue-Ribbon Panel on COVID Failures

Biorisk, Epistemic Institutions

Since effective altruism came to exist as a movement, COVID was the first big test of a negative event that was clearly within our areas of concern and expertise. Despite many high-profile warnings, the world was clearly not prepared to meet the moment and did not successfully contain COVID and prevent excess deaths to the extent that should've been theoretically possible if these warnings had been properly heeded. What went wrong?

We'd like to see a project that goes into extensive detail about the global COVID response - from governments, non-profits, for-profit companies, various high-profile individuals, and the effective altruism movement - and understands what the possibilities were for policy action given what we knew at the time and where things fell apart. What could've gone better and - more importantly - how might we be better prepared for the next disaster? And rather than try to re-fight the last war, what needs to be done now for us to better handle a future disaster that may not be bio-risk at all?

Disclaimer: This is just my personal opinion and not the opinion of Rethink Priorities. This project idea was not seen by anyone else at Rethink Priorities prior to posting.

Minor note about the name: "Never Again" is a slogan often associated with the Holocaust. I think that people using it for COVID might be taken as appropriation or similar. I might suggest a different name. 

2Peter Wildeford1y
Sorry - I was not aware of this
2Ozzie Gooen1y
No worries! I assumed as such.

Are you thinking of EAs running this themselves?  We already have an informal sense of what some top priorities are for action in biosafety/pandemic-preparedness going forwards (ramp up investment in vaccines and sterilizing technology, improve PPE, try to ban Gain of Function research, etc), even if this has never been tied together into a unified and rigorously prioritized framework.

I think the idea of a blue-ribbon panel on Covid failures could have huge impact if it had (in the best-case) official buy-in from government agencies like the CDC, or (failing that) at least something like "support from a couple prestigious universities" or "participation from a pair of senators that care about the issue" or "we don't get the USA or UK but we do get a small European country like Portugal to do a Blue Ribbon Covid Panel".   In short, I think this idea might ideally look more like "lobby for the creation of an official Blue Ribbon Panel, and also try to contribute to it and influence it with EA research" rather than just running it entirely as an internal EA research project.  But maybe I am wrong and a really good, comprehensive EA report could change a lot of minds.

This is a great point. Also worth noting that there have been some retrospectives already, e.g. this one by the WHO: It would be worth considering the right balance between putting resources toward conducting an original analysis vs. mustering the political will for implementing recommendations from retrospectives like those above.
Note that CSER is running a project roughly in this direction.
An early output from this project: Research Agenda (pre-review) Lessons from COVID-19 for GCR governance: a research agenda [] The Lessons from Covid-19 Research Agenda offers a structure to study the COVID-19 pandemic and the pandemic response from a Global Catastrophic Risk (GCR) perspective. The agenda sets out the aims of our study, which is to investigate the key decisions and actions (or failures to decide or to act) that significantly altered the course of the pandemic, with the aim of improving disaster preparedness and response in the future. It also asks how we can transfer these lessons to other areas of (potential) global catastrophic risk management such as extreme climate change, radical loss of biodiversity and the governance of extreme risks posed by new technologies. Our study aims to identify key moments- ‘inflection points’- that significantly shaped the catastrophic trajectory of COVID-19. To that end this Research Agenda has identified four broad clusters where such inflection points are likely to exist: pandemic preparedness, early action, vaccines and non-pharmaceutical interventions. The aim is to drill down into each of these clusters to ascertain whether and how the course of the pandemic might have gone differently, both at the national and the global level, using counterfactual analysis. Four aspects are used to assess candidate inflection points within each cluster: 1. the information available at the time; 2. the decision-making processes used; 3. the capacity and ability to implement different courses of action, and 4. the communication of information and decisions to different publics. The Research Agenda identifies crucial questions in each cluster for all four aspects that should enable the identification of the key lessons from COVID-19 and the pandemic response.
2Sean_o_h1y []

Cognitive enhancement research and development (nootropics, devices, ...)

Values and Reflective Processes, Economic Growth

Improving people's ability to think has many positive effects on innovation, reflection, and potentially individual happiness. We'd like to see more rigorous research on nootropics, devices that improve cognitive performance, and similar fields. This could target any aspect of thinking ability---such as long/short term memory, abstract reasoning, creativity---and any stage of the research and development pipeline---from wet lab research or engineering over testing in humans to product development.


Additional notes on cognitive enhancement research:

  • Importance:
    • Sign of impact: You already seem to think that AI-based cognitive aids would be good from a longtermist perspective, so you will probably think that non-AI-based cognitive enhancement is also at least positive. (I personally think that's somewhat likely but not obvious and would love to see more analysis on it).
    • Size of impact: AI-based cognitive enhancement is probably more promising right now. But non-AI-based cognitive enhancement is still pretty promising, there is some precedent (e.g. massive benefit
... (read more)
5Jackson Wagner1y
I think this is an underrated idea, and should be considered a good refinement/addition to the FTX theme #2 of "AI-based cognitive aids".  If it's worth kickstarting AI-based research assistant tools in order to make AI safety work go better, then doesn't the same logic apply towards: * Supporting the development of brain-computer interfaces like Neuralink. * Research into potential nootropics (glad to hear you are working on replicating the creatine study!) or the negative cognitive impact of air pollution and other toxins []. * Research into tools/techniques to increase focus at work [], management best practices for research organizations, and other factors that increase productivity/motivation. * Ordinary productivity-enhancing research software like better note-taking apps, virtual reality remote collaboration tools, etc.   The idea of AI-based cognitive aids only deserves special consideration insofar as: 1. Work on AI-based tools will also contribute to AI safety research directly, but won't accelerate AI progress more generally.  (This assumption seems sketchy to me.) 2. The benefit of AI-based tools will get stronger and stronger as AI becomes more powerful, so it will be most helpful in scenarios where we need help the most.  (IMO this assumption checks out.  But this probably also applies to brain-computer interfaces, which might allow humans to interact with AI systems in a more direct and high-bandwidth way.)

Create and distribute civilizational restart manuals

A number of "existential risks" we are worried about may not directly kill off everybody, but would still cause enough deaths and chaos to make rebuilding extremely difficult. Thus, we propose that people design and distribute "civilizational restart manuals" to places that are likely to survive biological or nuclear catastrophes, giving humanity more backup options in case of extreme diasters.

The first version can be really cheap, perhaps involving storing paper copies of parts of Wikipedia plus 10 most important books sent to 100 safe and relatively uncorrelated locations -- somewhere in New Zealand, the Antarctica research base, a couple of nuclear bunkers, nuclear submarines, etc.

We are perhaps even more concerned about great moral values like concern for all sentient beings surviving and re-emerging than preserving civilization itself, so we would love for people to do further research and work into considering how to preserve cosmopolitan values as well.

My comment from another thread applies here too:

Agreed, very important in my view! I’ve been meaning to post a very similar proposal with one important addition:

Anthropogenic causes of civilizational collapse are (arguably) much more likely than natural ones. These anthropogenic causes are enabled by technology. If we preserve an unbiased sample of today’s knowledge or even if it’s the knowledge that we consider to have been most important, it may just steer the next cycle of our civilization right into the same kind of catastrophe again. If we make the information particularly durable, maybe we’ll even steer all future cycles of our civilization into the same kind of catastrophe.

The selection of the information needs to be very carefully thought out. Maybe only information on thorium reactors rather than uranium ones; only information on clear energy sources; only information on proof of stake; only information on farming low-suffering food; no prose or poetry that glorifies natural death or war; etc.

I think that is also something that none of the existing projects take into account.

Relatedly, see this post [] about continuing AI Alignment research after a GCR.
2Dawn Drescher1y
Very good!
If I may add, could this include:  - designated survivors who have some of the skills required to interpret these manuals and rebuild technologies. This could be a rotating duty, and designated survivors would live in fallout shelters []. (I;m super uncertain on this, keen on your thoughts :) )  - storing specific technological artifacts, such that scientists in the future can reverse-engineer these technologies from the artifacts. These too would be stored in such bunkers. I wonder if these could increase the odds of successful knowledge transmission, beyond written material.
Yes this sounds plausible. I'm generally excited to think about ways humanity can survive and/or flourish after civilizational collapse and other large-scale disasters.
Building on the above idea... Research the technology required to restart modern civilization and ensure the technology is understood and accessible in safe havens throughout the world A project could ensure that not only the know-how but also the technology exists dispersed in various parts of the world to enable a restart. For instance, New Zealand is often considered a relatively safe haven, but New Zealand’s economy is highly specialized and for many technologies, relies on importing technology rather than producing it indigenously. Kick-starting civilization from wikipedia could prove very slow. Physical equipment and training enabling strategic technologies important for restart could be planted in locations like New Zealand and other social contexts which are relatively safe. At an extreme, industries could be subsidized which localize technology required for a restart. This would not necessarily mean the most advanced technology; rather, it means technologies that have been important to develop to the point we are at now.  
Yes this is exciting to me, and related. Though of course generalist research talent is in short supply within EA, so the bar for any large-scale research project taking off is nontrivially high.
2Dawn Drescher1y
I didn’t write this up as a separate proposal as it seemed a bit self-serving, but creating underground cities for EAs with all the ALLFED technology and whatnot and all these backups could enable us to afterwards build a utopia with all the best voting methods and academic journals that require Bayesian analyses and publish negative results and Singer on the elementary school curriculum and universal basic income etc.
2Hauke Hillebrandt1y
All of wikipedia is just 20GB. [] Maybe there could be an way to share backups via Bittorrent or an 'offline version' of it... it would fit comfortably on most modern smartphones.
Digital solutions are not great because ideally you want something that can survive centuries or at least decades. But offline USBs in prominent + safe locations might still be a good first step anyway.
I've got a full version of the English Wikipedia, complete with images, on my phone (86GB). It's very easy to get using the Kiwix [] app.
I note there isn't much on Kiwix in terms of survival/post-apocalype collections (just a few TED talks and YouTube videos): a low-hanging fruit ripe for the picking.
Maybe someone should make an EA related collection and upload it to Kiwix? (Best books, EA Forum, AI Alignment Forum, LessWrong, SSC/ACX etc). This might be a good way of 80/20-ing preserving valuable information. As a bonus, people can easily and cheaply bury old phones with the info on, along with solar/hand-crank chargers.
The  group who discovers this restart manual could gain a huge advantage over the other  groups in the world population -- they might reach the industrial age within a few decades while everyone else is still in the stone age. This discoverer group will therefore have a huge influence over the world civilization they create. I wonder if there were a way to ensure that this group has good values, even better values than our current world. For example, imagine there were a series of value tests within the restart manual that the discoverers were required to pass in order to unlock the next stage of the manual. Either multiple groups rediscover the manual and fail until one group succeeds, or some subgroup unlocks the next step and is able to leap technologically above the others in the group fast enough to ensure that their values flourish. If those value tests somehow ensure that a high score means the test-takers care deeply about the values we want them to have, then only those who've adopted these values will rule the earth. As a side note, this would be a really cool short story or movie :)

SEP for every subject

Epistemic institutions

Create free online encyclopedias for every academic subject (or those most relevant to longtermism) written by experts and regularly updated. Despite the Stanford Encyclopedia of Philosophy being widely-known and well-loved there are few examples from other subjects. Often academic encyclopedias are both behind institutional paywalls and not accessible on sci-hub (e.g. This would provide decisionmakers and the public with better access to academic views on a variety of topics. 

5Peter S. Park1y
Can editing efforts be directed to Wikipedia? Or would this not suffice because everyone can edit it?
I've read that experts often get frustrated with wikipedia because their work ends up getting undone by non-experts. Also there probably needs to be financial support and incentives for this kind of work. 
Yeah make it accessible and normally accepted.
This would have to be a separate project from my proposed direct Wikipedia editing, but I'd  be very much in support of this (I see the efforts as being complementary)

Purchase a top journal


Journals give bad incentives to academics - they require new knowledge to be written in hard to understand language, without pre-registration at great cost and sometimes focused on unimportant topics. Taking over a top journal and ensuring it incentivised high quality work on the most important topics would begin to turn the scientific system around.

We could, of course, simply get the future fund to pay for this. There is, however, an alternative that might be worth thinking about.

This seems like the kind of thing that dominant assurance contracts are designed to solve. We could run a Kickstarter, and use the future fund to pay the early backers if we fail to reach the target amount. This should incentivise all those who want the journals bought to chip in.

Here is one way we could do this:

  1. Use a system like to identify points of consensus between universities. This should be about the rules going forward if we buy the journal. For example, do they all want pre-registration? What should the copyright situation be? How should peer-review work? How should the journal be ran? etc
  2. Whatever the consensus is, commit to implementing it if the buyout is successful
  3. Start crowdsourcing the funds needed. To maximise the chance of success, this should be done using a DAC (dominant assurance contract). This works like any other crowdfunding mechanism (GoFundMe, Kickstarter, etc), except we have a pool of money that is used to pay the early backers if we fail to meet the goal. If the standard donation size we're asking the unis for i
... (read more)
3Jonathan Nankivell1y
Update: I emailed Alex Tabarrok [] to get his thoughts on this. He originally proposed using dominant assurance contracts to solve public good problems, and he has experience testing it empirically. He makes the following points about my suggestion: * The first step is the most important. Without clarity of what the public good will be and who is expected to pay for it, the DAC won't work * You should probably focus on libraries as the potential source of funding. They are the ones who pay subscription fees, they are the ones who would benefit from this * DACs are a novel forum of social technology. It might be best to try to deliver smaller public goods first, allowing people to get more familiar, before trying to buy a journal He also suggested other ways to solve the same problem: * Have you considered starting a new journal? This should be cheaper. There would also be a coordination questions to solve to make it prestigious, but this one might be easier * Have you considered 'flipping' a journal? Could you take the editors, reviewers and community that supports an existing journal, and persuade them to start a similar but open access journal? (The Fair Open Access Alliance [] seem to have had success facilitating this. Perhaps we should support them?) My current (and weakly held) position is that flipping editorial boards to create new open access journals is the best way to improve publishing standards. Small steps towards a much better world. Would it be possible to for the Future Fund to entice 80% of the big journals to do this? The top journal in every field? Maybe.
This is a reputational loss risk of an actor in the broader EA community seeking to influence the scientific discourse by economic/peer unreviewed means? There are repositories, such as of the Legal Priorities Project [,the%20protection%20of%20future%20generations.], of papers, that are cool and the EA community pays attention to aggregate narratives to keep some of its terms rather exclusive and convincing. If you mean coordinating research, to learn from the scientific community, then it can make sense to read papers and corresponding with academics. Maybe on the EA Forum or so. No need to buy a journal.
2James Bailey1y
Agree, was thinking of submitting a proposal like this. A few ways to easily improve most journals: -Require data and code to be shared -Open access, but without the huge author fees most open access journals charge -If you do charge any fees, use them to pay reviewers for fast reviews
1Jonas Moss1y
Shouldn't reviewers be paid, regardless of fees? It is a tough job, and there should strong incentives to do it properly. 

A Longtermist Nobel Prize

All Areas

The idea is to upgrade the Future of Life Award to be more desirable. The prizemoney would be increased from $50k to$10M SEK (roughly $1.1M) per individual to match the Nobel Prizes. Both for prestige, and to make sure ideal candidates are selected, the selection procedure would be reviewed, adding extra judges or governance mechanisms as needed. This would not immediately mean that longtermism has something to match the prestige of a Nobel, but it would give a substantial reward and offer top longtermists something to strive for.

(A variation on a suggestion by DavidMoss)

How much of the prestige is the money value, how much just the age of the prize, and how much the association with a fancy institution like the Swedish monarchy?  I seem to remember that Heisenberg etc were more excited by the money than the prize, back in the day.
The money isn't necessary - see the Fields Medal. Nor is the Swedish Monarchy - see the Nobel Memorial Prize in Econ. Age obviously helps. And there's some self-reinforcement - people want the prize that others want. My guess is that money does help, but this could be further investigated.
4Hauke Hillebrandt1y
The Jacobs Foundation awards $1m prizes to scientist as a grant - I think this might be one of the biggest - one could award $5-10m to make it the most prestigious prize in the world.
1Taras Morozov1y
I think Templeton Prize has become prestigious because they give more money than the Nobel on purpose.

Megastar salaries for AI alignment work

Artificial Intelligence

Aligning future superhuman AI systems is arguably the most difficult problem currently facing humanity; and the most important. In order to solve it, we need all the help we can get from the very best and brightest. To the extent that we can identify the absolute most intelligent, most capable, and most qualified people on the planet – think Fields Medalists, Nobel Prize winners, foremost champions of intellectual competition, the most sought-after engineers – we aim to offer them salaries competitive with top sportspeople, actors and music artists to work on the problem. This is complementary to our AI alignment prizes, in that getting paid is not dependent on results. The pay is for devoting a significant amount of full time work (say a year), and maximum brainpower, to the problem; with the hope that highly promising directions in the pursuit of a full solution will be forthcoming. We will aim to provide access to top AI alignment researchers for guidance, affiliation with top-tier universities, and an exclusive retreat house and office for fellows of this program to use, if so desired.

Here []'s a more fleshed out version, FAQ style. Comments welcome.

Preventing factory farming from spreading beyond the earth

Space governance, moral circle expansion (yes I am also proposing a new area of interest.)


Early space advocates such as Gerard O’Neill and Thomas Heppenheimer had both included animal husbandry in their designs of space colonies. In our time, the European Space Agency, the Canadian Space Agency, the Beijing University of Aeronautics and Astronautics, and NASA, have all expressed interests or announced projects to employ fish or insect farming in space. 

This, if successful, might multiply the suffering of farmed animals by many times of the numbers of farmed animals on earth currently, spanned across the long-term future. Research is needed in areas like:

... (read more)

Longtermist Policy Lobbying Group

Biorisk, Recovery from Catastrophe, Epistemic Institutions, Values and Reflective Processes

Many social movements find a lot of opportunity by attempting to influence policy to achieve their goals . While longtermism can and should remain bi-partisan, there may be many opportunities to pull the rope sideways on policy areas of concern.

We'd like to see a project that attempts to carefully understand the lobbying process and explores garnering support for identified tractable policies. We think while such a project could scale to be very large once successful, anyone working on this project should really aim to start small and tred carefully, aiming to avoid issues around the unilateralist curse and ensuring to not make longtermism into an overly partisan issue. It's likely that longtermist lobbying might also be best done as lobbying for other clear areas related to longtermism but as a distinct idea, such as lobbying for climate change mitigation or lobbying for pandemic preparedness.

Disclaimer: This is just my personal opinion and not the opinion of Rethink Priorities. This project idea was not seen by anyone else at Rethink Priorities prior to posting.

I think some form of lobbying for longtermist-friendly policies would be quite valuable. However, I'm skeptical that running lobbying work through a single centralized "shop" is going to be the most efficient use of funds. Lobbying groups tend to specialize in a specific target audience, e.g., particular divisions of the US federal government or stakeholders in a particular industry, because the relationships are really important to success of initiatives and those take time to develop and maintain. My guess is that effective strategies to get desired policies implemented will depend a lot on the intersection of the target audience + substance of the policy + the existing landscape of influences on the relevant decision-makers. In practice, this would probably mean at the very least developing a lot of partnerships with colleague organizations to help get things done or perhaps more likely setting up a regranting fund of some kind to support those partners. Happy to chat about this further since we're actively working on setting something like this up at EIP.
4Peter Wildeford1y
I agree with you on the value of not overly centralizing this and of having different groups specialize in different policy areas and/or approaches.

Landscape Analysis: Longtermist Policy

Biorisk, Recovery from Catastrophe, Epistemic Institutions, Values and Reflective Processes

Many social movements find a lot of opportunity by attempting to influence policy to achieve their goals - what ought we do for longtermist policy? Longtermism can and should remain bi-partisan but there may be many opportunities to pull the rope sideways on policy areas of concern.

We'd like to see a project that attempts to collect a large number of possible longtermist policies that are tractable, explore strategies for pushing these policies, and also use public opinion polling on representative samples to understand which policies are popular. Based on this information, we could then suggest initiatives to try to push for.

Disclaimer: This is just my personal opinion and not the opinion of Rethink Priorities. This project idea was not seen by anyone else at Rethink Priorities prior to posting.

I really like this idea and think that having a global policy network could be valuable over the long term. Particularly if coordinated with other domains of EA work. For instance, I can imagine RT and various other researcher orgs and researchers providing evidence on demand to EAs who are directly embedded within policy production. 
If it shows that policies that safeguard the long-term objectives of the top lobbyists in the nation while disregarding others' preferences are the most popular, do you recommend them as attention-captivating conversation starters so that impartial consideration can be explained one-on-one to support its internalization by regulators by implementing measures to prevent the enactment of these, possible catastrophically risky (codified dystopia for some actors) popular policies, if I understand it correctly?
Hi Peter (if I may!), I love this and your other Longtermism suggestions, thanks for submitting them! Not sure if you saw my below suggestion of a Longtermism Policy Lab - but maybe this is exactly the kind of activity that could fall under such an organisation/programme (within Rethink even)? Likewise for your suggestion of a Lobbying group - by working directly with societal partners (e.g. National Ministries across the world) you could begin implementation directly through experimentation.  I've been involved in a similar (successful) project called the 'Transformative Innovation Policy Consortium (TIPC)', which works with, for example, the Colombian governement to shape innovation policy towards sustainable and just transformation (as opposed to systems optimisation).  Would love to talk to you about these your ideas for this space if you're interested. I'm working with the Institutions for Longtermism research platform at Utrecht University & we're still trying to shape our focus, so there may be some scope for piloting ideas.
JBPDavies, it sounds like you and I should connect as well -- I run the Effective Institutions Project [] and I'd love to learn more about your Institutions for Longtermism research and provide input/ideas as appropriate.
Sounds fantastic - drop me an email at [] and I would love to set up a meeting. In the meantime I'll dive into EIP's work!
2Peter Wildeford1y
Sure! Email me at [] and I will set up a meeting.

Experiments to scale mentorship and upskill people

Empowering Exceptional People, Effective Altruism

For many very important and pressing problems, especially those focused on improving the far future, there are very few experts working full-time on these problems. What's more, these fields are nascent, and there are few well-defined paths for young or early-career people to follow, it can be hard to enter the field. Experts in the field are often ideal mentors - they can vet newcomers, help them navigate the field, provide career advice, collaborate on projects and gain access to new opportunities, but there are currently very few people qualified to be mentors. We'd love to see projects that experiment with ways to improve the mentorship pipeline so that more individuals can work on pressing problems. The kinds of possible solutions possible are very broad - from developing expertise in some subset of mentorship tasks (such as vetting) in a scalable way, increasing the pool of mentors, improving existing mentors' ability to provide advice by training them, experimenting with better mentor-mentee matchmaking, running structured mentorship programs, and more.

Proportional prizes for prescient philanthropists

Effective Altruism, Economic Growth, Empowering Excetional People

A low-tech alternative to my proposal for impact markets is to offer regular, reliable prizes for early supporters of exceptionally impactful charities. These can be founders, advisors, or donors. The prizes would not only go to the top supporters but proportionally to almost anyone who can prove that they’ve contributed (or where the charity has proof of the contribution), capped only at a level where the prize money is close to the cost of the administrative overhead.

Donors may be rewarded in proportion to the aggregate size of their donations, advisors may be rewarded in proportion to their time investment valued at market rates, founders may be rewarded in proportion to the sum of both.

If these prizes are awarded reliably, maybe by several entities, they may have some of the same benefits as impact markets. Smart and altruistic donors, advisors, and charity serial entrepreneurs can accumulate more capital that they can use to support their next equally prescient project.

Reading this again, I want to register that I am much more excited about the idea of rewarding donors for early investment than I am about the other elements of the plan. As someone who has founded multiple organizations, the task of attaching precise retrospective monetary values to different people's contributions of time, connections, talent, etc. in a way that will satisfy everyone as fair sounds pretty infeasible. Early donations, by contrast, are an objective and verifiable measure of value that is much easier to reward in practice. You could just say that the first, say $500k that the org raises is eligible for retroactive reward/matching/whatever, with maybe the first $100k or something weighted more heavily. It's also worth thinking through the incentives that a system like this would set up, especially at scale. It would result in more seed funding and more small charities being founded and sustained for the first couple of years. I personally think that's a good thing at the present time, but I also know people who argue that we should be taking better advantage of economies of scale in existing organizations. There is probably a point  at which there is too much entrepreneurship, and it's worth figuring out what that point is before investing heavily in this idea.
4Dawn Drescher1y
Owen Cotton-Barrett and I have thought about this for a while and have mostly arrived at the solution that beneficiaries who collaborated on a project need to hash this out with each other. So make a contract, like in a for-profit startup, who owns how much of the impact of the project. I think that capable charity entrepreneurs are a scarce resource as well, so that we should try hard to foster them. So that’s probably where a large chunk of the impact is. When it comes to the incentive structures: We – mostly Matt Brooks and I but the rest of the team will be around – will hold a talk on the risks from perverse incentives in our system at the Funding the Commons II conference [] tomorrow. Afterwards I can also link the video recording here. My big write-up, which is more comprehensive than the presentation but unfinished, is linked from the other proposal proposal. That said … I don’t quite understand… More funding for donors -> more donors -> more money to charities -> higher scale, right? So this system would enable charities to hire more so people can specialize etc., not the opposite? Thanks!
This is really interesting. Setting up individual projects as DAOs could be an effective way to manage this.  The DAO issues tokens to founders, advisors, and donors.  If retrospectively it turns out that this was a particularly impactful project the funder can buy and burn the DAO tokens, which will drive up the price, thereby rewarding all of the holders.
2Dawn Drescher1y
Yep! There’s this other proposal for impact markets linked above. That’s basically that with slight tweaks. It’s all written in a technology-agnostic way, but one of the implementations that we’re currently looking into is on the blockchain. There’s even a bit of a prototype already. :-D
I really like this idea, and FWIW find it much more intuitive to grasp than your impact markets proposal.
2Dawn Drescher1y
Sweet, thanks! :-D Then it’ll also help me explain impact markets to people.

High quality, EA Audio Library (HEAAL)

all/meta, though I think the main value add is in AI

(Nonlinear has made a great rough/low quality version of this, so at least some credit/prize should go to them.)

Audio has several advantages over text when it comes to consuming long-form content, with one significant example being that people can consume it while doing some other task (commuting, chores, exercising) meaning the time cost of consumption is almost 0. If we think that broad, sustained engagement with key ideas is important, making the cost of engagement much lower is a clear win. Quoting Holden's recent post:

I think a highly talented, dedicated generalist could become one of the world’s 25 most broadly knowledgeable people on the subject (in the sense of understanding a number of different agendas and arguments that are out there, rather than focusing on one particular line of research), from a standing start (no background in AI, AI alignment or computer science), within a year

What does high quality mean here, and what content might get covered?

  • High quality means read by humans (I'm imagining paying maths/compsci students who'll be able to handle mathematical n

... (read more)
2Nathan Young1y
Frankly, I'd like the ability to send a written feed to somewhere and have it turned into audio, maybe crowdfunded. Clearly non-linear can do it, so why can't I have it for, say, Bryan Caplan's writing.
If you're ok with autogenerated content of roughly the quality of nonlinear, both Pocket [] and Evie [] are reasonable choices.

High-quality human performance is much more engaging than autogenerated audio, fwiw.

Hence the original pitch!
2Nathan Young1y
Non-Linear could be paid to repost the most upvoted posts but with voice actors. 

Our World in Base Rates

Epistemic Institutions

Our World In Data are excellent; they provide world-class data and analysis on a bunch of subjects. Their COVID coverage made it obvious that this is a very great public good. 

So far, they haven't included data on base rates; but from Tetlock we know that base rates are the king of judgmental forecasting (EAs generally agree). Making them easily available can thus help people think better about the future. Here's a cool corporate example. 


85% of big data projects fail”; 
10% of people refuse to be vaccinated because of fearing needles (pre-COVID so you can compare to the COVID hesitancy)”; 
"11% of ballot initiatives pass"
“7% of Emergent Ventures applications are granted”; 
“50% of applicants get 80k advice”; 
“x% of applicants get to the 3rd round of OpenPhil hiring”, "which takes y months"; 
“x% of graduates from country [y] start a business”.


  • come up with hundreds of baserates relevant to EA causes
  • scrape Wikidata for them, or
  • recurse: get people to forecast the true value, or later value (put them in a private competition on Foretold,  index them on

Later, Q... (read more)

I think this is neat. 

Perhaps-minor note: if you'd do it at scale, I imagine you'd want something more sophisticated than coarse base rates. More like, "For a project that has these parameters, our model estimates that you have a 85% chance of failure."

I of course see this as basically a bunch of estimation functions, but you get the idea.

Teaching buy-out fund

Allocate EA Researchers from Teaching Activities to Research

Problem: Professors spend a lot of their time teaching instead of researching. Many don’t know that many universities offer “teaching buy-outs”, where if you pay a certain amount of money, you don’t have to teach. Many also don’t know that a lot of EA funding would be interested in paying that.

Solution: Make a fund that's explicitly for this, to make it so more EAs know. This is the 80/20 of promoting the idea. Alternatively, funders can just advertise this offering in other ways.

Adversarial collaborations on important topics

Epistemic Institutions

There are many important topics, such as the level of risk from advanced artificial intelligence and how to reduce it, among which there are  reasonable people with very different views. We are interested in experimenting with various types of adversarial collaborations, which we define as people with opposing views working to clarify their disagreement and either resolve the disagreement or identify an experiment/observation that would resolve it. We are especially excited about combining adversarial collaborations with forecasting on any double cruxes identified from them. Some ideas for experimentation might be varying the number of participants, varying the level of moderation and strictness of enforced structure, and introducing AI-based aids.

Existing and past work relevant to this space include the Adversarial Collaboration Project, SlateStarCodex's adversarial collaboration contests, and the Late 2021 MIRI Conversations.

What topics? Which are not yet covered? (E. g. militaries already talk about peace) What adversaries? Are they rather collaborators (such as considering mergers and acquisitions and industry interest benefits for private actors and trade and alliance advantages for public actors)? Do you mean decisionmaker-nondecisionmaker collaborations - the issue is that systems are internalized, so you can get from the nondecisionmakers I want to be as powerful over others as the decisionmakers or also an inability to express or know their preferences (a chicken is in the cage so what can it say or a cricket is on the farm what do they know about their preferences) - probably, adversaries would prefer to talk about 'how can we get the other to give us profit' rather than 'how can we make impact' since the agreement is 'not impact, profit?'

Focus Groups Exploring Longtermism / Deliberative Democracy for Longtermism

Epistemic Institutions, Values and Reflective Processes

Right now longtermism is being developed within a relatively narrow set of stakeholders and participants relative to the broad set of people (and nonhumans) that would be affected by the decisions we make. We'd like to see focus groups that attempt to engage a more diverse group of people (diversity across many axes including but not limited to race, gender, age, geography, and socioeconomic status) and attempt to explain longtermism to them and explore what visions they have for the future of humanity (and nonhumans). Hopefully through many iterations we can find a way to go across what is likely rather large initial inferential distance to explore how a broader and more diverse group of people would think about longtermism once ideally informed. This can be related to and informed by engaging in deliberative democracy. This also could be helping to initiate what longtermists call "the long reflection".

Disclaimer: This is just my personal opinion and not the opinion of Rethink Priorities. This project idea was not seen by anyone else at Rethink Priorities prior to posting.

I absolutely love this idea and really hope it gets funded! It reminds me in spirit of the stakeholder research that IDinsight did [] to help inform the moral weights GiveWell uses in its cost-effectiveness analysis. At scale, it could parallel aspects of the process used to come up with the Sustainable Development Goals [].

Foundational research on the value of the long-term future

Research That Can Help Us Improve

If we successfully avoid existential catastrophe in the next century, what are the best pathways to reaching existential security, and how likely is it? How optimistic should we be about the trajectory of the long-term future? What are the worst-case scenarios, and how do we avoid them? How can we make sure the future is robustly positive and build a world where as many people are flourishing?

To elaborate on what I have in mind with this proposal, it seems important to conduct research beyond reducing existential risk over the next century – we should make sure that the future we have afterwards is good as well. I'd be interested in research following up on subjects like those of the posts:

... (read more)
This sounds great! I particularly liked that you brought up S-risks and MCE. I think these are important considerations.

Incubator for Independent Researchers

Training People to Work Independently on AI Safety

Problem: AI safety is bottlenecked by management and jobs. There are <10 orgs you can do AI safety full time at, and they are limited by the number of people they can manage and their research interests.

Solution: Make an “independent researcher incubator”. Train up people to work independently on AI safety. Match them with problems the top AI safety researchers are excited about. Connect them with advisors and teammates. Provide light-touch coaching/accountability. Provide enough funding so they can work full time or provide seed funding to establish themselves, after which they fundraise individually. Help them set up co-working or co-habitation with other researchers.

This could also be structured as a research organization instead of an incubator.

EA Marketing Agency

Improve Marketing in EA Domains at Scale

Problem: EAs aren’t good at marketing, and marketing is important.

Solution: Fund an experienced marketer who is an EA or EA-adjacent to start an EA marketing agency to help EA orgs.

Expected value calculations in practice

Invest in creating the tools to approximate expected value calculations for speculative projects, even if hard.

Currently, we can’t compare the impact of speculative interventions in a principled way. When making a decision about where to work or donate, longtermists or risk-neutral neartermists may have to choose an organization based on status, network effects, or expert opinion. This is, obviously, not ideal.

We could instead push towards having expected value calculations for more things. In the same way that GiveWell did something similar for global health and development, we could try to do something similar for longtermism/speculative projects. Longer writeup here.

AGI Early Warning System
Anonymous Fire Alarm for Spotting Red Flags in AI Safety

Problem: In a fast takeoff scenario, individuals at places like DeepMind or OpenAI may see alarming red flags but not share them because of myriad institutional/political reasons.

Solution: create an anonymous form - a “fire alarm” (like an whistleblowing Andon Cord of sorts) where these employees can report what they’re seeing. We could restrict the audience to a small council of AI safety leaders, who then can determine next steps. This could, in theory, provide days to months of additional response time.

Alignment Forum Writers

Pay Top Alignment Forum Contributors to Work Full Time on AI Safety

Problem: Some of AF’s top contributors don’t actually work full-time on AI safety because they have a day job to pay the bills.

Solution: Offer them enough money to quit their job and work on AI safety full time.

(Per Nick's note, reposting)

Political fellowships

Values and Reflective Processes, Empowering Exceptional People

We’re like to fund ways to pull people who don’t run for political office to run for political office. It's like a MacArthur. You get a call one day. You've been selected. You'd make a great public servant, even if you don't know it. You'd get some training, like DCCC and NRCC, and when you run, you get two million spent by the super-PAC run by the best. They've done the analysis. They'll provide funding. They've lined up endorsers. You've never thought about politics, but they've got your back. Say what you want to say, make a difference in the world: run the campaign you don't mind losing. And if you win, make it real.

Great idea, at TFG we have similar thoughts and are currently researching if we should run it and the best way to run a program like this. Would love to get input from people on this.

The Billionaire Nice List


A regularly updated list of how much impact we estimate billionaires have created. Billionaires care about their public image, people like checking lists. Let's attempt to create a list which can be sorted by different moral weights and incentivises billionaires to do more good. 

I really like this. I had a similar idea focused on trying to change the incentive landscape for billionaires to make it as high status as possible to be as high impact as possible. I think that lists and awards could be a good start. Would be especially good to have the involvement of some aligned ultrawealthy people who might have a good understanding of what will be effective.
3Nathan Young1y
Yeah, I would love those of us who know or are billionaires to give a sense of what motivates them.

Pro-immigration advocacy outside the United States

Economic Growth

Increasing migration to rich countries could dramatically reduce poverty and grow the world economy by up to 150%. Open Philanthropy has long had pro-immigration reform in the U.S. as a focus area, but the American political climate has been very hostile to and/or polarized on immigration, making it harder to make progress in the U.S. However, other high-income countries might be more receptive to increasing immigration, and would thus be easier places to make progress. For example, according to a 2018 Pew survey, 81% of Japanese citizens support increasing or keeping immigration levels about the same. It would be worth exploring which developed countries are most promising for pro-immigration advocacy, and then advocating for immigration there.

What this project could look like:

  1. Identify 2-5 developed countries where pro-immigration advocacy seems especially promising.
  2. Build partnerships with people and orgs in these countries with expertise in pro-immigration advocacy.
  3. Identify the most promising opportunities to increase immigration to these countries and act on them.

Related posts:

... (read more)
Japan is coming from a very low base - 2% of population is foreign-born - vs. 15% in the US []. A lot of room for more immigrants before "saturation" is reached I guess. Although I imagine that xenophobia and racism is anti-correlated with immigration, at least at low levels [citation needed].
Top countries by refugees per capita [] The world's most neglected displacement crises [] Should these countries be supported in their efforts (I read I think $0.1/person/day for food) and the crises prevented such as by supporting the source area parties to make and abide by legal agreements over resources, prevent drug trade by higher-yield farming practices and education or urban career growth prospects, improve curricula to add skills development in care for others (teaching preventive healthcare and others' preferences-based interactions), etc - as a possibly cost-effective alternative to pro-immigration advocacy - then, either privileged persons will be able to escape the poor situation, which will not be solved or unskilled persons with poor norms will be present at places which may not improve their subjective wellbeing, which is given by the norms' internalization?
Your question is very long and hard to understand. Can you please reword it in plain English?
Displacement crises are large and neglected. For example, for one of the top 10 crises, 6,000 additional persons are displaced per day. Displaced persons can be supported by very low amounts, which make large differences. For example, $0.1/day for food and low amount for healthcare. In some cases, this would have otherwise not been provided. So, supporting persons in crises in emerging economies, without solving the issues, can be cost-effective compared to  spending comparable effort on immigration reform. Second, supporting countries that already host refugees of neglected crises to better accommodate these persons (so that they do not need to stay in refugee camps reliant on food aid and healthcare aid), for example, by special economic zones, if these allow for savings accumulation, and education, so that refugees can better integrate and the public welcomes it due to economic benefits, can be also competitive in cost-effectiveness compared to immigration reform in countries with high public attention and political controversy and much smaller refugee populations, such as the US. The intervention is more affordable, makes larger difference for the intended beneficiaries, has higher chance of political support, and can be institutionalized while solving the problem. Third, allocating comparable skills to neglected crises rather than to immigration reform in industrialized nations where unit decisionmaker's attention can be much more costly, such as the US, can resolve the causes of these crises, which can include limited ability to draft and enforce legal agreements around natural resources or mitigate violence related to limited alternative prospects of drug farmers by sharing economic alternatives, such as higher-yield commodity farming practices, agricultural value addition skills, or upskilling systems related to work in urban areas. So, the cost-effectiveness of solving neglected crises by legal, political, and humanitarian assistance can be much higher th

Improving ventilation


Ventilation emerged as a potential intervention to reduce the risk of COVID and other pathogens. Additionally, poor air quality is a health concern in its own right, negatively affecting cognition and cognitive development. Despite this, there still does not seem to be commonly accepted wisdom about what kind of ventilation interventions ought to be pursued in offices, bedrooms, and other locations.

We'd like to see a project that does rigorous research to establish strong ventilation strategies in a variety of contexts and explores their effectiveness on various ventilation issues. Once successful ventilation strategies are developed, assuming it would be cost-effective to do so, this project could then aim to roll out ventilation and campaign/market for ventilation interventions either as a for-profit, non-profit, or hybrid.

Disclaimer: This is just my personal opinion and not the opinion of Rethink Priorities. This project idea was not seen by anyone else at Rethink Priorities prior to posting.

Advocacy organization for unduly unpopular technologies

Public opinion on key technologies.

Some technologies have enormous benefits, but they are not deployed very much because they are unpopular. Nuclear energy could be a powerful tool for enhancing access to clean energy and combating climate change, but it faces public opposition in Western countries. Similarly, GMOs could help solve the puzzle of feeding the global population with fewer resources, but public opinion is largely against them. Cellular agriculture may soon face similar challenges. Public opinion on these technologies must urgently be shifted. We’d like to see NGOs that create the necessary support via institutions and the media, without falling into the trap of partisan warfare with traditional environmentalists.

4Jackson Wagner1y
Probably want to avoid unifying all of these under one "we advocate for things that most people hate" advocacy group!  Although that would be pretty hilarious.  But funding lots of little different groups in some of these key areas is great, such as trying to make it easier to build clean energy projects of all kinds as I mention here [].
Right, it sounds absurd and maybe hilarious, but it's actually what I had in mind. The advantage is internal coherence. The idea is basically to let "ecomodernism" go mainstream, having a Greenpeace-like org that has ideas more similar to the Breakthrough Institute.  It's far from clear that this can work, but it's worth a try, in my view. About your suggestion: I love it and voted for it. 
2Jackson Wagner1y
Maybe so... like an economics version of the ACLU that builds a reputation of sticking up for things that are good even though they're unpopular. Might work especially well if oriented around the legal system (where ACLU operates and where groups like Greenpeace and the ever-controversial NRA have had lots of success), rather than purely advocacy? Having a unified brand might help convince people that our side has a point. For instance, a group that litigates to fight against nimbyism by complaining about the overuse of environmental laws or zoning regulations... the nimbys would naturally see themselves as the heroes of the story and assume that lawyers on the pro-construction side were probably villains funded by big greedy developers. Seeing that their opposition was a semi-respected ACLU-like brand that fought for a variety of causes might help change people's minds on an issue. (On the other hand, I feel like the legal system is fundamentally friendlier terrain for stopping projects than encouraging them, so the legal angle might not work well for GMOs and power plants. But maybe there are areas like trying to ban Gain-of-Function research where this could be a helpful strategy.) We'd still probably want the brand of this group to be pretty far disconnected from EA -- groups like Greenpeace, the NRA, etc naturally attract a lot of controversy and demonization.
2Andreas F1y
Since Lifecycle Analysis show that it most likely is the best option, I fully agree on the nuclear Part.  I also agree on the GMO part, since huge Meta Analysis show no adverse effects on the environment (compared as yield/area   & Biodiversity/dollar & Yield/dollar  labor/ yield), in comparison with other agriculture. I have no Assessment on Cellular Agriculture, but I do think, that it is fair to support such schemes ( at least until we have solid data regarding this, and then decide again).  
2Peter S. Park1y
Note: Wanted to share an example. I think that while nuclear fission reactors are unpopular and this unpopularity is sticky, it is possible that efforts to preemptively decouple the reputation of nuclear fusion reactors with those of nuclear fission reactors can succeed (and that nuclear fusion's hypothetical positive reputation can be sticky over time). But it is also possible that the unpopularity of nuclear fission will stick to nuclear fusion.  Which of these two possibilities occurs, and how proactive action can change this, is mysterious at the moment. This is because our causal/theoretical understanding of the science of human behavior is incomplete. (see my submission, "Causal microfoundations for behavioral science") Preemptive action regarding historically unprecendented settings like emergent technologies---for which much of the relevant data may not yet exist---can be substantially informed by externally valid predictions of people's situation-specific behavior in such settings.
Interesting thought. FWIW, I think it's more realistic that we can turn around public opinion on fission first, reap more of the benefits of fission, and then have a better public public landscape for fusion, then that we accept the unpopularity of fission as a given but will have somehow popular fusion. But I may well be wrong.

Building the grantmaker pipeline

Empowering Exceptional People, Effective Altruism

The amount of funding committed to Effective Altruism has grown dramatically in the past few years, with an estimated  $46 billion dollars currently earmarked for EA. With this significant increase in available funding, there is now a greatly increased need for talented and thoughtful grantmakers, who can effectively deploy this money. It's plausible that yearly EA grantmaking could increase by a factor of 5-10x over the coming decade, and this requires finding and training new grantmakers on best practices, as well as developing sound judgement.  We'd love to see projects that build the grantmaker pipeline, whether that's grantmaking fellowships, grantmaker mentoring, more frequent donor lotteries, more EA funds-style organisations with rotating fund managers, and more. 

NB: This might be a refinement of fellowships, but I think it's particularly important.

7Jackson Wagner1y
This is such a good idea that I think FTX is already piloting a regranting scheme as a major prong of their Future Fund program []! But it would be cool to build up the pipeline in other more general/systematic ways -- maybe with mentorship/fellowships, maybe with more experimental donation designs like donor lotteries and impact certificates [], maybe with software that helps people to make EA-style impact estimates [].
4Cillian Crosson1y
It seems that FTX's Regranting Program could be a great way to scalably distribute funds & build the grantmaker pipeline. We (Training for Good []) are also developing a grantmaker training programme like what James has described here to help build up EA's grantmaking capacity (which could complement FTX's Regranting Program nicely). It will likely be an 8 week, part-time programme, with a small pot of "regranting" money for each participant and we're pretty excited to launch this in the next few months. In the meantime, we're looking for 5-10 people to beta test a scaled-down version of this programme (starting at the end of March). The time commitment for this beta test would be ~5 hours per week (~2 hrs reading, ~2 hrs projects, ~1 hr group discussion). If anyone reading this is interested, feel free to shoot me an email [] 

Top ML researchers to AI safety researchers

Pay top ML researchers to switch to AI safety

Problem: <.001% of the world’s brightest minds are working on AI safety. Many are working on AI capabilities.

Solution: Pay them to switch. Pay them their same salary, or more, or maybe a lot more.

EA Productivity Fund

Increase the output of top longtermists by paying for things like coaching, therapy, personal assistants, and more.

Problem: Longtermism is severely talent constrained. Yet, even though these services could easily increase a top EAs productivity by 10-50%, many can’t afford them or would be put off by the cost (imposter syndrome or just because it feels selfish).

Solution: Create a lightly-administered fund to pay for them. It’s unclear what the best way would be to select who gets funding, but a very simple decision metric could be to give it to anybody who gets funding from Open Phil, LTFF, SFF, or FTX. This would leverage other people’s existing vetting work.

Automated Open Project Ideas Board

 The Future Fund

All of these ideas should be submitted to a board where anyone can forecast their value in dollars lives saved per $ as rated by a trusted research organisation, say Rethink Priorities. The forecasts can be reputation or prediction markets.  Then that research organisation  checks 1% of the ideas and scores them. These scores are used to weight the other forecasts. This creates a scalable system for ranking ideas. Then funders can donate to them as they see fit.

Massive US-China exchange programme

Great power conflict, AI

Fund (university) students to live in the other country in a host family: between US-China, Russia-US, China-India, potentially India-Pakistan. This is important if one thinks that personal experience make it less likely that individuals incentivise or encourage escalation, war and certain competitive dynamics.

8Jackson Wagner1y
This might have a hard time meeting the same effectiveness bar as #13, "Talent Search" and #17, "Advocacy for US High-Skill Immigration", which might end up having some similar effects but seem like more leveraged interventions.
I disagree, as this idea seems much more explicitly targeted at reducing the potential for great power conflict, and I haven't yet seen many other tractable ideas in that domain.
5Alex D1y
My understanding is the Erasmus Programme [] was explicitly started in part to reduce the chance of conflict between European states.

Nuclear/Great Power Conflict Movement Building

Effective Altruism

Given the current situation in Ukraine, movement-building related to nuclear x-risk or great power conflict would likely be much more tractable than it was up until recently. We don't know how long this period will last for and the memory of the public can be short, so we intend to advantage of this opportunity. This outreach should focus on people with an interest in policy or potential student group organisers as these people are most likely to have an influence here.

(Per Nick's note, reposting)

Market shaping and advanced market commitments

Epistemic institutions; Economic Growth

Market shaping is when an idea can only be jump-started by committed demand or other forces. Operation Warp Speed is the most recent example of market-shaping through advanced market commitments, but it has been used several times for other vaccine development. We are interested in funding work to understand when market-shaping makes sense, ideas for creating and funding market-shaping methods, and specific market-shaping or advanced market commitments in our areas of interest.

(I drafted this then realized that it is largely the same as Zac's comment above - so I've strong upvoted that comment and I'm posting here in case my take on it is useful.)

Crowding in other funding

We're excited to see ideas for structuring projects in our areas of interest that leverage our funds by aligning with the tastes of other funders and investors. While we are excited about spending billions of dollars on the best projects we can find, we're also excited to include other funders and investors in the journey of helping these projects scale in the best way possible. We would like to maximize the chance that other sources of funding come in. Some projects are inherently widely attractive and some others are only ever likely to attract (or want) longtermist funding. But, we expect that there are many projects where one or more general mechanisms can be applied to crowd in other funding. This may include:

  • Offering financial incentives (e.g. advanced market commitments)
  • Highlighting financial potential in major projects we would like to see (e.g. especially projects of the scale of the Grok / Brookfield bid for AGL)
  • Portfolio structures / financial engineering (e.g. Bridge Bio)
  • Appealing to social preferences (e.g. highlight points of 'common sense' overlap between longtermist views and ESG)
I'll add that advanced market commitments are also useful in situations where a jump-start isn't explicitly required.  In that case, they can act similarly to prize based funding

An Organisation that Sells its Impact for Profit

Empowering Exceptional People, Epistemic Institutions

Nonprofits are inefficient in some respects: they don't maximize value for anyone the way for-profits do for their customers. Moreover, they lack market valuations, so successful nonprofits scale too slowly while unsuccessful ones linger too long. One way to address this is to start an organisation that only accepts funding that incentivizes impact. Its revenue would come from: (1) Selling Impact Cerificates, (2) Prizes, and/or (3) Grants (but only if they value the work at a similar level to the impact certificates). Such an organization could operate on an entirely for-profit basis. Funding would be raised from for-profit investors. Staff would be paid in salary plus equity. The main premise here is that increased salaries are a small price to pay for the efficiencies that can be gained from for-profit markets. Of course, this can only succeed if the funding mechanisms (1-3) become sufficiently popular, but given the increased funding in longtermist circles, this now looks increasingly likely.

See also Retrospective grant evaluations,  Retroactive public goods funding, Impact ... (read more)

Rationalism But For Group Psychology

Epistemic Institutions

LessWrong and the rationalist community have done well to highlight biases and help individuals become more rational, as well as creating a community around this. But most of the biggest things in life are done by groups and organizations.

We'd like to see a project that takes group psychology / organizational psychology and turns it into a rationalist movement with actionable advice to help groups be less biased and help groups achieve more impact, like how the original rationalist movement did so with individuals. We imagine this would involve identifying useful ideas from group psychology / organizational psychology literature and popularizing them in the rationalist community, as well as trying to intentionally experiment. Perhaps this could come up with better ideas for meetings, how to hire, how to attract talent, better ways to help align employees with organizational goals, better ways to keep track of projects, etc.

Disclaimer: This is just my personal opinion and not the opinion of Rethink Priorities. This project idea was not seen by anyone else at Rethink Priorities prior to posting.

The Epistea Summer Experiment [] was a glorious example of this.  
1[comment deleted]1y

Wild animal suffering in space

Space governance, moral circle expansion.


Terraforming other planets might cause animals to come to exist in these planets, either because of intentional or unintentional behaviors. These animals might live net negative lives. 

Also, we cannot rule out the possibility that there are already wild "animals" (or any form of sentient beings) who might be suffering from net negative lives in other planets. (this does not relate directly to the Fermi Paradox, which is highly intelligent lives, not lives per se)

Relevant research include:

  • Whether wild animals lead net negative or positive lives on earth, under what conditions. And whether this might hold the same in different planets.
  • Tracking, or even doing research on using AI and robotics to monitor and intervene with habitats. This might be critical if there planets there are planets that has wild "animals", but are uninhabitable for humans to stay close and monitor (or even intervene with) the welfare of these animals.
  • Communication strategies related to wild animal welfare, as it seem to tend to cause controversy, if not outrage.
  • Philosophical research, including population ethics, environmental ethics, comparing welfare/suffering between species, moral uncertainty, suffering-focused vs non-suffering focused ethics.
  • General philosophical work on the ethics of space governance, in relation to nonhuman animals.
6Dawn Drescher1y
Another great concern of mine is that even if biological humans are completely replaced with ems or de novo artificial intelligence, these processes will probably run on great server farms that likely produce heat and need cooling. That results in a temperature gradient that might make it possible for small sentient beings, such as invertebrates, to live there. Their conditions may be bad, they may be r-strategists and suffer in great proportions, and they may also be numerous if these AI server farms spread throughout the whole light cone of the future. My intuition is that very few people (maybe Simon Eckerström Liedholm?) have thought about this so far, so maybe there are easy interventions to make that less likely to happen.
5Dawn Drescher1y
Brian Tomasik [] and Michael Dello-Iacovo [] have related articles.
Here's a related question [] I asked.

AI alignment prize suggestion: Introduce AI Safety concepts into the ML community

Artificial Intelligence

Recently, there have been several papers published at top ML conferences that introduced concepts from the AI safety community into the broader ML community. Such papers often define a problem, explain why it matters, sometimes formalise it, often include extensive experiments to showcase the problem, sometimes include some initial suggestions for remedies. Such papers are useful in several ways: they popularise AI alignment concepts, pave the way for further research,  and demonstrate that researchers can do alignment research while also publishing in top venues. A great example would be Optimal Policies Tend To Seek Power, published in NeurIPS. Future Fund could advertise prizes for any paper that gets published in a top ML/NLP/Computer Vision conference (from ML, that would be NeurIPS, ICML, and ICLR) and introduces a key concept of AI alignment.

2Yonatan Cale1y
Risk:  The course presents possible solutions to these risks, and the students feel like they "understood" AI risk, and in the future it will be harder to these students about AI risk since they feel like they already have an understanding, even though it is wrong. I am specifically worried about this because I try imagining who would write the course and who would teach it. Will these people be able to point out the problems in the current approaches to alignment? Will these people be able to "hold an argument" in class well enough to point out holes in the solutions that the students will suggest after thinking about the problem for five minutes? I'm not saying this isn't solvable, just a risk.

EA Macrostrategy:

Effective Altruism

Many people write about the general strategy that EA should take, but almost no-one outside of CEA has this as their main focus. Macrostrategy involves understanding all of the different organisations and projects in EA, how they work together, what the gaps are and the ways in which EA could fail to achieve its goals. Some resources should be spent here as an exploratory grant to see what this turns up.

Evaluating large foundations

Effective Altruism

Givewell looks at actors: object-level charities, people who do stuff. But logically, it's even more worth scrutinising megadonors (assuming that they care about impact or public opinion about their operations, and thus that our analysis could actually have some effect on them).

For instance, we've seen claims that the Global Fund, who spend $4B per year, meet a 2x GiveDirectly bar but not a Givewell Top Charity bar.

This matters because most charity - and even most good charity - is still not by EAs or run on EA lines. Also, even big cautious foundations can risk waste / harm, as arguably happened with the Gates Foundation and IHME - it's important to understand the base rate of conservative giving failing, so that we can compare hits-based giving. And you only have to persuade a couple of people in a foundation before you're redirecting massive amounts.

Refining EA communications and messaging

Values and Reflective Processes, Research That Can Help Us Improve

If we want to motivate a broad spectrum of people about the importance of doing good and ensuring the long-term goes well, it's imperative we find out which messages are "sticky" and which ones are forgotten quickly. Testing various communication frames, particularly for key target audiences like highly talented students, will support EA outreach projects in better tailoring their messaging. Better communications could hugely increase the number of people that consume EA content, relate to the values of the EA movement, and ultimately commit their life to doing good. We'd be excited to see people testing various frames and messaging, across a range of target audiences, using methodologies such as surveys, focus groups, digital media, and more.

1Jack Lewars1y
I think this exists [] (but could be much bigger and should still be funded by this fund).

TL;DR: EA Retroactive Public Good's Funding

In your format:

Deciding which projects to fund is hard, and one of the reasons for that is that it's hard to guess which projects will succeed and which will fail. But wait, startups have solved this problem perfectly: Anybody is allowed to vet a startup and decide to invest (bet) their money on this startup succeeding, and if the startup does succeed, then the early investors get a big financial return.

The EA community could do the same, only it is missing the part where we give big financial returns to projects that turned out good.

This would make the fund's job much easier: They would have to vet which project helped IN RETROSPECT, which is much easier, and they'll leave the hard prediction work to the market.

Context for proposing this

I heard of a promising EA project that is for some reason having trouble raising funds. I'm considering funding it myself, though I am not rich and that would be somewhat broken to do. But I AM rich enough to fund this project and bet on it working well enough to get a Retroactive Public Good grant in the future, if such a thing existed. I also might have some advantage over the EA Fund in vetting this project.

In Vitalik's words:

2Ben Dean1y
Related: Impact Certificates []

EA Forum Writers

Pay top EA Forum contributors to write about EA topics full time

Problem: Some of the EA Forum’s top writers don’t work on EA, but contribute some of the community’s most important ideas via writing.

Solution: Pay them to write about EA ideas full time. This could be combined with the independent researcher incubator quite well.

5Nathan Young1y
Pay users based on post karma.  (but not comment or question karma which are really easy to get in comparison)
could lead to disincentive to post more controversial ideas there though
2Chris Leong1y
Goodharts law
2Nathan Young1y
don't think we'd be wedded to a single metric. Also isn't karma already weak to goodhearts law? I think we should already be concerned with this.
2Nathan Young1y
I don't think we'd be wedded to this metric

A “Red Team” to rigorously explore possible futures and advocate against interventions that threaten to backfire

Research That Can Help Us Improve, Effective Altruism, Epistemic Institutions, Values and Reflective Processes

Motivation. There are a lot of proposals here. There are additional proposals on the Future Fund website. There are additional proposals also on various lists I have collected. Many EA charities are already implementing ambitious interventions. But really we’re quite clueless about what the future will bring.

This week alone I’ve discussed with friends and acquaintances three decisions in completely different contexts that might make the difference between paradise and hell for all sentient life, and not just in the abstract in the way that cluelessness forces us to assign some probability to almost any outcome but in the sense were we could point to concrete mechanisms along which the failure might occur. Yet we had to decide. I imagine that people in more influential positions than mine have to make similar decisions on almost a daily basis and on hardly any more information.

As a result, the robustness of an intervention has been the key criterion for prioritiza... (read more)

I had a similar idea, and I think that a few more things need to be included in the discussion of this.  There are multiple levels of ideas in EA, and I think that a red team becomes much more valuable when they are engaging with issues that are applicable to the whole of EA.  I think ideas like the institutional critique of EA, the other heavy tail, and others are often not read and internalized by EAs. I think it is worth having a team that makes arguments like this, then breaks them down and provides methods for avoiding the pitfalls pointed out in them.  Things brought up in critique of EA should be specifically recognized and talked about as good. These ideas should be recognized, held up to be examined, then passed out to our community so that we can grow and overcome the objections.  I'm almost always lurking on the forum, and I don't often see posts talking about EA critiques.  That should change. 
2Dawn Drescher1y
I basically agree but in this proposal I was really referring to such things as “Professor X is using probabilistic programming to model regularities in human moral preferences. How can that backfire and result in the destruction of our world? What other risks can we find? Can X mitigate them?” I also think that the category that you’re referring to is very valuable but I think those are “simply” contributions to priorities research as they are published by the Global Priorities Institute (e.g., working papers by Greaves and Tarsney come to mind). Rethink Priorities, Open Phil, FHI, and various individuals also occasionally publish articles that I would class that way. I think priorities research is one of the most important fields of EA and much broader than my proposal, but it is also well-known. Hence why my proposal is not meant to be about that.

Subsidise catastrophic risk-related markets on prediction markets

Prediction markets and catastrophic risk

Many markets don't exist because there isn't enough liquidity. A fund could create important longtermist markets on biorisk, AI safetry and nuclear war by pledging to provide significant liquidity once created. This would likely still only work for markets resolving in 1-10 years, due to inflation, but still*.

*It has been suggested to run prediction markets which use indices rather than currency. But people have shown reluctance to bet on ETH markets, so might show reluctance here too.

FTX, which itself runs prediction markets, might be particularly well-suited for prediction-market interventions like this.  I myself think that they could do a lot to advance people's understanding of prediction markets if in addition to their presidential prediction market, they also offered a conditional prediction market of how an indicator like the S&P 500 would do 1 week after the 2024 election, conditional on the Republicans winning vs the Democrats winning.  Conditional prediction markets for important indicators on big national elections would provide both directly useful info in addition to educating people about prediction markets' potential.

1Alex D1y
My company seeks to predict or rapidly recognize health security catastrophes, and also requires an influx of capital when such an event occurs (since we wind up with loads of new consulting opportunities to help respond). Is there currently any way for us to incentivize thick markets on topics that are correlated with our business? The idea of getting the information plus the hedge is super appealing!

Pandemic preparedness in LMIC countries


COVID has shown us that biorisk challenges fall on all countries, regardless of how prepared and well-resourced the countries are. While there certainly are many problems with pandemic preparedness high-income countries that need to be addressed, LMIC countries face even more issues in helping detect, identify, contain, mitigate, and/or prevent currently known and novel pathogens. Additionally, even after high income countries successfully contain a pathogen it may continue to spread within LMIC countries opening up risk of further more virulent mutations.

We'd like to see a project that works with LMIC governments to understand their current pandemic prevention plans and understand their local context. This project would especially focused on novel pathogens that are more severe than currently known pathogens -- and help provide the resources and knowledge needed to upgrade their plans to match the best practices of current bio-risk experts. Such a project would likely benefit from a team that contains expertise working with LMIC countries. An emergency fund and expert advice can also be provisioned to be ready to go when pathogens are... (read more)

Language models for detecting bad scholarship 

Epistemic institutions

Anyone who has done desk research carefully knows that many citations don't  support the claim they're cited for - usually in a subtle way, but sometimes a total nonsequitur. Here's a fun list of 13 features we need to protect ourselves.

This seems to be a side effect of academia scaling so much in recent decades - it's not that scientists are more dishonest than other groups, it's that they don't have time to carefully read everything in their sub-sub-field (... while maintaining their current arms-race publication tempo). 

Take some claim P which is below the threshold of obviousness that warrants a citation. 

It seems relatively easy, given current tech, to answer: (1) "Does the cited article say P?" This question is closely related to document summarisation - not a solved task, but the state of the art is workable. Having a reliable estimate of even this weak kind of citation quality would make reading research much easier - but under the above assumption of unread sources, it would also stop many bad citations from being written in the first place.

It is very hard to answer (2) "Is the cited ar... (read more)

Getting former hiring managers from quant firms to help with alignment hiring

Artificial Intelligence, Empowering Exceptional People

Despite having lots of funding, alignment seems to not have been very successful at attracting top talent to date. Quant firms, on the other hand, have become known for very successfully acquiring talent and putting them to work on difficult conceptual and engineering problems. Although buy-in to alignment before one can contribute is often cited as a reason, this is, if anything, even more of a problem for quant firms, since very few people are inherently interested in quant trading as an end. As such, importing some of this know how could help substantially improve alignment hiring and onboarding efficiency.

On malevolence: How exactly does power corrupt?

Artificial Intelligence / Values and Reflective Processes

How does it happen, if it happens? Some plausible stories:

  • Backwards causation: People who are “corrupted” by power always had a lust for power but deluded others and maybe even themselves about their integrity;
  • Being a good ruler (of any sort) is hard and at times very unpleasant, even the nicest people will try to cover up their faults, covering up causes more problems... and at some point it is very hard to admit that you were incompetent ruler all along.
  • Power changes your incentives so much that it corrupts all but the strongest. The difference with the last one is that value drift is almost immediate upon getting power.
  • A mix of the last two would be: you get more and more adverse incentives with every rise in power.
  • It might also be the case that most idealist people come into power under very stressful circumstances, which forces them to make decisions favouring consolidation of power (kinda instrumental convergence).
  • See also this on the personalities of US presidents and their darknesses.
Yes, that's interesting and plausibly very useful to understand better. Might also affect some EAs at some point. The hedonic treadmill might be  part of it. You get used to the personal perks quickly, so you still feel motivated & justified to still put ~90% of your energy into problems that affect you personally -> removing threats to your rule, marginal status-improvements, getting along with people close to you And some discussion about the backwards causation idea is here in an oldie from Yudkowsky: Why Does Power Corrupt? []

Bounty Budgets

Like Regranting, but for Bounties

Problem: In the same way that regranting decentralizes grantmaking, so do the same thing for bounties. For example, give the top 20 AI safety researchers up to $100,000 to create bounties or RFPs for, say, technical research problems. They could also reallocate their budget to other trusted people, creating a system of decentralized trust.

In theory, FTX’s regrantors could already do this with their existing budgets, but this would encourage people to think creatively about using bounties or RFPs.

Bounties are great because you only pay out if it's successful. If hypothetically each researcher created 5 bounties at $10,000 each that’d be 100 bounties - lots of experiments.

RFPs are great because it puts less risk on the applicants but also is a scalable, low-management way to turn money into impact.

Examples: 1) I’ll pay you $1,000 for every bounty idea that gets funded
2) Richard Ngo

More public EA charity evaluators

Effective Altruism

There are dozens of EA fundraising organizations deferring to just a handful of organizations that publish their research on funding opportunities, most notably GiveWell, Founders Pledge and Animal Charity Evaluators. We would like to see more professional funding opportunity research organizations sharing their research with the public, both to increase the quality of research in the areas that are currently covered - through competition and diversity of perspectives and methodologies - and to cover important areas that aren’t yet covered such as AI and EA meta.

Longtermist risk screening and certification of institutions

Artificial Intelligence, Biorisk and Recovery from Catastrophe

Companies, nonprofits and government institutions participate and invest in activities that might significantly increase global catastrophic risk like gain-of-function research or research that might increase the likelihood of unaligned AGI. We’d like to see an organisation that evaluates and proposes policies and practices that should be followed in order to reduce these risks. Institutions that commit to following these practices and submit themselves to independent audits could be certified. This could help investors and funders to screen institutions for potential risks. It could also be used in future corporate campaigns to move companies and investors into adopting responsible practices.

2Nathan Young1y
How would this be effective, rather than creating additional work on granmakers and increasing the entry barriers for grantees. Seems to many similar schemes for other kinds of risk end up as meaningless box-ticking enterprises which would lead to less effectiveness and possibly reputational harm to EA. This is my prior when I hear a new audit proposed, though I hope it won't apply in your case.
1Patrick Gruban1y
I agree that there is a risk that this leads to additional burden without meaningful impact.  Seeing the numbers of certifications currently deployed that are used public-facing for marketing as well as to reduce supply-chain risks (see for example this certifier []) I would see the chance that longtermist causes like biosecurity risks will be incorporated into existing standards or launched as new standards within the next 10 years at 70%.  If we can preempt this with building one or more standards based on actual expected impact instead of just using it to tick boxes. If this bet works out then we might make a counterfactual impact however I would also like to see the organisation shut down after doing research if it doesn't see a path to a certification having impact.

Resilient ways to archive valuable technical / cultural / ecological information
Biorisk and recovery from catastrophe

 In ancient Sumeria, clay tablets recording ordinary market transactions were considered disposable.  But today's much larger and wealthier civilization considers them priceless for the historical insight they offer.  By the same logic, if human civilization millennia from now becomes a flourishing utopia, they'll probably wish that modern-day civilization had done a better job at resiliently preserving valuable information.  For example, over the past 120 years, around 1 vertebrate species has gone extinct each year, meaning we permanently lose the unique genetic info that arose in that species through millions of years of evolution.
There are many existing projects in this space -- like the internet archive, museums storing cultural artifacts, and efforts to protect endangered species.  But almost none of these projects are designed robustly enough to last many centuries with the long-term future in mind.  Museums can burn down, modern digital storage technologies like CDs and flash memory aren't designed to last for centuries, and many... (read more)

2Dawn Drescher1y
Agreed, very important in my view! I’ve been meaning to post a very similar proposal with one important addition: Anthropogenic causes of civilizational collapse are (arguably) much more likely than natural ones. These anthropogenic causes are enabled by technology. If we preserve an unbiased sample of today’s knowledge or even if it’s the knowledge that we consider to have been most important, it may just steer the next cycle of our civilization right into the same kind of catastrophe again. If we make the information particularly durable, maybe we’ll even steer all future cycles of our civilization into the same kind of catastrophe. The selection of the information needs to be very carefully thought out. Maybe only information on thorium reactors rather than uranium ones; only information on clear energy sources; only information on proof of stake; only information on farming low-suffering food; no prose or poetry that glorifies natural death or war; etc. I think that is also something that none of the existing projects take into account.

AI Safety “school” / More AI safety Courses

Train People in AI Safety at Scale

Problem: Part of the talent bottleneck is caused by there not being enough people who have the relevant skills and knowledge to do AI safety work. Right now, there’s no clear way to gain those skills. There’s the AGI Fundamentals curriculum, which has been a great success, but aside from that, there’s just a handful of reading lists. This ambiguity and lack of structure lead to way fewer people getting into the field than otherwise would.

Solution: Create an AI safety “school” or a bunch more AI safety courses. Make it so that if you finish the AGI Fundamentals course there are a lot more courses where you can dive deeper into various topics (e.g. an interpretability course, values learning course, an agent foundations course, etc). Make it so there’s a clear curriculum to build up your technical skills (probably just finding the best existing courses, putting them in the right order, and adding some accountability systems). This could be funded course by course, or funded as a school, which would probably lead to more and better quality content in the long run.

Offer paid sabbatical to people considering changing careers

Empowering Exceptional People

People sometimes are locked-in in their non-EA careers because while working, they do not have time to:

  • Prioritize what altruistic job would fit them best
  • Learn what they need for this job

Create an organization that will offer paid sabbaticals to people considering changing careers to more EA-aligned jobs to help this transition. During the sabbatical, they could be members of a community of people in a similar situation, with coaching available.

Agree. I think that having an Advance Market Commitment system for this makes sense. E.g., FTX says 'We will fund mid-career academics/professionals for up to x months to do y. ' My experience is that most of the high value people I know who are good professional are sufficiently time poor and dissuaded by uncertainty that they won't spend 2-5 hours to apply for something they don't know they will get. The barriers and costs are probably greater than most EA funders realise.

An alternative/related idea is to have a simple EOI system where people can submit a fleshed out CV and a paragraph and then get  a AMC on an application - e.g., We think that there is a more than 60% chance that we would fund this and would therefore welcome a full application.

A public EA impact investing evaluator

Effective Altruism, Empowering Exceptional People

Charity evaluators that publicly share their research - such as GiveWell, Founders Pledge and Animal Charity Evaluators - have arguably not only helped move a lot of money to effective funding opportunities but also introduced many people to the principles of effective altruism, which they have applied in their lives in various ways. Apart from some relatively small projects (1) (2) (3) there is currently no public EA research presence in the growing impact investing sector, which is both large in the amount of money being invested and in its potential to draw more exceptional people’s attention to the effective altruism movement. We’d love to see an organization that takes GiveWell-quality funding opportunity research to the impact investing space and publicly shares its findings.

Seeing this late, but this is a wonderful idea! Will Roderick and I worked on "GiveWell for Impact Investing" a while ago and published this research [] on the EA Forum. We ultimately pursued other professional priorities, but we continue to think the space is very promising, stay involved, and may reenter it in the future.

Predicting Our Future Grants

Epistemic Institutions, Research That Can Help Us Improve

If we had access to a crystal ball that allowed us to know exactly what our grants five years from now otherwise would have been, we can make substantially better decisions now. Just making the grants we'd otherwise have made five years in the future can save a lot of grantmaking time and money, as well as cause many amazing projects to happen more quickly.

We don't have a  crystal ball that lets us see future grants. But perhaps high-quality forecasts can be the next best thing. Thus, we're extremely excited about people experimenting with Prediction-Evaluation setups to predict the Future Fund's future grants with high accuracy, helping us to potentially allocate better grants more quickly. 

Participatory longtermism

Values and reflective processes, Effective Altruism

Most longtermist and EA ideas come from a small group of people with similar backgrounds, but could affect the global population now and in the future. This creates the risk of longtermist decisionmakers not being aligned with that wider population. Participatory methods aim to involve people decisionmaking about issues that affect them, and they have become common in fields such as international development, global health, and humanitarian aid. Although a lot could be learned from existing participatory methods, they would need to be adapted to issues of concern to EAs and longtermists. The fund could support the development of new participatory methods that fit with EA and longtermist concerns, and could fund the running of participatory processes on key issues. 

Additional notes:

  • There is a field called participatory futures, however it seems not very rigorous [based on a very rough impression, however see comment below about this], and as far as I know hasn't been applied to EA issues.
  • Participedia has writeups of participatory methods and case studies from a variety of fields.
This comments section is pretty participatory.
Cool idea! :) You might be interested in skimming the report Deliberation May Improve Decision-Making [] from Rethink Priorities. > In this essay from Rethink Priorities [], we discuss the opportunities that deliberative reforms offer for improving institutional decision-making. We begin by describing deliberation and its links to democratic theory, and then sketch out examples of deliberative designs. Following this, we explore the evidence that deliberation can engender fact-based reasoning, opinion change, and under certain conditions can motivate longterm thinking. So far, most deliberative initiatives have not been invested with a direct role in the decision-making process and so the majority of policy effects we see are indirect. Providing deliberative bodies with a binding and direct role in decision-making could improve this state of affairs. We end by highlighting some limitations and areas of uncertainty before noting who is already working in this area and avenues for further research.
Love the idea - just writing to add that Futures Studies, participatory futures in particular & future scenario methodologies could be really useful for Longtermist research. Methods in these fields can be highly rigorous (I've been working with some futures experts as part of a project to design 3 visions of the future - which have just finished going through a lengthly stress-testing and crowd-sourcing process to open them up to public reflection and input), especially if the scenario design is approached in a systematised way using a well-developed framework. I could imagine various projects that aim to create a variety of different desirable visions of the future through participatory methods, identifying core characteristics, pathways towards them, system dynamics and so on to illustrate the value and importance of longtermist governance to get there. Just one idea, but there are plenty of ways to apply this field to EA/Longtermism! Would love to talk about your idea more as it also chimes with a paper I'm drafting, 'Contesting Longtermism', looking at some of the core tensions within the concept and how these could be opened up to wider input. If you're interested in talking about it, feel free to reach out to me at
Thanks for the point about rigor - I'm not that familiar with participatory futures but had encountered it through an organisation that tends to be a bit hypey. But good to know there is rigorous work in that field.  I agree that there are lots of opportunities to apply to EA/Longtermism and your paper sounds interesting. I'll send an email. 

Research on the long-run determinants of civilizational progress
Economic growth

What factors were the root cause of the industrial revolution?  Why did industrialization happen in the time and place and ways that it did?  How have the key factors supporting economic growth changed over the last two centuries?  Why do some developing countries manage to "catch up" to the first world, while others lag behind or get stuck in a "middle-income trap"?  Is the pace of entrepreneurship or scientific innovation slowing down -- and if so, what can we do about it?  Is increasing amounts of "vetocracy" an inevitable disease that afflicts all stable and prosperous societies (as Holden Karnofsky argues here), or can we hope to change our culture or institutions to restore dynamism?  At FTX, we'd be interested to fund research into these "progress studies" questions.  We're also interested in funding advocacy groups promoting potential policy reforms derived from the ideas of the progress studies movement.

2Jackson Wagner1y
See also many of Zac Townsend's ideas [], the idea of nuclear power & GMO advocacy [], and my list of object-level planks in the progress-studies platform [].

Pay prestigious universities to host free EA-related courses to very large numbers of government officials from around the world

Empowering Exceptional People

The direct benefit of the courses would be to give government officials better tools for thinking and talking with each other.

 The indirect benefit could be to allow large numbers of pre-disposed officials to be seen by <some organisation> who could use the opportunity to identify those with particular potential and offer them extra support or opportunities so they can make an even bigger impact.

The need for it to be free is to overcome the blocker of otherwise needing to write a business case for attendance which may then require some sort of tortuous approval process.

The need for it to be hosted at a prestigious university is to overcome the blocker of justifying to bosses or colleagues why the course is worthwhile by allowing piggybacking off the University's brand.

Infrastructure to support independent researchers

Epistemic Institutions, Empowering Exceptional People  

The EA and Longtermist communities appear to contain a relatively large proportion of independent researchers compared to traditional academia. While working independently can provide the freedom to address impactful topics by liberating researchers from the perversive incentives, bureaucracy, and other constraints imposed on academics, the lack of institutional support can impose other difficulties that range from routine (e.g. difficulties accessing pay-walled publications) to restrictive (e.g. lack of mentorship, limited opportunities for professional development). Virtual independent scholarship institutes have recently emerged to provide institutional support (e.g. affiliation for submitting journal articles, grant management) for academic researchers working independently. We expect that facilitating additional and more productive independent EA and Longtermist research will increase the demographic diversity and expand the geographical inclusivity of these communities of researchers. Initially, we would like to determine the main needs and limitations independent... (read more)

4Jackson Wagner6mo
(I think this is a good idea!  For anyone perusing these FTX project ideas in the future, here is a post I wrote [] exploring drawbacks and uncertanties that prevent people like me from getting excited about independent research as a career.)

EA Health Institute/Chief Wellness Officer 

Empowering Exceptional People, Effective Altruism, Community Building 

Optimizing physical and mental health can improve cognitive performance and decrease burnout. We need EAs/longtermists to have the health resilience to weather the storm - physical fitness, sleep, nutrition, mental health.  An institution could be created to assist EA aligned organizations and individuals. Using best practices from high performance workplace health, both personal and organizational, and innovative new ideas, a wellness team could help EAs have sustainable and productive careers. This could be done through consulting, coaching, preparation of educational materials or retreats. From a community growth perspective, EA becomes more attractive to some when one doesn’t have to sacrifice health for deeply meaningful work.

(Disclosure -I'm a physician/physician wellness SME - helping with this could be a good personal fit)

Unified, quantified world model

Epistemic Institutions, Effective Altruism, Values and Refelctive Processes, Research That Can Help Us Improve

Effective altruism started out, to some extend, with a strong focus on quantitative prioritization along the lines of GiveWell’s quantitative models, the Disease Control Priorities studies, etc. But they largely ignore complex, often nonlinear effects of these interventions on culture, international coordination, and the long-term future. Attempts to transfer the same rigor to quantative models of the long-term future (such as Tarsney’s set of models in The Epistemic Challenge to Longtermism) are still in their infancy. Otherwise effective altruist prioritization today is a grab bag of hundreds of considerations that interact in complex ways that (probably) no one has an overview over. Decision-makers may forget to take half of them into account if they haven’t recently thought about them. That makes it hard to prioritize, and misprioritization becomes more and more costly with every year.

A dedicated think tank could create and continually expand a unified world model that (1) is a repository of all considerations that affect altruistic decisi... (read more)

3Max Ghenis1y
Cool - you might also be interested in my submission, "Comprehensive, personalized, open source simulation engine for public policy reforms" []. It's not in the pitch but my intent is for it to be global as well.
3Dawn Drescher1y
Awesome, upvoted! You can also have a look at my “Red team” proposal []. It proposes to use methods from your field applied to any EA interventions (political and otherwise) to steel them against the risk of having harmful effects.

Civic sector software

Economic Growth, Values and Reflective Processes

Software and software vendors are among the biggest barriers to instituting new public policies or processes. The last twenty years have seen staggering advances in technology, user interfaces, and user-centric design, but governments have been left behind, saddled with outdated, bespoke, and inefficient software solutions. Worse, change of any kind can be impractical with existing technology systems or when choosing from existing vendors. This fact prevents public servants from implementing new evidence-based practices, becoming more data-driven, or experimenting with new service models.

Recent improvements in civic technology are often at the fringes of government activity, while investments in best practices or “what works” are often impossible for any government to implement because of technology. So while over the last five years, there has been an explosion of investments and activity around “civic innovation,” the results are often mediocre. On the one hand, governments end up with little more than tech toys or apps that have no relationship to the outcomes that matter (e.g. poverty alleviation, service deli... (read more)

3Yonatan Cale1y
Hey, this is somewhat my domain. The bottleneck is not building software, it is more like "governments are old gray organizations that don't want to change anything". If you find any place where the actual software development is the bottleneck, I'd be very happy to hear and maybe take part in it. I also expect many other EA developers to want to take part, it sounds like a good project

(For context, I was the Chief Data Officer of the California State Government and CTO of Newark, NJ when Cory Booker was Mayor). 

I actually think the way to do this is to partner with one city and build everything they need to run the city. The problem is that people can't use piecemeal systems very well. It would just take a huge initial set of capital -- like exactly the type of capital that could be provided here. 

1Yonatan Cale1y
Ah ok forget about it being somewhat my domain :P Sounds like a really interesting suggestion. Especially if it would be for a city that "matters" (that will help people do important things?), I think this project could interest me and others   (I'm interested if you have opinions about, [,] as a domain expert)
1Max Ghenis1y
Somewhat related, I submitted "Comprehensive, personalized, open source simulation engine for public policy reforms" []. Governments could also use the simulation engine to explore policy reforms and to improve operations, e.g. to establish individual households' eligibility for means-tested benefit programs.

Teaching secondary school students about the most pressing issues for humanity's long-term future

Values and Reflective Processes, Effective Altruism

Secondary education focuses mostly on the past and present, and tends not to address the most pressing issues for humanity’s long-term future. I would like to see textbooks, courses, and/or curriculum reform that promote evidence-based and thoughtful discourse about the major threats facing the long-term future of humanity. Secondary school students are a promising group for such outreach and education because they have their whole careers ahead of them, and numerous studies have shown that they  care about the future. This may serve a significant benefit in making more young people care about  these issues and support them with either their time or money

High-quality human data

Artificial Intelligence

Most proposals for aligning advanced AI require collecting high-quality human data on complex tasks such as evaluating whether a critique of an argument was good, breaking a difficult question into easier subquestions, or examining the outputs of interpretability tools. Collecting high-quality human data is also necessary for many current alignment research projects. 

We’d like to see a human data startup that prioritizes data quality over financial cost. It would follow complex instructions, ensure high data quality and reliability, and operate with a fast feedback loop that’s optimized for researchers’ workflow. Having access to this service would make it quicker and easier for safety teams to iterate on different alignment approaches

Some alignment research teams currently manage their own contractors because existing services (such as and don’t fully address their needs; a competent human data startup could free up considerable amounts of time for top researchers.

Such an organization could also practice and build capacity for things that might be needed at ‘crunch time’ – i.e., rapidly producing moderately la... (read more)

Advocacy for digital minds

Artificial Intelligence, Values and Reflective Processes, Effective Altruism

Digital sentience is likely to be widespread in the most important future scenarios. It may be possible to shape the development and deployment of artificially sentient beings in various ways, e.g. through corporate outreach and lobbying. For example, constitutions can be drafted or revised to grant personhood on the basis of sentience; corporate charters can include responsibilities to sentient subroutines; and laws regarding safe artificial intelligence can be tailored to consider the interests of a sentient system. We would like to see an organization dedicated to identifying and pursuing opportunities to protect the interests of digital minds. There could be one or multiple organizations. We expect foundational research to be crucial here; a successful effort would hinge on thorough research into potential policies and the best ways of identifying digital suffering.

X-risk Art Competitions

Fund competitions to make x-risk art to create emotion

Problem: Some EAs find longtermism intellectually compelling but not emotionally compelling, so they don’t work on it, yet feel guilty.

Solution: Hold competitions where artists make art explicitly intended to make x-risk emotionally compelling. Use crowd voting to determine winners.

Translate EA content at scale

Reach More Potential EAs in Non-English Languages

Problem: Lots of potential EAs don’t speak English, but most EA content hasn’t been translated

Solution: Pay people to translate the top EA content of all time into the most popular languages, then promote it to the relevant language communities.

7Dawn Drescher1y
Little addition: I imagine that knowledgeable EAs in the respective target countries should do that as opposed to professional translators so that they can do full language and cultural mediation rather than just translating the words.

Provide personal assistants for EAs

 Empowering Exceptional People

Many senior EAs spend way too much with busywork because it is hard to get a good personal assistant. This is currently so because: 

  1. There is no obvious source of reliable, vetted assistants.
  2. If an EA wants to become an assistant, it is harder for them to find a job for EA or on EA-related projects.
  3. Assistants have an incentive to have many clients, to avoid loss of income if they would lose their client. This leads to assistants having less time per client, and thus more time is spent on communication and less on work itself.
  4. Assistants tend to be paid personally by EAs instead of by their employers. That leads to using them less than would be optimal.
  5. There is no community of assistants that would be sharing knowledge and helping each other.

All these factors would be removed if an agency managed personal assistants.

4Dawn Drescher1y
Kat Woods (Nonlinear) is someone to talk to when it comes to this project.

Institutions as coordination mechanisms

Artificial Intelligence, Biorisk and Recovery from Catastrophe, Great Power Relations, Space Governance, Values and Reflective Processes

A lot of major problems - such as biorisk, AI governance risk and the risks of great power war - can be modeled as coordination problems, and may be at least partially solved via better coordination among the relevant actors. We’d love to see experiments with institutions that use mechanism design to allow actors to coordinate better. One current example of such an institution is NATO: Article 5 is a coordination mechanism that aligns the interests of NATO member states. But we could create similar institutions for e.g. biorisk, where countries commit to a matching mechanism - where “everyone acts in a certain way if everyone else does” - with costs imposed to defectors to solve a tragedy of the commons dynamic.

Sjir, you may be interested in Roote's [] work on meta existential risk []!
Thank you!

Experiments with and within video games

Values and Reflective Processes, Empowering Exceptional People

Video games are a powerful tool to reach hundreds of millions of people, an engine of creativity and innovation, and a fertile ground for experimentation. We’d love to see experiments with and within video games that help create new tools to address major issues. For instance, we’d love experiments with new governance and incentive systems and institutions, new ways to educate people about pressing problems, games that simulate actual problems and allow players to brainstorm solutions, and games that help identify and recruit exceptional people.

Replicate the Project Ideas Competition for other types of communities than EAs

Research That Can Help Us Improve

People have contributed a lot of really insightful and promising ideas here. Given that "there are no wrong ideas in brainstorming" and that there may be systematic blind spots for effective altruists/longtermists' paradigm, perhaps doing this broad-idea-crowdsourcing exercise in other types of communities could get us new, potentially promising ideas. 

Regular prizes/awards for EA art

Effective Altruism

Works of art (e.g. stories, music, visual art) can be a major force inspiring people to do something or care about something. Prizes can directly lead to work (see for example the creative writing contest), but might also have an even bigger role in defining and promoting some type of work or some quality in works. Creating a (for example) annual prize/award scheme might go a long way towards defining and promoting an EA-aligned genre (consider how the existence of Hugo and Nebula awards helps define and promote science fiction). The existence of a prestigious / high-paying prize for the presence of specific qualities in a work is also likely to draw attention to those qualities more broadly; news like "Work X wins award for its depiction of [thoughtful altruism] / [the long-term future] / [epistemic rigor under uncertainty]" might make those qualities more of a conversation topic and something that more artists want to depict and explore, with knock-on effects for culture.

Impact markets to smooth out retroactive funding

Effective Altruism, Empowering Excetional People, Economic Growth, Epistemic Institutions

Yonatan Cale already made the case for retroactive funding, i.e. that it’s easier to tell what has succeeded than what will succeed. The questions of what will succeed, in turn, can be answered by a market.

Investors will try to predict which charities will succeed to the point of receiving retroactive funding. A retroactive funder can make larger grants in proportion to their reduction in uncertainty (5–10x), time savings from having to do less vetting (~ 2x), and delay (~ 1.5x). Hence investors with enough foresight can even make a profit and turn the prediction of retro fund decisions into their business model. Promising charities can bootstrap rapidly with these early financial injections, successful serial charity entrepreneurs can accumulate more and more capital to reinvest into their next charity venture, and funders save time because they have to do only a fraction of the vetting.

We – Kenny Bambridge, Matt Brooks, Dony Christie, Denis Drescher, and a number of advisors – are actively working toward this goal. I’ve been thinking about the m... (read more)

Studying Economics Growth Deterrents and Cost Disease

Economic growth

Economic growth has forces working against it. Cost disease is the most well-known and pernicious of these in developed economies. We are interested in funding work on understanding, preventing, and reversing cost disease and other mechanisms that are slowing economic growth. 

(Inspired by Patrick Collison)

Secure full-stack open-source computing for information security

Artificial Intelligence, Biorisk, Research that will help us improve

Much of our sensitive research and weaponry, like AI, biolabs, nuclear weapons, etc, are built upon insecure infrastructure. Think of a scenario in the future where one hacker could hack and control fleets of self-driving cars and essentially have a swarm of missiles. Real information security would need to build the full stack of computing from the hardware, OS, compilers, to application layers. It would also ideally be open-source and inspectable to ensure security.

Funding Stress/Penetration Tests of vital orgs/infrastructure

Cyber Risks, Cybersecurity

Most orgs don't spend enough on ensuring their infrastructure is safe from hackers and we should ensure that labs working on AI safety, biorisk companies, EA orgs etc. are safe from malicious hackers.

Longtermist democracy / institutional quality index

Values and Reflective Processes, Epistemic Institutions

Several indices exist to quantify the degree of liberal democracy in all countries and territories around the world, like Freedom in the World and the EIU's Democracy Index. These indices are convenient for describing and comparing the state of liberal democracy in different countries, because they distill the various complicated aspects of a state's political system into one or more numbers that are easy for a layperson to understand.

We propose a "democracy index" that emphasizes the qualities of political systems that are most relevant to making the long-term future go well. Such qualities could include voting systems, free and fair elections, voter competence, and capacity for long-term planning in government - and the set of qualities used could be based on research such as this post. This index would help make analysis of countries and territories' political systems more accessible to EAs/longtermists who aren't political scientists, since it would distill them down to a few easy-to-understand numbers. It would also help the longtermist community track progress towards bet... (read more)

Fund Sentinel, a nationwide pandemic early response system (originally suggested by alexrjl)


Fund the biosecurity program explained on this podcast. Any time anyone gets sick you sequence a sample. Any unknown genetic material gets sequenced again at a higher level. This allows for rapid response to new pathogens.


Politician forecasting stipend

Politics, better epistemics

Many people think politicians are underpaid. Many think they have a poor grasp of the likelihood of future events. Offer every Senator and Representative a yearly sum to make public predictions about future public statistics. The forecasting would help them correct their own errors and provide a valuable source of information on who makes good decisions about the future and who doesn't.

See one version of this here:

Making Future Grantmaking More Optimal

Effective altruism

  • The EA community will likely spend much more money in the future than what they spend know. Grantmaking is hard and the right setup is controversial. Hence, it might make sense to spend money on how to do it well.
  • One could invite people to so called "donation parliaments" with 100 randomly selected citizens who get expert/EA input, invite 100 top academics to give away 10 million. Try out expert committees or democratic control. Organising such donation parliaments etc could also receive positive media attention.

Moderators for EA/Longtermist FB/Groups or Discords

Effective Altruism

(Refinement of EA-relevant Substacks, Youtube, social media, etc. )

Given the huge amount of funding available to EA, we probably don't want to skimp on moderators for major Facebook or Slack or Discord groups even though these have traditionally been run by volunteers. It'd be worthwhile at least experimenting to see if paid part-time moderators would be able to add extra value by writing up summaries/content for the groups, running online calls, setting up networking spreadsheets and spending more time thinking through strategy.

Risks: We might end up paying money for work that we would have gotten for free. Attempts to set up networking spreadsheets or run calls might have minimal participation and hence minimal impact.

Fyi paid part-time moderators would need buy-in from the online community – the EA Corner Discord seems against paid moderators, for example. I really appreciate how many ideas you're proposing, Chris!
2Chris Leong1y
Of course. I guess I could see some negative effects from how it could encourage people to seek the mod roles as a way of being paid rather than because they could do a good job. I think that this issue could mostly be avoided by offering student level rates. Running a Facebook group seems like a nice entrypoint into EA movement building. Thanks, you're welcome!
Incentives could also be aligned by offering existing volunteer mods a salary to spend more time moderating.

More Insight Timelines

In 2018, the Median Group produced  an impressive timeline of all of the insights required for current AI, stretching back to China's Han Dynasty(!)

The obvious extension is to alignment insights. Along with some judgment calls about relative importance, this would help any effort to estimate / forecast progress, and things like the importance of academia and non-EAs to AI alignment. (See our past work for an example of something in dire need of an exhaustive weighted insight list.)

Another set in need of collection are more general breakthroughs - engineering records broken, paradigms invented, art styles inaugurated - to help us answer giant vague questions about e.g. slowdowns in physics, perverse equilibria in academic production, and "Are Ideas Getting Harder to Find?"

Research differential technological progress and trajectory changes
Research That Can Help Us Improve, Values and Reflective Processes

The idea of Differential technological progress (DTP) may be a crucial consideration for many at-first-glance good ideas like:

- improving scientific publishing

- increasing GDP

- increasing average intelligence


But given its importance, there hasn`t been much research and publications on GTP.

Central question for research is how to use DTP to prioritize interventions. Examples of subquestion to research are:

- when in the past there were intentional trajectory changes.

- what subgoals seem to be good when DTP is considered.

- and so on.

Bridging-based Ranking for  Recommender Systems

Artificial Intelligence, Epistemic Institutions, Values and Reflective Processes, Great Power Relations

Recommender systems are used by platforms like FB/Meta, YouTube/Google, Twitter, TikTok, etc. to direct the attention of billions of people every day. These systems, due to a combination of psychological, sociological, organizational, etc. factors are currently most likely to reward content producers with attention if they stoke division (e.g.  outgroup animosity). Because attention is a currency that can be converted into money, power, and status, this "bias toward division” has impacts groups at every scale; from local school boards to Congress to geopolitics.  

Ensuring that recommender systems can mitigate this bias is crucial to functional democracy, to cooperation on catastrophic risks (e.g. AGI, pandemics,  climate change), and simply to reducing the likelihood of escalating wars. We urgently need more research on how to better design recommender systems;  we need to create open source implementations that do the right thing from the start which can be adopted by cash-strapped startups; and we need a mix of pressure and support to ensure these improvements will be rapidly deployed at platform scale.

Headhunter Office: targeted recruitment of aligned MDs, and other mid-career professionals

Effective Altruism, Community Growth and Diversity

I am a physician, and I have several conversations a week with bright, altruistic, and burned out colleagues. These professionals are often in a position to earn to give, and also can be entrepreneurial and adept at navigating complex systems and could be future organizational leaders or 'founder types'. Currently, there are cosmetic MLM groups and others recruiting from this group of physicians looking to make their lives more fulfilling and meaningful while still earning an income - there is literally an MD Exit strategy facebook group. 

I propose an EA headhunter office to recruit for the community. For example, recruiting physicians explicitly, using some of the successful techniques that pharma uses like having physicians recruit their peers. Perhaps there are similar aligned mid-career professionals in law, public administration, engineering, etc.

Support for EAs having children 

Empowering Exceptional People, Effective Altruism 

Children of EAs are much more likely to become EAs (100-1000x?) and future generations of EAs may have a large impact. Having children usually means a pause in work which is poorly compensated and difficult to time. I propose an institute to support EAs wishing to have children. EAs could be could be supported with fertility costs, including egg freezing, and be given grants for parental leave which improve parental and child health outcomes. There are many trade offs in parenting, which could be discussed in an EA parenting forum. Building a community could benefit these EA parents and their children.

Evidence for 100-1000x estimate? What is the base rate for children following their parents? When I've seen this discussed before, the conclusion is usually that memetic transfer of EA is much easier than genetic transfer of EA.

1Lauren Reid1y
That’s a good point. I agree it’s the culture more than the DNA which matters and I don’t know the real numbers. Of course, my husband and I were identified as gifted and it looks very likely our children will be too, and we also have read them Open Boarders as a bedtime story. Adopted children of EAs are also probably much more likely than average to become EAs. I think of organized religions and how they encourage children and future generations - there may be a lesson for us there.
3Jack Malde1y
I would have thought it would be higher value on the margin to spread EA to talented people who already exist than to make more people.
Very minor, but just wanted to check you were aware of, and had joined if interested, the EA parents facebook group [].
3Lauren Reid1y
I didn’t know that, nor did my husband (Alex D), who is much more in the EA space. Thank you for posting, I will join. We are seriously considering going to Nassau with our neuro atypical kids (4 and 7) next winter, and are trying to figure out how it would work for schooling/childcare in particular.

DIY decentralized nucleic acid observatory

Biorisk and Recovery from Catastrophes

As part of the larger effort of building an early detection center for novel pathogens, a smaller self-sustaining version is needed for remote locations. The ideal early-detection center would not only have surveillance stations in the largest hubs and airports of the world, but also in as many medium sized ones as possible. For this it is needed to provide a ready-made, small and transportable product which allows meta-genomic surveillance of wastewater or air ventilation. One solution would be designing a workflow utilizing the easily scalable and portable technology of nanopore sequencing and combining it with a workflow to extract nucleic acids from wastewater. The sharing of instructions on how to build and use this method could lead to a "do it yourself" (DIY) and decentralized version of a nucleic acid observatory. Instead of staffing a whole lab at a central location, it would be possible to only have one or two personnel in key locations who use this product to sequence samples directly and only transmit the data to the larger surveillance effort. 


A global observatory for institutional improvement opportunities

Research That Can Help Us Improve, Great Power Relations, Epistemic Institutions

Actions taken by powerful institutions—such as central governments, large corporations, influential media outlets, and R&D labs—can dramatically shape people's lives today and cast a shadow long into the future. It can be hard to know what philanthropic strategies would be most likely to drive better would outcomes, however, because each individual institution is itself a complex ecosystem of incentives, external pressures, norms, policies, and bureaucratic structures. An ongoing project to document how important institutions operate in practice and spot relevant windows of opportunity (e.g., legislation under consideration, upcoming leadership transitions, etc.) as they emerge would be very helpful for mapping the strategic landscape across virtually all of our other interest areas.

EA content translation service

Effective Altruism, Movement Building

(Maybe add to #30 - diversity in EA)

EA-related texts are often using academic language needed to convey complex concepts. For non-native speakers reading and understanding those texts takes a lot more time than reading about the same topic in their native language would. Furthermore, today many educated people in important positions, especcially in non-western countries, do not at or only poorly speak English. (This is likely part of the reason that EA currently mainly exists in English speaking countries and almost exclusively consists of people speaking English well.)

To make EA widely known and easy to understand there needs to be a translation service enabling e.g. 80k, important Forum posts or the Precipice to be read in different languages. This would not only make EA easier to understand - and thus spread ideas further - but also likely increase epistemic diversity of the community by making EA more international.

Pipeline for writing books

Effective altruism

It's plausible that more EAs/longtermists should be writing books on the interesting subjects they are experts in, but they currently do not because of a lack of experience or other types of friction. Crowdsourced resources, networks, and grants may help facilitate this. Books written by EAs would have at least two benefits: (a) dissemination of knowledge, and (b) earning-to-give opportunities (via royalties).

This is an interesting idea; it definitely seems plausible that EAs (who often have a lot of unique knowledge!) might be underrating the benefits of writing books.  Could you expand a little on what you are thinking here?  (I'd also be interested to hear from anyone else with relevant experience.)  How hard is it to publish a book?  If you try, do you have a high chance of getting rejected?  How do people usually do marketing and get people to read their stuff? 

Maybe this is too cynical of me (or too internet-centric), but I doubt the main benefits would come from earning royalties (not likely to be very profitable relative to other things skilled EAs could be doing!) or spreading knowledge (just read the blog posts!).  But I think trying to publish more EA books might help greatly with:

  1. Prestige and legibility (just like how academic papers are considered more legit than blog posts by academics and governments).  It might be easier for, say, the US Democratic Party to get behind an EA-inspired pandemic-prevention plan, foreign-aid revamp, or predction-markets-y institutional-reform agenda if they could point to a prestigious book rather than
... (read more)
1Peter S. Park1y
Thanks so much, Jackson! I have never published a book, but some EAs have written quite famous and well-written books. In addition to what you suggested, I was thinking "80,000 pages" could organize mentoring relationships for other EAs who are interested in writing a book, writer's circles, a crowdsourced step-by-step guide, etc. Networking in general is very important for publishing and publicizing books, from what I can gather, so any help on getting one's foot in the door could be quite helpful.
1Logan Riggs1y
My brother has written several books and currently coaches people on how to publish it and market it on Amazon. He would be open to being paid for advice in this area (just dm me) I think the dissemination and prestige are the best arguments so far.

Institute/Grants for improving the science of indoor air quality


During Covid we learned that ‘air is the new poop’ in terms of hygiene. Improving indoor air quality can prevent respiratory pathogen transmission both in the case of a pandemic and for general health. A granting agency could support advances in indoor air quality and their implementation such as in airplanes and classrooms.

Longtermism Policy Lab

Epistemic Institutions, Values and Reflective Processes, Great Power Relations, Space Governance, Research That Can Help Us Improve

Despite the growing recognition of the importance of long-term perspectives, governance remains oriented around short-term incentives. More coordination and collaboration between researchers and policymakers, practicioners and industry professionals is needed to translate research into policy. The Longtermism Policy Lab will bridge this gap, working with societal partners and governments at all levels (local to global) to undertake policy experiments. The Lab will also contain a research component, establishing and pursuing an ambitious interdisciplinary Longtermism research agenda, including an emphasis on research that doesn't fit well within either academia or traditional research institutes. We want to see this organisation serve as a direct link between longtermism as a governance approach and its implementation within all levels of governance across the globe.

(Per Nick's post, reposting)

Private-sector ARPA models


Many of the technological innovations of the last fifty years have their genesis in experiments run by DARPA. ARPA models are characterized by individual decision-makers taking on risky bets within defined themes, setting ambitious goals, and mobilizing top researchers and entrepreneurs to meet them. We are interested in funding work to study these funding models and to create similar models in our areas of interest. 

In case you drew inspiration from some of our suggestions in the megaprojects article, we would like to retroactively apply. 

Promote Ethical Corporate Behavior

EA to purchase 5% of Blackrock and 5% of Vanguard shares. To be clear, I don't mean 5% of their index funds, but rather 5% of the underlying fund management companies.

EA's investment of circa $10 billion can be leveraged into a board seat on companies that manage circa $20 trillion in assets. EA could lobby these companies to apply a corporate ethics tests on all their index funds. E.g. excluding coal and promoting other EA priorities.

Thanks for this suggesting, I'm really interested in this general direction, in case anybody wants to dig into it a bit more. It spontaneously seems unlikely to me that investing this large share of EA money is the best bet, but I wonder if there are other ways to influence them (e.g. as I understand Blackrock and Vanguard senior managers simply got convinced that climate change is a downside for their longterm profit, and they probably should believe the same for misaligned AI / AI races). Maybe another route would be to ensure that those investment firms will be able to influence & coordinate the behavior of tech firms to reduce competitive dynamics.

Anti-Pollution of the universe

Space governance

As we take one small step for man, our giant leap for humanity leaves footprints of toxicity that we justify as ‘negative externalities’. There are currently 20,000 catalogued objects comprised of rocket shards, collision debris and inactive satellites which cause major traffic risks in orbit around our planet whilst also likely polluting the universe. As Boeing, OneWeb, SpaceX etc increase their launches, we similarly add to the congestion and space collision probabilities. (read as disasters waiting to happen). There are currently NO debris removal methods. If we’ve learnt anything from our current micro history of mankind on Earth – it’s that the nature/universe around us is important since we’re intricately linked and that there are costs to our polluting behaviour in the pursuit of ‘territory/energy etc’. Hence when we’re playing at the macro cosmic level – it is even more imperative that we get this framework/relationship/thought process right.

Nuclear Funding Shortfall

Nuclear Risk

There has been a significant shortfall in nuclear risk funding. The most effective elements of this could be covered by the fund.

Superforecasting team

Global catastrophic risks

We know that top forecasters exist, but few are currently employed to predict around long term risks. These forecasters should be supported by developers to help maximise their accuracy and output. Multiple organisations could employ 100s or 1000s of top forecasters to analyse developing situations and suggest outcomes that are most likely to resolve them for the interests of all consciousness.

CEA for the developing world

Effective Altruism

The main EA movement building organization, CEA, focuses primarily on talented students in top universities of developed countries. This seems to be due to a combination of geographical and cultural proximity, quantity of English speakers, and ease of finding top talent. However, there is a huge amount of untapped talent in developing countries that may be more easily reached through dedicated organizations optimized for being culturally, linguistically, and geographically close to such talent, such as a CEA for India or Brazil. Such an organization would develop its own goals and strategies tailored to their respective regions, such as prioritizing nationwide prizes over group-by-group support, hiring local EA talent to lead projects, and identifying and partnering with regionally influential universities and institutions. This project would not only contribute to increasing diversity in EA, but also foster organizational competition by allowing different movement building strategies, and better position the EA movement for unexpected geopolitical power shifts.

An ecosystem of organizations to initiate a “Hasty Reflection”

Values and Reflective Processes, Epistemic Institutions, Effective Altruism, Research That Can Help Us Improve

The Long Reflection appears to me to be robustly desirable. It only suffers from being more or less unrealistic depending on how it is construed.

In particular, I feel that two aspects of it are in tension: (1) delaying important, risky, and irreversible decisions until after we’ve arrived at a Long Reflection–facilitated consensus on them, and (2) waiting with the Long Reflection itself until after we’ve achieved existential security.

I would expect, as a prior, that most things happen because of economic or political necessity, which is very hard to influence. Hence the Long Reflection either has to ramp up early enough that we can arrive at consensus conclusions and then engage in the advocacy efforts that’ll be necessary to improve over the default outcomes or else risk that flawed solutions get locked in forever. But the first comes at the risk of diverting resources from existential security. This indicates that there is some optimal trade-off point between existential security and timely conclusions. (From ... (read more)

Scandinavian-like parental leave (25 weeks +) in EA organizations

Leading the way with a policy combating demographic decline, while supporting talent selection and diversity in EA community 

Paid parental leave creates an incentive to have (more) kids - or rather, it takes away part of the large financial incentive not to have kids. My concrete suggestion is to fund Scandinavian-like parental leaves for employees in specified EA organizations. This would open up more access to the large pool of talented family oriented persons. Further, having an unusually beneficial parental leave benefit could inspire other organizations to follow, and thus help combat demographic decline. The idea should be quite easy to pilot, implement and scale, and the results relatively easy to measure.

Good idea, just in terms of talent selection and diversity. Has such parental leave had a noticeable effect on fertility / demographic decline in Scandinavia?
2Kjersti Moss1y
Hi, Greg.  Thank you for your question. I'm very interested in exploring this idea further. First, I want to say that I have not done deep research on the topic. But I know some stuff, and I suspect some stuff. I could be wrong. Here are some of my thoughts: 1) My first point is that it is very plausible that paid leave period has a substantial effect on birth rates. I would sort of have a null hypothesis that there is a large effect, rather than zero/small effect.  2) I'm a statistician, and normally don't put too much weight on personal experience. But before all my three pregnancies a thorough analysis with the conclusion "this is doable financially", was very important in my decision-making. This N=1 (or N=3 if counting three kids), partly forms my opinion on the null hypothesis above. My impression is that most responsible adults have similar thought processes before getting pregnant.  Further, after graduating, I was very motivated to have an impactful career. The apparent lack of job security and paid parental leave made sure I was 0% interested in any job at an EA org. Not the end of the world in this specific case, but there are probably a lot of more talented women out there, who also have a 0% interest for the same reasons. 3) I wrote Scandinavian-like because these are countries with generous parental leaves, and I know the setup well, as I'm Norwegian. Again, I have not researched in detail, but all three Scandinavian countries have birth rates well above the average in Europe. I also know France is on top of the birth statistics, and have generous set-ups. If I was French, the headline would maybe point to France instead of Scandinavia :) 4) Apart from (3), it is not straightforward to see a direct effect on leave times and birth rates in Scandinavia. Policies change very slowly over time (might add 1 week every now and then), and changes go hand-in-hand with other policies as free/subsidized kindergardens.  5) It seems to be a bit under-researche

Monitoring Nanotechnologies and APM
Nanotechnologies, and a catastrophic scenario linked to it called “Grey goo”, have received very little attention recently (more information here ), whereas nanotechnologies keep moving forward and that some think that it’s one of the most plausible ways of getting extinct. 

We’d be excited that a person or an organization closely monitors the evolution of the field and produces content on how dangerous it is. Knowing whether there are actionable steps that could be taken now or not would be very valuable for both funders and researchers of the longtermist community.

Quick start kit for new EA orgs

EA ops

Stipe atlas for longtermist orgs. Rather than figuring out the best tools, registrations, and practices for every new org, figure out the best default options and provide an easy interface to start up faster.

7Dawn Drescher1y
I just read the the Charity Entrepreneurship handbook How to Launch a High-Impact Nonprofit []. That seems to fit the bill. Maybe having country-specific versions of it and versions for longtermist orgs, would be even better.

Campaign to eliminate lead globally

Economic Growth

Lead exposure limits IQ, takes over 1M lives every year and costs Africa alone $130B annually, 4% of GDP: an extraordinary limit on human potential. Most lead exposure is through paint in buildings and toys. The US banned lead paint in 1978 but 60% of countries still permit it. We would like to see ideas for a global policy campaign, perhaps similar to Bloomberg’s $1B tobacco advocacy campaign (estimated to have saved ~30M lives), to push for regulations and industry monitoring.

Epistemic status: The “prize” feels very large but I am not aware of proven interventions for lead regulations. 30 minutes of Googling suggests the only existing implementer ( might be too small for this level of funding so there may not be many applicants. 

Conflict of interest: I work for a small, new non-profit focused on $B giving. We are generally focused on projects with large, existing implementers so have not pursued lead elimination policy beyond initial light research

Build LMIC university capacity

Economic growth, Empowering exceptional people

Universities in LMICs often have limited access to funding. Additional funding could enable many good outcomes including:

  • Greater  opportunities for exceptional people born in LMICs
  • Better and more influential academic contributions from people in LMICs, which would increase the diversity of backgrounds of people contributing to global academia, and perhaps uncover key errors and blindspots in Western academic thought.
  • Boost economic growth in LMICs
  • Boost epistemic standards in LMICs
  • Help improve LMIC capacity to undersand and plan for catastrophes, e.g. from pandemics and climate change. 

Funding could be focussed on issues of concern to EAs, such as pandemics, or could be unrestricted to boost overall university capacity. As well as funding universities, funds could be provided for networks, independent labs, access to journals, travel and conferences, spinout companies etc. 

Increasing Earth’s probability of survival

Space governance

Currently as we transition from a Kardashev Type 0 to Type 1 civilization, our probability of encountering/alerting other civilisations increases exponentially. This is somewhat ironic that we may fall upon our impetus. Citing dark forest theory that the end game is such that ‘lacking assurances, the safety option for any species is to annihilate other life forms before they have a chance to do the same’, means that humanity is immediately on the defensive. (Applying a chronological framework and assuming linearity of time) As of such – we should fund ways to increase our probability of survival (by either deterrence mechanisms, signalling non threat or camouflage) such that we may evolve uninterrupted. (This also assumes we don’t kill ourselves first, the probability of which is sadly also non zero) 

Just throwing out crazy suggestions (I’m sensing that’s what the thematic is here) would be something like hyper gravity generation device that bends observable light emitted from our planet, so much so that when observed – we would look like a blackhole.

Combatting DeepFake

Epistemic Institutions, Artificial Intelligence

As AI advances –Numerous high quality deepfake videos/images are being produced at an alarmingly increasing rate. Delving into the question ‘What happens when we can’t trust our eyes and ears anymore?’, This immediately raises obvious signals that it will affect many industries such as journalism, military, celebrities, government etc. Proactively funding a superior ML anti-deepfake bot for commercial use is  important such that images/videos can be properly verified. The end game will likely come down to some degree of superior computing power since both are ML based algos – hence the advantage here would be first mover and/or altruistic (think along the same example of free antivirus software) in nature. 

Targeted practical statistical training

Economic Growth, Values and Reflective Processes

"Human cognition is characterized by cognitive biases, which systematically lead to errors in judgment: errors that can potentially be catastrophic (e.g., overconfidence as a cause of war). For example, a strong case can be made that Russia's invasion of Ukraine has been an irrational decision of Putin, a consequence of which is potential nuclear war. Overconfidence is a cause of wars and of underpreparation for catastrophes (e.g., pandemics, as illustrated by the COVID-19 pandemic).

One way to reduce detrimental and potentially catastrophic decisions is to provide people with statistical training that can help empower beneficial decision-making via correct calibration of beliefs. (Statistical training to keep track of the mean past payoff/observation can be helpful in a general sense; see my paper on the evolution of human cognitive biases and implications.) At the moment, statistical training is provided to a very small percentage of people, and most provisions of statistical training are not laser-focused on the improvement of practical learning/decision-making capabilities, but for other indir... (read more)

Mental health treatment to prevent anthropogenic catastrophic/existential risks

Biorisk and Recovery from Catastrophe

Issues of mental health can be very harmful to the well-being of the self and others. The degree to which this harm can occur can, when combined with technology, even result in catastrophic/existential risks. (The Russian invasion of Ukraine, the cause of which may be the mental state of Putin, can plausibly lead to nuclear war. Another example is engineered pandemics.) Given the disproportionately anthropogenic skew of catastrophic/existential risks, research/funding/advocacy for mental health treatment (general or targeted) may help prevent such risks.

Reminds me of some of the proposals here:

We therefore consider interventions to reduce the expected influence of malevolent humans on the long-term future.

  • The development of manipulation-proof measures of malevolence seems valuable, since they could be used to screen for malevolent humans in high-impact settings, such as heads of government or CEOs. (More)
  • We also explore possible future technologies that may offer unprecedented leverage to mitigate against malevolent traits. (More)
  • Selecting against psychopathic and sadistic tendencies in genetically enhanced, highly intelligent humans might be particularly important. However, risks of unintended negative consequences must be handled with extreme caution. (More)
1Peter S. Park1y
Yes, I think these proposals together could be especially high-impact, since people who pass screening may develop issues of mental health down the line.

Rule of Law Fund

Values and Reflective Processes and Economic Growth

A strong rule of law helps ensure equity, human rights, property rights, contract enforcement, and due process. Many countries are still developing their legal systems. Between 2010 and 2020 twenty-four different countries ratified a constitution.  The legal systems that evolve today will have a lasting impact on future generations.

This fund would offer funding for organizations and individuals engaged in legal scholarship and litigation that align with  the Future Fund’s guiding principles, with a specific focus on strengthening the rule of law in countries with less developed legal institutions.

Reflection Retreats

Effective Altruism

There are certain points in our life when the decisions we make can greatly affect the trajectory of our lives. This could include deciding what degree to study, graduating or making a major career change. These retreats would bring together a bunch of EAs together (possibly some non-EAs too) to reflect on these decision and start making applications/plans, ect.

AI alignment prize suggestion: Improve our ability to evaluate (and provide training signal for) fuzzy tasks

Artificial Intelligence

There are many tasks that we want AI systems to do, for which performance cannot be evaluated automatically (and thus training signal provision is hard). If we don't make progress on our ability to train systems for such tasks, we might end up in a world full of systems that optimise for that which is easy to measure, rather than what we actually want. One example of such a task is the evaluation of free-form text; there is currently no automated method to evaluate free-form text (with respect to criteria such as usefulness or correctness) that matches human evaluation. The Future Fund could offer prizes for work that takes a task for which the gold-standard of evaluation is humans, and demonstrates an automated evaluation method that matches human evaluation very closely (or work that demonstrates an automated evaluation method to be superior to human evaluation).

Note: This is crucially not the same as "training models to perform well on the task in question". There are a number of technical reasons why what I suggest is easier. Intuitively, evaluating performance is often considerably easier than generating good performance. For example, I can watch a movie and say if it's good, but I can't make a good movie.

EA Programming Bootcamp

Effective Altruism

Providing a programming bootcamp to members of the Effective Altruism community could be a way of assisting struggling community members whilst avoiding the issues inherent with directly providing cash assistence. It could also allow communities members to accelerate their career progression.

Notes: See the comments here for some of the issues with giving cash.

I suspect that the impact of this would be larger than it first appears as a) talented people generally want to be part of a community where people are successful b) if community members are struggling then that takes up the time of other community members who try to help them.

5Yonatan Cale1y
I think funding programming bootcamps is a great idea (if anyone needs it) and I intend to fund the first 3 people who'll ask me even just to see how it goes. [This is not a formal commitment because I don't want to think of all the edge cases like a $10k course; but I do currently intend to do it. DM me if you want]
6Chris Leong1y
Most proper bootcamps are very expensive, like about that kind of rate I'd guess.
1Yonatan Cale1y
Ah, I meant online courses (Sorry, language mistake on my side)

Research institute focused on civilizational lock-in

Values and Reflective Processes, Economic Growth, Space Governance, Effective Altruism

One source of long-term risks and potential levers to positively shape the future is the possibility that certain values or social structures get locked in, such as via global totalitarianism, self-replicating colonies, or widespread dominance of a single set of values. Though organizations exist dedicated to work on risks of human extinction, we would like to see an academic or independent institute focused on other events that could have an impact on the order of millions of years or more. Are such events plausible, and which ones should be of most interest and concern? Such an institute might be similar in structure to FHI, GPI, or CSER, drawing on the social sciences, history, philosophy, and mathematics.

Nonprofit Growth Research Think Tank/Consultancy

EA Ops, Effective Altruism

Most EA organisations and projects will be faced (at several times during their organisational lifecycle) with changes to their organisations due the growth of their teams.
If handled poorly a team can grow with many "growing pains" such as processes, policies, financial systems, (project) management and organisational/team structures that are not fit to the new status quo.

We'd love to see an organization that guides other EA organisations on their path to growth by identifying the right strategies and blind spots to manage the change phase in a period of growth.

Prevent stable global totalitarian regimes through uncensorable broadcasts

Great Power Relations, Epistemic Institutions

Human civilization may get caught in a stable global totalitarian regime. Current and past totalitarian regimes have struggled with influences from the outside. So it may be critical to make sure now that future global totalitarian regimes will also have influences from the outside.

North Korea strikes me as a great example of a totalitarian regime straight out of 1984. Its systematic oppression of its citizens is so sophisticated that I could well imagine a world-wide regime of this sort to be stable for a very long time. Even as it exists today, it’s remarkably stable.

The main source of instability is that there’s a world all around North Korea, and especially right to its south, that works so much better in terms of welfare, justice, prospecity, growth, and various moral preferences that are widely shared in the rest of the world.

There may be other sources of instability – for example, I don’t currently understand why North Korea’s currency is inflated to worthlessness – but if not, then we, today, are to a hypothetical future global totalitarian state what the r... (read more)

Interesting idea. Have you thought more about sources of instability and weighed them? Would be interested. Others that come to mind are: - North Korean citizens must be fairly unhappy about a lot of the government and wouldn't take much to support a coup against the government - the military leadership is never perfectly aligned with the government and historically seems ready to coup under certain circumstances - having successors that can sustain autocratic rule
4Dawn Drescher1y
I’ve written this article about human rights in North Korea []. Some parts are probably outdated now, but others are not, and the general lessons hold, I think. 1. All but very few of the citizens are isolated from all information from the outside, so that they have no way to know that the rest of world isn’t actually envious of the prosperity of North Korea and they aren’t under a constant threat from the US, and the south isn’t just US-occupied territory, etc. The only things that can weaken this information monopoly are phone networks from China that extend a bit across the border, leaflets from South Korea, and similar influences from the outside. But they are localized because people are not allowed to move freely within the country. The information monopoly of the government is probably fairly complete a bit further away from the borders. But note that I haven’t been following this closely in the past 5 years. They also have this very powerful system in place where everyone is forced to snitch on everyone else if they learn that someone else knows something that they shouldn’t know or else you and your whole family can go to prison or concentration camp. The snitching is also systematically, hierarchically organized so that there are always overseers for small groups of citizens, and those overseers have their own overseers and so on, so that everyone can efficiently be monitored 24/7. A big exception to that is all the “corruption” and the gray markets. They’ve basically become the real economy of the country. But those are mostly based on Chinese currency, Chinese phones and networks, etc. So again I think black markets would be easier to prevent if there were no outside influences. 2. Without outside forces to defend against, you can concentrate completely on
1Peter S. Park1y
It's plausible that compared to a stable authoritarian nuclear state, an unstable or couped authoritarian nuclear state could be even worse (in worst-case scenario and even potentially in expected value).  For a worst-case scenario, consider that if a popular uprising is on the verge of ousting Kim Jong Un, he may desperately nuke who-know's-where or order an artillery strike on Seoul.  Also, if you believe these high-access defectors' interviews, most North Korean soldiers genuinely believe that they can win a war against the U.S. and South Korea. This means that even if there is a palace coup rather than a popular uprising, it's plausible that an irrational general rises to power and starts an irrational nuclear war with the intent to win. So I think it's plausible that prevention is an entirely different beast than policy regarding already existing stable, authoritarian, and armed states.

Facilitating relocation

Economic growth, Effective altruism

People are over-averse to moving, even if it moving leads to much better opportunities (e.g., when a volcano destroyed a fraction of nearby houses, their inhabitants who were forced to move ended up better off on earnings and education, conditional on being young; see this paper). Research and incentivization can help reduce this over-aversion. 

It is plausible that even EAs underconsider relocation.If so, it means a lot of impactful value may be achieved by convincing and facilitating  EAs' relocation to high-impact career opportunities.

5Jackson Wagner1y
Personally I believe that we should go even further, and look into using assurance contracts to help create "affinity cities" and zoom-towns based on common interests -- we should create new EA hubs in well-chosen parts of the USA, then when people move there we can experiment with various kinds of community support (childcare, etc) and exciting new forms of community governance/decisionmaking (maybe all the EAs who use a coworking space pay a fee that gets spent on community-improvement projects as decided by a quadratic-funding process). Besides the direct effect of creating new, well-functioning EA community hubs in a variety of useful locations, I think that supporting "affinity cities" in general (making them easier for other groups to start, providing a best-practices template of what they can be, etc) would have powerful effects for creating "governance competition" (cities and towns trying to improve and reform themselves in order to sell themselves as a zoom-town destination) and encouraging more cultural/legal/institutional experimentation which has positive externalities for the whole society (since everyone benefits from adopting the fruits of the most successful experiments). I have numerous additional thoughts on this subject, which unfortunately this comment is too small to contain.  Hopefully it'll become a Forum post soon.  In the meantime, just facilitating individual moves like you're saying would probably be helpful, although it would be strange to have an independent group working solely on this.  Better perhaps to build a culture where large EA organizations especially willing to help their employees with moving.  (IMO they are already trying to do this to some extent, for instance many EA orgs try to have the ability to easily hire internationally.)  This would be similar to how many EA orgs make a special effort to compensate people for time spent applying for EA jobs -- getting paid for time spent on a job application is much more common i
1Peter S. Park1y
Thanks for the great big-picture suggestions! Some of these are quite ambitious (in a good way!) and I think this is the level of out-of-the-box thinking needed on this issue.  This idea goes hand-in-hand with a previous post "Facilitate U.S. voters' relocation to swing states." For a project aiming to facilitate relocation to well-chosen parts of the US, it could be additionally impactful to consider geographic voting power as well, depending on the scale of the project.

EA/AI Hiring Round

Effective Altruism/ AI Safety

 Meet with a variety of organisations and design an short set of questions to best predict good candidates for roles. Allow anyone to take this test every 3 months and apply for a broad range of positions eg all EA ops roles in their city or all AI safety roles. Hire more, higher quality candidates.

Candidates could also be matched with orgs using an algorithm like the one used by the National Resident Matching Program [].

Funding private versions of Longtermist Political Institutions to lay groundwork for government versions

Some of the seemingly most promising and tractable ways to reduce short-termist incentives for legislators are Posterity Impact Assessments (PIA) and Futures Assemblies (see Tyler John's work). But, it isn't clear just how PIAs would actually  work, e.g. what would qualify as an appropriate triggering mechanism, what evaluatory approaches would be employed to judge policies, how far into the future policies can be evaluated. It seems like it would be relatively inexpensive to fund an organization to do PIAs in order to build a framework which a potential in-government research institute could adopt instead of having to start from scratch. The precedent set by this organization seems like it would also contribute to reducing the difficulty of advocating for longtermist agency/research institutes within government. 

Similarly, it would be reasonably affordable to run a trial Futures Assembly wherein a representative sample of a country's population is formed to deliberate over how and to what extent policy makers should consider the interests of future persons/generations. This would provide a precedent for potential government funded versions as well as a democratically legitimate advocate for longtermist policy decisions. 

Basically, EAs could lay the groundwork for some of the most promising/feasible longtermist political institutions without first needing to get legislation passed. 

Movement-building targeted at existential-risk-relevant fields' international scientific communities

Biorisk, Artificial Intelligence

Scientists of the Manhattan Project built the first nuclear bombs, the development and use of which normalized nuclear proliferation. Contrast this with bioweapons, which in principle could also have been normalized if not for the advocacy of scientists like Matthew Meselson, which led to a lasting international agreement to not develop bioweapons (Biological Weapons Convention).

Targeted efforts to build the movement of reducing catastrophic/existential risks (and longtermism in general) specifically in the international scientific communities of fields that are highly relevant to certain existential risks, whose lasting cooperation would be crucial for the non-realization of these risks, could potentially be very impactful. Some potential approaches include funding of fellowships/grants/collaboration opportunities, creating scientific societies/conferences, and organizing advocacy/outreach/petitions.

Towards Better Epistemology in Medicine

Epistemic Institutions, Values and Reflective Processes

Medicine is a field subject to an incentive landscape that can, among other issues, encourage pathological risk aversion in treatment and research, which holds back patients getting the care
with the greatest expected value to them and limits our ability as a society to adapt
to new and changing health issues such as global pandemics. Medical professionals are often trained in a narrow set of epistemic norms that lead to  slow updates on new evidence,  overreliance on individual decisionmaking, and difficulty communicating about complex tradeoffs. The unavoidable closeness to moral and ethical issues, as well as difficulties in reasoning about decisions that hold lives directly in the balance, exacerbate the problem.

We're interested in projects that address these problems, perhaps including the following:
- Literature and media that promotes truth-seeking and expected-value-thinking norms 
in medicine, whether explicit in non-fiction or training material, or in fictional settings
- Resources that seek to aggregate medical evidence relevant to a specific condition or
clinical appli... (read more)

3Jackson Wagner1y
I think this is quite important insofar as: * It could help change the existing academic culture of overly-restrictive "bioethics" around public health issues like pandemics to think more rationally about when to approve things like rapid tests and vaccines, when to impose mandates and travel bans versus not, etc. * It might lead to broader reforms [] and readjustments of focus, leading to a faster pace of developing medicines (ultimately saving many QALYs), reductions in healthcare cost, more progress in understanding aging, etc. One reason not to focus on this intervention is if you thought that general epistemology-improving efforts across academia would work well, and there's no particular reason to target medicine/bioethics/etc first.

AI safety university groups

Artificial Intelligence

Leading computer science universities appear to be a promising place to increase interest in working to address existential risk from AI, especially among the undergraduate and graduate student body. In Q1 2022, EA student groups at Oxford, MIT, Georgia Tech, and other universities have had strong success with AI safety community-building through activities such as facilitating the semester-long AGI Safety Fundamentals program locally, hosting high-profile AI safety researchers for virtual guest speaker events, and running a research paper reading group. We'd also like to see student groups which engage students with opportunities to develop relevant skills and which connect them with mentors to work on AI safety projects, with the goal of empowering students to work full-time on AI safety. We'd be happy to fund students to run AI safety community-building activities alongside their studies or to take a gap semester, or to sponsor other people to support an EA group at leading university in building up the AI safety community.


Some additional comments on why I think AI safety clubs are promising:

  • For those unfamiliar, the AGI Sa
... (read more)
Thanks for sharing this idea, super exciting to me that there is so much traction for getting junior CS people excited about AI Safety. I'd love to see much more of this happen and will likely (70%?) try to spend > a day thinking about this in the next month. If you have more ideas or pointers to look into, would highly appreciate it.

EA Crisis Fund:

Effective Altruism/X-risk

The EA Crisis Fund would respond to crisises around the world such as the current crisis of Ukranian refugees. This would help develop the capabilities of EA to respond to novel situations on short timelines, provide great publicity and build connections and credibility with government. This would increase the chance that EA would have a seat at the table in important discussions.

Potential Downside: It may be hard to respond to these crisises in a way that builds credibility without burning a lot of money.

8Jackson Wagner1y
I think if we are just jumping into the same highly-salient crises as everybody else (Ukraine today, Afghanistan yesterday, Black Lives Matter, Covid, etc), we burn a lot of money quickly at only middling effectiveness (even if we try to identify specific "most effective" interventions in each crisis, like providing oxygen tanks to Indian hospitals during their covid surge) and don't even get a huge amount of publicity because everybody else is also playing that same game (see: Elon Musk giving starlinks to Ukraine, etc). This idea maybe works better if we are trying to respond to other crises elsewhere in the world that everyone else isn't already going bananas over -- like doing famine/disaster relief in countries that aren't getting headlines, or doing pandemic early-response stuff before the world realizes it's a problem, or having some kind of "Pivotal Action Fund" on hair-trigger alert to attempt a response to the potential emergence of transformative AGI capabilities.  I'm not sure what specific approaches such a fund would use to reliably improve response times above the current situation (which is presumably "OpenPhil has the ability to spend a lot of money fast if they all start really freaking out about an emerging crisis"), but I'd certainly be interested to hear someone explore this idea.
I think the experience of the FRAPPE donor circle [], which formed in response to the first COVID wave in spring 2020, is relevant. We found that it didn't take that much money or time for us to be able to 1) get access to high-quality, often non-public information about the crisis and how it was unfolding and 2) find strong giving opportunities that not enough other people were paying attention to. I like Chris's idea because the combination of high salience + fast-moving environment is often a good one for finding high-leverage opportunities, but it's easier to intervene effectively and take on a leadership role when you have gone to the trouble of setting up some infrastructure for it in advance.

Mentors/tutors for AI safety:

AI Safety

Many people want to contribute to AI safety, but they may not be able to get up to the level where they would be able to conduct useful research. On the other hand, given time, many of these people could probably become knowledgeable enough about on a particular agenda in order to mentor potential researchers pursuing this agenda. These mentors could help people understand the reasons for and against pursuing a particular agenda, help people navigate the content that has been written on that topic, address common misconceptions and help people who are confused about a particular point.

Academic AI Safety Journal
Start an Academic Journal for AI Safety Research

Problem: There isn’t one. There should be. It would boost prestige and attract more talent to the field.

Solution: Fund someone to start one.

This has come up a few times before and is controversial.


  • more incentive for academics to work on pure safety without shoehorning their work
  • higher status
  • better peer review / less groupthink


  • risks putting safety into an isolated ghetto. Currently a lot of safety stuff is published in the best conferences
  • Journals matter 100x less than conferences in ML
  • I think academics are a minority in AIS at the moment (weighted by my subjective sense of importance anyway)


FWIW I take the first con to be decisive against it. Higher status takes a long time to build, and better peer review is (sadly) a mirage.

1Logan Riggs1y
You can still have a conference for AI safety specifically and present at both conferences, with a caveat. From NeurIPS: > Can I submit work that is in submission to, has been accepted to, or has been published in a non-archival venue (e.g. arXiv or a workshop without any official proceedings)? Answer: Yes, as long as this does not violate the other venue's policy on dual submissions (if it has one). The AI Safety conference couldn't have an official proceeding. This would still be great for networking and disseminating ideas, which is definitely worth it.
2Dawn Drescher1y
Another option may be a conference (that forms more of a Schelling point in the field than all the existing ones). These seem to be more popular in the wider field. But both solutions also have the risk that fewer people outside AI safety may read the AI safety papers.

A better overview of the effective altruism community

Effective Altruism

The effective altruism movement has grown large enough that it has become hard for any individual to have a good overview of ongoing projects and existing organizations. There is currently no central repository on what is happening across different causes and parts of the movement, which means many opportunities for coordination may be left on the table. We would like to see more initiatives like the yearly EA survey and a more detailed version of Ben Todd’s recent post that research and provide an overview of what is happening across the effective altruism movement.

Increase diversity with more ‘medium term’ plans to enable participation when travel is required

Community Building and Diversity, Values and Reflective Processes

I’m new here and it seems like many opportunities are planned with short notice. This can work well for people with lots of flexibility, but may discourage participation from people who are mid-career/working and people with families. I propose that organizations within EA encourage diversity by lengthening some planning horizons. Funding a stable hub with enough runway to have a 6 month planning horizon would be helpful for professionals and parents like my family.

Enlightenment at scale (provocative title :-) )

Values and Reflective Processes (?), X-risk (?)

A strong meditation practice promises enticing benefits to the meditator---less suffering, more control over ones attention and awareness, more insight, more equanimity. Brahmavihara practice promises the cultivation of loving-kindness, compassion, and empathetic joy. The world would be a much better place if everybody suffered less, had more equanimity, and felt strong compassion and empathy with other beings. But meditation is hard! Becoming a skilled meditator, and reaping these benefits, requires probably thousands of hours of dedicated practice. Most people will just not put in this amount of effort. But maybe it doesn't need to be this way. The field of meditation teaching seems underdeveloped, and innovative methods that make use of technology (e.g. neurofeedback) seem largely unexplored. We are interested in supporting scalable solutions that bring the benefits of meditation to many people.


  • I don't actually know if meditation really has these benefits; this would needed to be established first (there should be quite some research on this by now). It seems plausible to me that m
... (read more)
2Dawn Drescher1y
Are there high-quality safety trials for different meditation practices? I’ve heard of a variety of really bad side effects, usually from very intense, very goal-oriented meditative practice. The Dark Night that Daniel Ingram describes, the profligacy that Scott Alexander warned of, more inefficient perception that Holly Elmore experienced, etc. I have no idea how common those are and whether one is generally safe against them if one only meditates very casually… It would be good to have more certainty about that, especially since a lot of my friends are casual meditators.

Optimal 90-second pitches for EA/longtermism

Effective altruism

Longtermism is nuanced; a full discussion requires a large amount of time. It is possible that more people than currently may be interested in learning more about the movement if they are presented with a short but compelling pitch that is suited for the quickness of many people's lifestyle. (I've stated a spontaneous and very suboptimal pitch for EA on at least one occasion, which I regret.) 

Optimized 90-second or so pitches may potentially help the movement's outreach. Persuasive pitches (each focused on each of a myriad of topics/angles that the listener may be interested in) can be selected by community contests/focus groups and posted online, both for viewing and for informing movement builders' efforts. 

3Peter S. Park1y
Addendum: Found out that this is just a special case of James Ozden's 'Refining EA communications and messaging'

Monitoring and advocacy to make Zoonotic Risk Prediction projects safer

Biorisk and recovery from catastrophe

Following COVID-19, a great deal of funding is becoming available for "Zoonotic Risk Prediction" projects, which intend to broadly sample wildlife pathogens, map their evolutionary space for pandemic potential, and publish rank-ordered lists of the riskiest pathogens. Such work is of dubious biodefence value, presents a direct risk of accidental release in the field and lab, and the resulting information is a clear biosecurity infohazard.

We would be excited to fund projects to collect, monitor, and report on the activities of these projects. ZRP projects have multiple components- field sampling, computational modelling, and lab characterization - each of which carry distinct risks and leaves an information trail. Monitoring and reporting on open source information associated with ZRP projects could disincentivize the riskiest aspects of this work, target resources for event surveillance and early warning of accidental release, and provide material for advocacy efforts.

There is some overlap with portions of the BWC project, but I think this is best tackled as a separate body of work/by a different team (due to radically different OPSEC, deception, and scrutiny profiles). I've thought about this a fair bit and am happy to discuss offline.

Better Reporting on Other Countries' Perspectives

Epistemic Institutions

(Refinement of better news)

It's very hard for a regular person to understand what the Russian or Chinese or Turkish perspectives on events are from reading Western media. It would be valuable to have a high-quality mainstream news media source that takes special effort to make sure that this is explored, including by having on-staff anthropologists. This would increase understanding between countries and reduce the chance of Great Power Conflict.

4Dawn Drescher1y
I wonder whether Larissa MacFarquhar (author of Strangers Drowning) may be someone to talk to about this. She managed to understand Julia Wise so well that I learned new things about myself from reading her chapter in the book. The only other people who can do that are close friends of mine who’ve known me for years. Maybe Larissa is just similar to Julia and so had this level of insight, but maybe she’s also exceptionally gifted at perspective-taking.

Agent Foundations and Philosophy Engagement Fund:

AI Safety

Agent Foundations research may potentially be important for AI Safety, but currently it has received very little engagement from the philosophical community. This fund would offer funding and or scholarships for people who want to engage with these ideas in an academic philosophical context. This project aims to improve clarity about whether this research actually worthwhile and, if so, to help make progress on these problems.

EA Founders Camp

Effective altruism, empowering exceptional people

The EA community is scaling up, and funding ambitious new projects. To support continued growth of new organisations and projects, we would be excited to fund an organisation to run EA Founders Camps. These events would provide an exciting, sparky environment for (1) Potential founders to meet co-founders, (2) Founders to hear about and generate great ideas for impactful projects and organisations, (3) Founders to get key training tailored to their project area, (4) Founders to build a support network of other new and existing founders, (5) Founders to connect with funders and advisers.

Regulating AI consciousness.

Artificial intelligence,  Values and reflective process

The probability that AIs will be capable of conscious processing in the incoming decades is not negligible. With the right information dynamics, some artificial cognitive architecture could support conscious experiences. The global neural workspace is an example of a leading theory of consciousness compatible with this view. Furthermore, if it turns out that conscious processing improves learning efficiency then building AI capable of consciousness might become an effective path toward more generally capable AI. Building conscious AIs would have crucial ethical implications given their high expected population. To decrease the chance of bad moral outcomes we could follow two broad strategies. First, we could fund policy projects aiming to work with regulators to ban or slow down research that poses a substantial risk to building conscious AI. Regulations slowing the arrival of conscious AIs could be in place until we gain more moral clarity and a solid understanding of machine consciousness. For example, philosopher Thomas Metzinger advocated a moratorium on synthetic phenomenology in ... (read more)

Vetting and matchmaking organization of consultants and contractors for EA founders
Empowering Exceptional People, Effective Altruism

Founders of new projects, charities and other EA-aligned organisations can have an extremely high impact. These individuals tend to suffer more from issues such as overwhelm, burnout, etc. which can easily lead them to have much less impact both short and long-term. A potential intervention against this is decreasing the decision-making overload by helping them outsource some of their decision-making.

We'd love to see an organization that offers vetting and matchmaking for independent consultants and contractors in several relevant areas of decision-making for these people so they can tap into knowledge and expertise faster with less effort and cognitive load.
This service can be considered and expansion of this idea by aviv.

Open-source intelligence agency

Great Power Relations

Create an organization that will collect and analyze open-source intelligence information on critical topics (e.g. US nuclear arsenal, more examples below) and publish it online.

Many documents on US nuclear arsenal and military activities were obtained through Freedom of Information Act. Still, they were never analyzed properly because it is a lot of tedious work that journalists do not have the capacity or incentive to do. Standard open-source intelligence gathering methods can provide even more information. As a result, there is only a limited public understanding of important sources of x-risk.

Possible subjects of investigation:

  1. State of nuclear arsenals of US and Russia.
  2. Military developments of artificial intelligence
  3. Propaganda and hacking capabilities of Russia and China.
  4. State of AI Armsraces, both between states and between companies.
  5. Monitoring activities of secret services of both Russian and USA. (For example, to better estimate the capabilities of GRU, NSA, and others).
  6. (bioweapons has its own comment)

Scaling successful policies

Biorisk and Recovery from Catastrophe, Economic Growth

Information flow across institutions (including national governments) is far from optimal, and there could be large gains in simply scaling what already works in some places. We’d love to see an organization that takes a prioritized approach to researching which policies are currently in place to address major global issues, identifying which of these are most promising to bring to other institutions and geographies, and then bringing these to the institutions and geographies where they are most needed.

Reduce meat consumption

Biorisk, Moral circle expansion

Research and efforts to reduce broad meat consumption would help moral circle expansion, pandemic prevention, and climate change mitigation. Perhaps messaging from the pandemic-prevention angle (in addition to the climate change angle and the moral circle expansion angle) may help. 

Slowing AI Contingency Planning

AI Governance

AI progress has been especially rapid over the last 4 years. Because of visible success in diverse tasks by OpenAI, DeepMind, and others, it is likely that even more money and talent will flow into accelerating AI progress in the future. However, there is substantial controversy over whether AI safety/alignment technology is advancing as quickly as capability. Given that, we are interested in funding work on 1) identifying nonviolent ways to reversibly slow AI progress and 2) more research into whether and when such an intervention would be net-good. 


Platform Democracy Institutions

Artificial Intelligence (Governance), Epistemic Institutions, Values and Reflective Processes, Great Power Relations

Facebook/Meta, YouTube/Google, and other platforms make incredibly impactful decisions about the communications of billions. Better choices  can significantly impact geopolitics, pandemic response,  the incentives on politicians and journalists, etc. Right now, those decisions are primarily in the hands of corporate CEO’s—and heavily influenced by pressure from partisan and authoritarian governments aiming to entrench their own power. There is an alternative: platform democracy. In the past decade, a new suite of democratic processes have been shown to be surprisingly effective at navigating challenging and controversial issues, from nuclear power policy in South Korea to abortion in Ireland. 

Such processes have been tested around the world, overcome the pitfalls of elections and referendums, and can work at platform scale. They enable the creation of independent ‘people’s mandates’ for platform policies—something invaluable for the impacted populations, well-meaning governments which are unable to act on speech, and ... (read more)

Polis lobbying

Political is a tool for mapping coalitions (mentioned in this 80,000 Hours podcast).  Rather than running standard polls on issues, large Polis pols could be run, as Taiwan does. These would seek to build solutions which hold broad support before taking them to lobbyists.

Backup communication systems

Biorisk and Recovery from Catastrophe

In the event of GCRs, conflicts or disasters, communication systems are key to sensemake and coordinate effectively. They prevent chaos and further escalation of conflicts. Today, there are many threats to the global communication infrastucutre including EMPs, widespread cyber attacks,  and solar flares.

Metaculus Competitor


Prediction markets don't incentivise long term questions and the Good Judgement Open has slow question creation. This leaves Metaculus as the only place to forecast questions over long time horizons. This is too important a problem to have a single organisation solving. At least one more forecasting organisation should exist to try and build the infrastructure necessary to take forecasts,  improve individual forecasting, display track records and make 5 - 1000 year forecasts.

Securing offices and schools against SARS-3


The COVID-19 pandemic has demonstrated failures of our scientific, political, and epistemic institutions, but also of our physical structures. We believe that accurate and high-quality designs of offices and schools to be secure against pathogen spread of airborne viruses can be a) directly useful, b) potentially generalize well to future pandemics, and c) provide the necessary training ground for building more robust and ambitious projects in the future, including large-scale civilizational refuges.

We picked offices and schools to limit the threat model and surface area, but we're in theory excited about designs that can contain pathogen spread in any well-trafficked built environment.

2Jackson Wagner1y
Is there a way to get more leverage on this?  Maybe: - Research new sterilization tech (like shining UV-C light horizontally across the ceiling in a way that cleans the air but doesn't harm people) so that buildings can be retrofitted more easily, without redoing the whole HVAC system?  This would count under FTX's project idea #8. - Lobbying for better air-filtration systems to be made a requirement for schools and offices as a matter of government budgets (for schools) and regulation (for offices)?  I'm sure we could swing a state or local ballot proposition in a covid-cautious and wildfire-plagued place like California.  
I think we're bottlenecked more on really good designs than on the politics, but I'm not sure. I also vaguely have this cached view that a lot of whether built-environment innovations are used in practice depends on things that look more like building codes than office politics, but this is a pretty ill-formed view that I have low confidence in. I guess that I sort of believe all three should be done in a sane world, and which things we ought to prioritize in practice will depend on a combination of "POV of the universe" modeling and personal fit considerations of whoever wants to implement any of these considerations.

Consulting on best practices around info hazards

Epistemic Institutions, Effective Altruism, Research That Can Help Us Improve

Information about ways to influence the long-term future can in some cases give rise to information hazards, where true information can cause harm. Typical examples concern research into existential risks, such as around potential powerful weapons or algorithms prone to misuse. Other risks exist, however, and may also be especially important for longtermists. For example, better understanding of ways social structures and values can get locked in may help powerful actors achieve deeply misguided objectives. 

We would like to support an organization that can develop a set of best practices and consult with important institutions, companies, and longtermist organizations on how best to manage information hazards. We would like to see work to help organizations think about the tradeoffs in sharing information. How common are info hazards? Are there ways to eliminate or minimize downsides? Is it typically the case that the downsides to information sharing are much smaller than upsides or vice versa?

Comprehensive, personalized, open source simulation engine for public policy reforms

Epistemic Institutions, Economic Growth, Values and Reflective Processes

Policy researchers apply quantitative modeling to estimate the impacts of immigration reform on GDP, child benefits on fertility, safety net reform on poverty, carbon pricing on emissions, and other policies. But these analyses are typically narrow, impersonal, inflexible, and closed-source, and the public can rarely access the models that produce them.

We'd like to see a general simulation engine—built with open source code and freely available to researchers and the public—to estimate the impact of a wide variety of public policy reforms on a wide variety of outcomes, using a wide variety of customizable parameters and assumptions. Such a simulation engine could power analyses like those above, while opening up policy analysis to more intricate reforms, presented as a technology product that estimates impacts on society and one's own household.

A common technology layer for public policy analysis would promote empiricism across institutions from government to think tanks to the media. Exposing households to society-wide and pers... (read more)

Create an organization doing literature reviews and research on demand

Values and Reflective Processes,  Effective Altruism

Create a research organization that will offer literature reviews and research to other EA organizations. They will focus on questions that are not theory-heavy and can be approached by a generalists without previous deep knowledge of the field. Previous examples of such research are publications of AI Impacts or literature reviews by Luke Muehlhauser.
Besides research itself, this is useful also for:
- it frees up the time of senior researchers
- It can be a good training place for junior researchers.
- it may enable a larger infusion of valuable ideas from academia.

I think that this is a good idea and something that READI []could be interested in supporting. We have extensive experience doing reviews as research consultant  and in providing related training (both as volunteers and professionals). One related idea that some of us are exploring, is developing a sort of 'micro-course and credential' to i) train EAs to do reviews and ii) curate teams of credential junior researcher who can to support the undertaking of literature reviews under the supervision an expert.
Would this be another organization like Rethink Priorities, or is it different from what they are doing? (Note: I don't think this space is crowded yet, so even if it is another organization doing the same things, it could still be very helpful!)

Creating more EA relevant credentials
Movement building

EA wants to equip young people with knowledge and motivation to improve the long-term future by providing high quality online educational resources for anyone in the world to learn about effective altruism and longtermism. Most young people follow established education paths (e.g., school, university, and professional courses) and seek related credentials during this time. There are relatively few credentialed courses or activities which provide exposure to EA ideals and core capabilities. We would therefore like to fund more of these. For instance, these might include talent based scholarships (e.g., a ‘rising social impact star award’), cause related Olympiads (e.g.,  AI safety), MOOCs/university courses (e.g., on causes or, key skill sets, with an EA tie-in), and EA themed essay writing competitions (.e.g., asking high school students to write about 'the most effective ways to improve the world’ and giving awards to 

the best ones).

New EA incubation and scaling funds and organisations
Movement building, coordination, coincidence of wants problems, & scaling

Charity entrepreneurship, Y Combinator, Rocket Internet and similar have had notable and disproportionate economic and social impacts and accelerated the dissemination of innovative ideas. The EA community has also called for more founders. We would therefore like to support EA and social impact funds that initiate, or scale relevant initiatives such as charities (e.g., tax-deductible EA charity funds,  long term future fund equivalents or research institutes) 

Job application support for underrepresented groups
Increasing diversity in EA, Effective Altruism

Underrepresented groups usually face additional (or exacerbated) challenges in job applications: language barriers, impostor-syndrome, smaller networks, etc. that affect their application success. There are organisations within the EA ecosystem that provide career coaching but none provides dedicated, on demand support with job applications. 

We'd love to see an organisation that provides ongoing support to people from underrepresented groups in job applications including: finding the right opportunities, preparing application documents, preparing for interviews, etc. so they are more likely to land high impact roles.

Economic growth

Work with developing countries to buy an area of land to form an EA special economic area. This can be a place where EA can congregate and innovate in IT and other fields. It can also be a place where EA can demonstrate new policies, technologies and pioneer new ways of thinking.

EA could expand on this idea to build communities in remote places that are likely to survive extinction events. It will provide a good opportunity to test technology that will be used in any space colonies.

Generous prizes to attract young top talent to EA in big countries

Effective altruism

Prizes are a straightforward way to attract top talent to engage with EA ideas. They also require relatively low human capital or expertise and therefore are conceivably scalable for different countries. Through a nationwide selection process optimized for raw talent, ability to get things done, and altruistic alignment, an EA prize could quickly make the movement become well-known and prestigious in big countries. High school graduates and early university students would probably be the best target audience. The prize could come with a few strings attached, such as participating in a two-week-long EA fellowship, or with more intense commitments, such as working for a year on an EA-aligned project. Brazil and India are probably the best fit, considering their openness to Western ideas and philanthropic investment (in comparison to China and Russia). Other candidates may include the Philippines, where EA groups have been relatively successful, Indonesia, Argentina, Nigeria, and Mexico.

Fund/Create training for mental health workers

Effective Altruism

A limiting reagent in health care in Canada right now is that there aren’t enough psychologists/psychiatrists/ mental health workers. People don’t have access to these services and end up in the emergency department and crashing the health system in other ways. Mental health is fundamental for participation in societal roles, and highly conscientious people are at risk and children are waiting years for assessments (like for ADHD) which can change the course of their lives.

Psychiatry is one of the least well paid medical specialties, it takes many years to train psychologists and psychiatrists.

I propose looking at funding the training of the mental health workforce, as well as lobbying to have mental health services to be included as essential health care services.

Website for coordinating independent donors and applicants for funding

Empowering exceptional people, effective altruism

At EAG London 2021, many attendees indicated in their profiles that they were looking for donation opportunities. Donation autonomy is important to many prospective donors, and increasing the range of potential funding sources is important to those applying for funding. A curated website which allows applicants to post requests for funding and allows potential donors to browse those requests and offer to fully or partially fund applicants, seems like an effective solution.

Nuclear arms reduction to lower AI risk

Artificial Intelligence and Great Power Relations

In addition to being an existential risk in their own right, the continued existence of large numbers of launch-ready nuclear weapons also bears on risks from transformative AI. Existing launch-ready nuclear weapon systems could be manipulated or leveraged by a powerful AI to further its goals if it decided to behave adversarially towards humans. We think understanding the dynamics of and policy responses to this topic are under-researched and would benefit from further investigation.

Strongly agree with this. There are only a handful of weapons that threaten catastrophe to Earth’s population of 8 billion. When we think about how AI could cause an existential catastrophe, our first impulse shouldn’t be to think of “new weapons we can’t even imagine yet”. We should secure ourselves against the known credible existential threats first. Wrote up some thoughts about doing this as a career path here: []
On the flip side, you could make part of your 'pivotal act [,pivotal%20acts,-%E2%80%9D%20%E2%80%94%20in%20particular%2C%20actions]' be the neutralisation of all nuclear weapons [,neutrinos%20straight%20through%20the%20Earth.].

Pilot emergency geoengineering solutions for catastrophic climate change

Research That Can Help Us Improve

Toby Ord puts the risk of runaway climate change causing the extinction of humanity by 2100 at 1/1000, a staggering expected loss. Emergency solutions, such as seeding oceans with carbon-absorbing algae or creating more reflective clouds, may be our last chance to prevent catastrophic warming but are extraordinarily operationally complex and may have unforeseen negative side-effects. Governments are highly unlikely to invest in massive geoengineering solutions until the last minute, at which point they may be rushed in execution and cause significant collateral damage. We’d like to fund people who can:

  • Identify and pilot at large scale top geoengineering initiatives over the next 5-10 years to develop operational lessons. E.g. promote algae growth in a large, private lake, launch a small cluster of mirrors into space
  • Develop advanced supercomputer models, potentially with input from the above pilots, of the potential negative side-effects of geoengineering solutions
  • Identify and pilot harm-mitigation responses for geoengineering solutions

Epistemic status: there seems to be rea... (read more)

I thought China has already done some low-key geoengineering? []
1Rory Fenton1y
Thanks for sharing!    My initial sense is that China's method is focused on controlling rainfall, which might mitigate some of the effects of climate change (e.g. reduce drought in some areas, reduce hurricane strength) but not actually prevent it. The ideas I had in mind were more emergency approaches to actually stopping climate change either by rapidly removing carbon (e.g. algae in oceans) or reducing solar radiation absorbs on the Earth's surface (making clouds/oceans more reflective, space mirrors). 

Incremental Institutional Review Board Reform

Epistemic Institutions, Values and Reflective Process

Institutional Review Boards (IRBs) regulate biomedical and social science research. In addition to slowing and deterring life-saving biomedical research, IRBs interfere with controversial but useful social science research, eg, Scott Atran was deterred from studying Jihadi terrorists; Mark Kleiman was deterred from studying the California prison system, and a Florida State University IRB cited public controversy as a reason to deter research. We would like to see a group focused on advocating for plausible reforms to IRBs that allow more social science research to be performed. Some plausible examples:

  1. Prof. Omri Ben-Shahar’s proposal to replace exempt IRB reviews with an electronic checklist or
  2.  Zachary Schrag’s proposal (from Ethical Imperialism) that Congress remove social science research from OHRP jurisdiction by amending the National Research Act of 1974. 

Concrete steps to these goals could be: 

  1. sponsoring a prize for the first university that allowed use of Prof. Omri Ben-Shahar’s electronic checklist tool;
  2.  setting up a journal for “Deterred Social Science Resea
... (read more)

Longtermism movement-building/election/appointment efforts, targeted at federal and state governments

Effective altruism

Increasing knowledge of and alignment with longtermism in government by targeted movement-building and facilitating the election/appointment of sympathetic people (and of close friends and family of sympathetic people) could potentially be very impactful. If longtermism/EA becomes a social norm in, say, Congress or the Washington 'blob', we could benefit from the stickiness of this social norm.

Studying stimulants' and anti-depressants' long-term effects on productivity and health in healthy people (e.g. Modafinil, Adderall, and Wellbutrin)

Economic Growth, Effective Altruism

Is it beneficial or harmful for long-term productivity to take Modafinil, Adderall, Wellbutrin, or other stimulants on a regular basis as a healthy person (some people speculate that it might make you less productive on days where you're not taking it)? If it's beneficial, what's the effect size? What frequency hits the best trade-off between building up tolerance vs short-term productivity gains? What are the long-term health effects? Does it affect longevity?

Some people think that taking stimulants regularly provides a large net boost to productivity. If true, that would mean we could relatively cheaply increase the productivity of the world and thereby increase economic growth. In particular, it could also increase the productivity of the EA community (which might be unusually willing to act on such information), including AI and biorisk researchers.

My very superficial impression is that many academics avoid researching the use of drugs in healthy people and that there is a bias against taking medic... (read more)

Sub-extinction event drills, games, exercises

Civilizational resilience to catastrophes

Someone should build up expertise and produce educational materials / run workshops on questions like 

  1. Nuclear attacks on several cities in a 1000 mile radius of you, including one within 100 miles. What is your first move? 
  2. Reports of a bioweapon in the water supply of your city. What do you do? 
  3. You're a survivor of an industrial-revolution-erasing event. What chunks of knowledge from science can be useful to you? After survival, what are the steps to rebuilding? 
  4. 6 billion people died and the remaining billion are uniformly distributed throughout the planet's former population centers. How can you build up robustness of basic survival, food and water production, shelter, etc.?
  5. (for the IT folks) 5 years after number 4, basic needs are largely met, and scavengers have filled a garage with old laptops and computer parts. Can you begin rebuilding the internet to connect with other clusters around the world? 

Differentially distributing these materials/workshops to people who live in geographical areas likely to survive at all could help rebuilding efforts in worlds where massive sub-extinction events occur. 

Centralising Information on EA/AI Safety

Effective Altruism, AI Safety

There are many list of opportunities available in EA/AI Safety and many lists of what organisations exist. Unfortunately these lists tend to get outdated. It would be extremely valuable to have a single list that is up to date and filterable according to various criteria. This would require someone being paid to maintain these part-time.

Another opportunity for centralisation would be to create an EA link shortener with pretty URLs. So for example, you'd be able to type in to see information on careers or to jump to the forum.

Notes: I own the URL so I'd be able to donate it.

Physical AI Safety 

Drawing from work done in the former Soviet Union to improve safety in their bioweapons and nuclear facilities (e.g. free consultations and install of engineering safety measures, at-cost upgrades of infrastructure such as ventilation and storage facilities, etc), developing a standard set of physical/infrastructure technologies to help monitor AI Development labs/hardware and provide physical failsafes in the event of unexpectedly rapid takeoff (e.g., a FOOM scenario). Although unlikely, some standard guidelines modifying current best-practices for data center safety (e.g., restrictions on devices, physical air gaps between critical systems and the broader world, extensive onsite power monitoring and backup generators) could be critical to prevent anxiety over both physical and digital security from encouraging risk-taking behaviors by AI Development programs (Such as rushing builds, hiding locations, inappropriate dual-use or shared facilities which decrease control over data flows). In particular, physical low-tech hardware such as low-voltage switches have already provided demonstrable benefit in safeguarding high-tech, high-risk activity (See the Goldsb... (read more)

Acquire and repurpose new AI startups for AI safety

Artificial intelligence

As ML performance has recently improved there is a new wave of startups coming. Some are composed of top talent, carefully engineered infrastructure, a promising product, well-coordinated teams, with existing workflows and management capacity. All of these are bottlenecks for AI safety R&D.

It should be possible to acquire some appropriate startups and middle-sized companies. Examples include HuggingFace, AI21, Cohere, and smaller, newer startups. The idea is to repurpose the mission of some select companies to align them more closely with socially beneficial and safety-oriented R&D. This is sometimes feasible since their missions are often broad, still in flux, and their product could benefit from improving safety and alignment.

Trying this could have very high information value. If it works, it has enormous potential upside as many new AI startups are being created now that could be acquired in the future. It could potentially  more than double the size of the AI alignment R&D.

Paying existing employees to do safety R&D seems easier than paying academics. Academics often like to follow the... (read more)

Thanks, I think that's a really interesting and potentially great idea. I'd encourage you to post it as a short stand-alone post, I'd be interested in hearing other people's thoughts.

A center applying epistemic best practices to predicting & evaluating AI progress

Artificial  Intelligence and  Epistemic Institutions

Forecasting and evaluating AI progress is difficult and important. Current work in this area is distributed  across multiple organizations or individual researchers, not all of whom possess (a) the technical expertise, (b) knowledge & skill in applying epistemic best practices,  and (c) institutional legitimacy  (or otherwise suffer from cultural constraints). Activities of the center could include providing services to AI groups (e.g. offering superforecasting training or prediction services), producing bottom-line reports on "How capable is AI system X?", hosting adversarial collaborations, pointing out deficiencies in academic AI evaluations, and generally pioneering "analytic tradecraft" for AI progress.

Tradable impact certificates

Effective Altruism, Research That Can Help Us Improve, Economic Growth

Issuing and trading impact certificates can popularize and normalize impact investment and profitable strategic research among the world's economic influencers. Then, economic growth will have an approximately good direction, only the relative popularization of impact certificates management/incentivization would remain.

Better understanding the needs of organisational leaders
Coincidence of wants problems

In EA, organisational leaders and potential workers often don't have good information about each other’s needs and offerings (See EA needs consultancies). The same is true for researchers who might like to do research for organisations but don't know what to do. We would like to fund work to help to resolve this. This could involve collecting advanced market commitments for funders (e.g., org group x would pay up to x for y hours of design time next year, on average).  It could involved identifying unknowns for key decision makers in EA in relevant areas (e.g., instructional decision-making, longtermism, or animal welfare) which could be used to develop a research agendas and kickstart research.

Organization to push for mandatory liability insurance for dual-use research

Biorisk and Recovery from Catastrophe

Owen Cotton-Barratt for the Global Priorities Project in 2015:

Research produces large benefits. In some cases it may also pose novel risks, for instance work on potential pandemic pathogens. There is widespread agreement that such ‘dual use research of concern’ poses challenges for regulation.

There is a convincing case that we should avoid research with large risks if we can obtain the benefits just as effectively with safer approaches. However, there do not currently exist natural mechanisms to enforce such decisions. Government analysis of the risk of different branches of research is a possible mechanism, but it must be performed anew for each risk