The FTX Foundation's Future Fund is a philanthropic fund making grants and investments to ambitious projects in order to improve humanity's long-term prospects.
We have a longlist of project ideas that we’d be excited to help launch.
We’re now announcing a prize for new project ideas to add to this longlist. If you submit an idea, and we like it enough to add to the website, we’ll pay you a prize of $5,000 (or more in exceptional cases). We’ll also attribute the idea to you on the website (unless you prefer to be anonymous).
All submissions must be received in the next week, i.e. by Monday, March 7, 2022.
We are excited about this prize for two main reasons:
- We would love to add great ideas to our list of projects.
- We are excited about experimenting with prizes to jumpstart creative ideas.
To participate, you can either
- Add your proposal as a comment to this post (one proposal per comment, please), or
- Fill in this form
Please write your project idea in the same format as the project ideas on our website. Here’s an example:
Early detection center
Biorisk and Recovery from Catastrophes
By the time we find out about novel pathogens, they’ve already spread far and wide, as we saw with Covid-19. Earlier detection would increase the amount of time we have to respond to biothreats. Moreover, existing systems are almost exclusively focused on known pathogens—we could do a lot better by creating pathogen-agnostic systems that can detect unknown pathogens. We’d like to see a system that collects samples from wastewater or travelers, for example, and then performs a full metagenomic scan for anything that could be dangerous
You can also provide further explanation, if you think the case for including your project idea will not be obvious to us on its face.
Some rules and fine print:
- You may submit refinements of ideas already on our website, but these might receive only a portion of the full prize.
- At our discretion, we will award partial prizes for submissions that are proposed by multiple people, or require additional work for us to make viable.
- At our discretion, we will award larger prizes for submissions that we really like.
- Prizes will be awarded at the sole discretion of the Future Fund.
We’re happy to answer questions, though it might take us a few days to respond due to other programs and content we're launching right now.
We’re excited to see what you come up with!
(Thanks to Owen Cotton-Barratt for helpful discussion and feedback.)
Acquire and repurpose new AI startups for AI safety
Artificial intelligence
As ML performance has recently improved there is a new wave of startups coming. Some are composed of top talent, carefully engineered infrastructure, a promising product, well-coordinated teams, with existing workflows and management capacity. All of these are bottlenecks for AI safety R&D.
It should be possible to acquire some appropriate startups and middle-sized companies. Examples include HuggingFace, AI21, Cohere, and smaller, newer startups. The idea is to repurpose the mission of some select companies to align them more closely with socially beneficial and safety-oriented R&D. This is sometimes feasible since their missions are often broad, still in flux, and their product could benefit from improving safety and alignment.
Trying this could have very high information value. If it works, it has enormous potential upside as many new AI startups are being created now that could be acquired in the future. It could potentially more than double the size of the AI alignment R&D.
Paying existing employees to do safety R&D seems easier than paying academics. Academics often like to follow the... (read more)
High-quality human data
Artificial Intelligence
Most proposals for aligning advanced AI require collecting high-quality human data on complex tasks such as evaluating whether a critique of an argument was good, breaking a difficult question into easier subquestions, or examining the outputs of interpretability tools. Collecting high-quality human data is also necessary for many current alignment research projects.
We’d like to see a human data startup that prioritizes data quality over financial cost. It would follow complex instructions, ensure high data quality and reliability, and operate with a fast feedback loop that’s optimized for researchers’ workflow. Having access to this service would make it quicker and easier for safety teams to iterate on different alignment approaches
Some alignment research teams currently manage their own contractors because existing services (such as surgehq.ai and scale.ai) don’t fully address their needs; a competent human data startup could free up considerable amounts of time for top researchers.
Such an organization could also practice and build capacity for things that might be needed at ‘crunch time’ – i.e., rapidly producing moderately la... (read more)
Thanks for running this competition, looks like there are plenty of great ideas to choose from!
I submitted my entry on improving human intelligence through non-invasive brain stimulation through the Google form, it said my entry was recorded but I got no email confirmation.
Has anyone else submitted through the Google Form, and did they also get no email confirmation?
Does anyone know when the winners of the competition will be announced?
Just came to stay that this ideas competition really turned me on - I loved it. I hope this becomes an ongoing community ‘suggestion box’, perhaps monitored once a month.
I understand that one could write a blog post with an idea, but I think this is an even better low barrier way of getting ideas quickly.
Personally, this competition helped me realize that I have a different lens that many EAs, and that my ideas and skills could be valued. Thank you.
We see now that dictatorships slow down the progress of humanity and can plausibly threaten large-scale nuclear wars. Dictatorships are often toppled from inside with public protests (e.g. Poland 1988-1989, Tunisia 2011) but public protests face the coordination problem. There are many people willing to protest in dictatorships (e.g. Russia), but protesting in large groups is both more efficient and less risky because law enforcement has the cap on the number of detained. Idea: develop an app to sign-up for a prospective protest in advanc... (read more)
Sad that I missed this! Only saw this the day after it closed.
Language models for detecting bad scholarship
Epistemic institutions
Anyone who has done desk research carefully knows that many citations don't support the claim they're cited for - usually in a subtle way, but sometimes a total nonsequitur. Here's a fun list of 13 features we need to protect ourselves.
This seems to be a side effect of academia scaling so much in recent decades - it's not that scientists are more dishonest than other groups, it's that they don't have time to carefully read everything in their sub-sub-field (... while maintaining their current arms-race publication tempo).
Take some claim P which is below the threshold of obviousness that warrants a citation.
It seems relatively easy, given current tech, to answer: (1) "Does the cited article say P?" This question is closely related to document summarisation - not a solved task, but the state of the art is workable. Having a reliable estimate of even this weak kind of citation quality would make reading research much easier - but under the above assumption of unread sources, it would also stop many bad citations from being written in the first place.
It is very hard to answer (2) "Is the cited ar... (read more)
A service/consultancy that calculates the value of information of research projects
Epistemic Institutions, Research That Can Help Us Improve
When undertaking any research or investigations, we want to know whether it's worth spending money or time on it. There are a lot of research-type projects in EA and the best way to evaluate and prioritise them is to calculate their value of information (VOI). However, VoI calculations can be complex and we need to build a team of experts that can form a VoI consultancy or service provider.
Examples of use cases:
1... (read more)
Potential Test Case for AGI
Attempt to simulate an artificial general intelligence using Ouijably
Low-odds it works, but I thought if you could put enough people on a spirit board it might exhibit behaviour similar to an oracle-type AGI. This implementation(https://github.com/ably-labs/ouija) means it wouldn't take much organising to attempt. Maybe tweak so participants are predicting the direction the planchette will move rather than relying on the ideomotor effect. I thought the idea would be outside of the rationalist's window of consideration as somethin... (read more)
Situational Analysis Agency
Epistemics
When events of great global importance occur, they often have a bearing on EA projects. Sometimes EAs will want to do something in response. Take for example, the invasion of Ukraine, the coronavirus pandemic and supply chains. At the moment, most of the investigation of these issues is conducted on the side by EAs who are busy with other projects. It would be great to have some researchers available to investigate these issues on short notice so that we are better able to navigate these situations.
Effective Altruism Promotional Materials
Effective Altruism
We are looking to invest in the production of high-quality materials for promoting Effective Altruism and Effective Altruism cause areas including posters, brochures and booklets. Effective Altruism is heavily focused on the fidelity of transmission, so these materials should be designed to avoid low-quality transmission. This could be achieved by distributing materials that promote opportunities for deeper engagement or by designing materials very carefully. Such an organisation would likely conduct studies and focus groups to understand the effectiveness of the material being distributed and whether it is maintaining its fidelty.
A center applying epistemic best practices to predicting & evaluating AI progress
Artificial Intelligence and Epistemic Institutions
Forecasting and evaluating AI progress is difficult and important. Current work in this area is distributed across multiple organizations or individual researchers, not all of whom possess (a) the technical expertise, (b) knowledge & skill in applying epistemic best practices, and (c) institutional legitimacy (or otherwise suffer from cultural constraints). Activities of the center could include providing services to AI groups (e.g. offering superforecasting training or prediction services), producing bottom-line reports on "How capable is AI system X?", hosting adversarial collaborations, pointing out deficiencies in academic AI evaluations, and generally pioneering "analytic tradecraft" for AI progress.
Historical investigation on the relation between incremental improvements and paradigm shifts
Artificial Intelligence
One major question that heavily influences the choice of alignment research directions is the degree to which incremental improvements are necessary for major paradigm shifts. As the field of alignment is largely preparadigmatic, there is a high chance that we may require a paradigm shift before we can make substantial progress towards aligning superhuman AI systems, rather than merely incremental improvements. The answer to this question det... (read more)
Funds for study efficient logistic or run a logistic company
We can see logistics is one of the bottlenecks for goods and services. It makes uneven distribution of resources. Especially in pandemic and lockdowns, not enough delivery guys leads to shortage of foods. In outbreak area, there's food shortages while a surplus in other regions. Digitalisation can help information transfer in a speedy and cheap way. But what about real products delivery? It's something more than autonomous vehicles. Human beings is a fragile part of the procedure. Now, in hk,... (read more)
EA from the ground up
Effective Altruism
Intellectual movements tend to develop by building upon the work of the previous generation and rejecting some of its foundational assumptions. We'd be keen to see an experiment to accelerate this. We'd suggest that the first step would be to identify the assumptions that are underlying EA or specific EA cause areas or specific EA strategies and try to figure out when these break. The project would then focus on those that are most likely to be false, particularly those which would be high impact if false. Efforts wou... (read more)
EA Hotel / CEEALAR except at EA Hubs
Effective Altruism
CEEALAR is currently located in Blackpool, UK. It would be a lot more attractive if it were in e.g. Oxford, the Bay Area, or London. This would allow guests to network with local EAs (as well as other smart people, of which there are plenty in all of the above cities). In as far as budget is less of a constraint now and in as far as EA funders are already financing trips to such cities for select individuals (for conferences and otherwise), it seems an EA Hotel would similarly be justified on the same grounds. (E.g. intercontinental flights can sometimes be more expensive than one month's rent in those cities)
Happy Altruist Hotel
I have submitted this idea: creating a Happy Altruist Hotel.
My project idea focusses on improving the wellbeing of effective altruists by creating a center for that. I am thinking of a physical location, preferably in a nature environment. I will call it (for now) " Happy Altruist Hotel". The way I see it, the happy altruist hotel is a place where all kinds of programs, workshops, retreats and trainings will be organized for (aspiring) effective altruists.
The happy altruist hotel will be a place where EA's come together for inspiration,... (read more)
An EA Space Agency
Effective Altruism, Space Governance
Let’s build an organization which formulates and implements space programs, missions, and systems, which are aimed at the highest-priority things that humanity can be doing in space. There is currently no space organization, public or private, which formulates and implements programs and missions aligned solely with doing the most good, in an impartial and longtermist sense. There are many organizations which do some or much good, such as NASA, ESA, SpaceX, and others, but there is no example today whic... (read more)
Improving Critical Infrastructure
Effective Altruism
Some dams are at risk of collapse, potentially killing hundreds of thousands. The grid system is very vulnerable to electromagnetic pulse attack. Infrastructural upgrades could prevent sudden catastrophes from failure of critical systems our civilization runs on.
Legalization of MDMA & psychedelics to reduce trauma and cluster headaches
Values and Reflective Processes, Empowering Exceptional People
Millions of people have PTSD that causes massive suffering.
MDMA and psychedelics are being legalized in the U.S., and there are both non-profit and for-profit organizations working in this space. Making sure everyone who wants it has access, via more legalization, and subsidization, would reduce the amount of trauma, which could have knock-on benefits not just for them but the people they interact with.
- https://fo
... (read more)Build an intranet for the effective altruism community
Effective Altruism, Empowering Exceptional People
If effective altruism is going to be "the last social movement the world needs" it will need to operate differently from past movements in order to last longer and reach more people. Given that coordination is a crucial element for success within a distributed global network, a movement intranet could improve coordination on projects, funding and research and build a greater sense of community. An intranet would also help the movement (1) consolidat... (read more)
Build an Infrastructure Organization for The EA Movement (TEAM)
Effective Altruism, Empowering Exceptional People
Many high impact organizations in effective altruism have expressed issues with sourcing operations talent which takes time away from the key programs these charities provide, reducing overall impact. An infrastructure organization could provide operational support and build valuable tools that would alleviate the burden from these meta charities and streamline processes across organizations to improve movement coordination. This organizati... (read more)
Research into Goodhart’s Law
Artificial Intelligence, Epistemic Institutions, Values and Reflective Processes, Economic Growth, Space Governance, Effective Altruism, Research That Can Help Us Improve
Goodhart’s Law states: “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes”, or more simply, "When a measure becomes a target, it ceases to be a good measure.”
The problem of ‘Goodharting’ seems to crop up in many relevant places, including alignment of artificial intelligence and the social and economic... (read more)
Reframe U.S. college EA chapters as an alternative to Greek life
Values and Reflective Processes, Empowering Exceptional People, Effective Altruism
Following the model of Alpha Phi Omega, the largest coed service fraternity in the U.S. with ~335 chapters and 400,000 alumni, reframing EA chapters as social organizations may help with recruitment and retention. It could also encourage a broader range of activities for chapters to run throughout the year including things like hosting workshops for other students on how to think about careers, hosting film scree... (read more)
Publish an EA-inspired magazine like Time Magazine's "Time for Kids" (TFK)
Empowering Exceptional People, Values and Reflective Processes, Effective Altruism
Time for Kids has almost 2 million subscribers and has been used by educators for over 25 years to introduce elementary students to issues in science, history and civic engagement, while empowering students to take action and have a positive impact on the world. An EA-oriented magazine could do something similar by introducing students to topics like current pressing issues, relevant career pathwa... (read more)
Antarctic Colony as Civilizational Backup
Recovery from Catastrophe
Antarctica could be a good candidate for a survival colony. It is isolated, making it more likely to survive a nuclear war, pandemic, or roving band of automated killer drones. It is tough, making it easier to double up as a practice space for a Mars colony. Attempting to build and live there at a larger scale than has been done may spur some innovations. One bottleneck here that may likely need resolving is how to get cheaper transportation to Antarctica, which currently relies on flying there or a limited number of specialized boats.
Research into the dual-use risks of asteroid safety
Space Governance
There is a small base rate of asteroids/comets hitting the Earth naturally. There are efforts out there to deflect/destroy asteroids if they were about to hit Earth. However, based on the relative risk of anthropogenic vs natural risk, we think that getting better at manipulating space objects is dual-use as it would allow malevolent actors to weaponize asteroids, and that this risk could be orders of magnitudes larger. We want to see research on what kinds of asteroid defense techniques ar... (read more)
Creating a giving what we can for volunteering time and bequesting (last minute)
Given the success of GWWC we would like to see organisation emerge to seek pledges and build communities around the effective use of resources , but in different ways (e.g., time rather than mone or by bequesting rather than donating) [inspired by this].
[fairly unsure, would be interested in thoughts]
Facilitate global cooperation via economic relationships and shared ownership
Values and Reflective Processes
We live in an economically connected world that is characterized by mutually beneficial trades. On top of that, countries are generally heavily invested in diverse financial securities of other countries. This way, economic progress in one country is generally to the benefit of the whole international community. Consequently there are strong incentives for peaceful coexistence, internalization of proble... (read more)
Making significant improvements to the EA wiki (last minute submission)
See this for a range of ideas for improving the EA wiki which could be funded. I'd suggest that all changes made to the wiki should also be replicated and linked across the EA ecosystem and onto normal Wikipedia.
Creating materials for alignment onboarding
Artificial Intelligence
At present, the pipeline from AI capabilities researcher to AI alignment researcher is not very user friendly. While there are a few people like Rob Miles and Richard Ngo who have produced excellent onboarding materials, this niche is still fairly underserved compared to onboarding in many other fields. Creating more materials for a field has the advantage that because there are different formats that different people find more helpful, having more increases the likelihood that something wor... (read more)
An EA insurance and finance fund to make it easier for people to fund and take significant personal risk for important social benefits, e.g., due to early career change, founding a startup etc.
Movement building & Helping exceptional people
Risk avoidance is a major reason why people don't change careers or take risks relating to having greater impact. We'd therefore like to see more attempts to establish financial services which can help to reduce risk and promote more rapid and impactful impact amongst exceptional individuals. We note that there may be advantages in combining long term investing initiatives for patient philanthropy with insurance offerings.
Getting former hiring managers from quant firms to help with alignment hiring
Artificial Intelligence, Empowering Exceptional People
Despite having lots of funding, alignment seems to not have been very successful at attracting top talent to date. Quant firms, on the other hand, have become known for very successfully acquiring talent and putting them to work on difficult conceptual and engineering problems. Although buy-in to alignment before one can contribute is often cited as a reason, this is, if anything, even more of a problem for quant firms, since very few people are inherently interested in quant trading as an end. As such, importing some of this know how could help substantially improve alignment hiring and onboarding efficiency.
A living 'cause prioritisation flowchart' /Better visualisation template or graphic design copy for EA communicators (quick submission)
[Inspired by this comment]
EA has many aims and a complex causal logic behind these aims. Visualisation helps to explain this better. Flow charts are one established way we do this. These could be used effectively in many communication settings but there is a coordination problem as most individual actors who need such a chart also lack sufficient expected ROI or experience to create one. We would therefore welcome more work... (read more)
Research on Competitive Sovereignties
Governance, New Institutions, Economic Growth
The current world order is locked in stasis and status quo bias. Enabling the creation of new jurisdictions, whether via charter cities, special economic zones, or outright creation of new lands such as seasteading, could allow more competition between countries to attract subscriber-citizens, increasing welfare.
It would also behoove us to think about standards for international interoperability in a world where '1000 nations bloom'. Greater decentralization of power could in... (read more)
Decentralized incentives for resilient public goods after global catastrophic risk
Recovery from Catastrophe
Using cryptoeconomics to bootstrap the incentivization of a resilient grid via which further cryptoeconomic incentives induce the bottom-up production of survival bunkers and other post-catastrophe public goods that could survive GCRs such as nuclear war. Figuring out how to reward people for preparing themselves, moreso if they help others or build critical infrastructure that lasts.
A Facebook comment I wrote that I am copy-pasting, I will like... (read more)
Research on how to minimize the risk of false alarm nuclear launches
Effective Altruism
Preventing false alarm nuclear launches (as Petrov did) via research on the relevant game theory, technological improvements, and organization theory, and disseminating and implementing this research, could potentially be very impactful.
On malevolence: How exactly does power corrupt?
Artificial Intelligence / Values and Reflective Processes
How does it happen, if it happens? Some plausible stories:
Evaluating large foundations
Effective Altruism
Givewell looks at actors: object-level charities, people who do stuff. But logically, it's even more worth scrutinising megadonors (assuming that they care about impact or public opinion about their operations, and thus that our analysis could actually have some effect on them).
For instance, we've seen claims that the Global Fund, who spend $4B per year, meet a 2x GiveDirectly bar but not a Givewell Top Charity bar.
This matters because most charity - and even most good charity - is still not by EAs or run on EA lines. Also, even big cautious foundations can risk waste / harm, as arguably happened with the Gates Foundation and IHME - it's important to understand the base rate of conservative giving failing, so that we can compare hits-based giving. And you only have to persuade a couple of people in a foundation before you're redirecting massive amounts.
More Insight Timelines
In 2018, the Median Group produced an impressive timeline of all of the insights required for current AI, stretching back to China's Han Dynasty(!)
The obvious extension is to alignment insights. Along with some judgment calls about relative importance, this would help any effort to estimate / forecast progress, and things like the importance of academia and non-EAs to AI alignment. (See our past work for an example of something in dire need of an exhaustive weighted insight list.)
Another set in need of collection are more general breakthroughs - engineering records broken, paradigms invented, art styles inaugurated - to help us answer giant vague questions about e.g. slowdowns in physics, perverse equilibria in academic production, and "Are Ideas Getting Harder to Find?"
Our World in Base Rates
Epistemic Institutions
Our World In Data are excellent; they provide world-class data and analysis on a bunch of subjects. Their COVID coverage made it obvious that this is a very great public good.
So far, they haven't included data on base rates; but from Tetlock we know that base rates are the king of judgmental forecasting (EAs generally agree). Making them easily available can thus help people think better about the future. Here's a cool corporate example.
e.g.
“85% of big data projects fail”;
“10% of people refuse to be vaccinated because of fearing needles (pre-COVID so you can compare to the COVID hesitancy)”;
"11% of ballot initiatives pass"
“7% of Emergent Ventures applications are granted”;
“50% of applicants get 80k advice”;
“x% of applicants get to the 3rd round of OpenPhil hiring”, "which takes y months";
“x% of graduates from country [y] start a business”.
MVP:
Later, Q... (read more)
I think this is neat.
Perhaps-minor note: if you'd do it at scale, I imagine you'd want something more sophisticated than coarse base rates. More like, "For a project that has these parameters, our model estimates that you have a 85% chance of failure."
I of course see this as basically a bunch of estimation functions, but you get the idea.
Biorisk and information hazard workshops for iGEM competitors
Biorisk and Recovery from Catastrophe, Empowering Exceptional People
iGEM competitions are interdisciplinary synthetic biology competitions for students. They bring together the best and brightest university students with a considerable interest in synthetic biology. They already have knowledge and skills in bioengineering and many of them will likely choose it as a career path and will be very good at it. Educating them on biorisks and especially information hazards would therefore be a great contribution to safeguarding. They could also be introduced to EA ideas and rationalist approaches in general, bringing talented young people on board.
Accident reporting in biology research labs
Biorisk and Recovery from Catastrophe
Currently, accident reporting is framed as an unpleasant and largely unimportant chore, even though there’s evidence of lab leaks causing massive harm. Encouraging research groups to report their accidents in a fast and thorough way could therefore be very impactful. We could build a reporting system in a variety of ways, this research in itself would be a good thing to fund. A potential system could be to implement insurance policies that require efficient and honest documenta... (read more)
Organising/sponsoring Hackathons
Epistemic institutions, empowering exceptional people
Many highly skilled programmers are lured into the private sector either to work for prestigious companies or found a startup, often with little positive impact. We’d like to see these people instead working for or starting their own EA aligned organisations.
To encourage this, we’d be excited to fund an organisation that involves themselves with programming hackathons, to scout for highly creative and skilled individuals and groups. This could mean sponsoring existing hackathons or running their own.
Evaluating powerful political groups and people (political parties/activists/…)
values and reflective processes
Currently GiveWell provides people with a guide for effective giving. We could apply a similar model to provide a guide for effective voting and advocacy.
We’d like to see an organisation that evaluates particularly powerful political individuals/groups/parties and advocates for those that align with EA values.
We could evaluate them on things like:
Commitment to using evidence and careful reasoning to work out how to maximise good (particularly long-... (read more)
Prestigious forecasting tournaments for students
Epistemic institutions, empowering exceptional people
To scale up forecasting efforts, we will need a large body of excellent forecasters to recruit from. Forecasting is a skill that improves over time, and it takes time to build a track record to distinguish excellent forecasters from the rest - particularly on long-term questions. Additionally, forecasting builds generally useful research and rationality skills, and supports model-building and detailed understanding of question topics. Therefore, getting stu... (read more)
Align university careers advising incentives with impact
Effective altruism
Students at top universities often have lots of exposure to a limited set of career paths, such as consulting and finance. Many graduates who would be well-suited to high-impact work don’t consider it because they are just unaware of it. Universities have little incentive to improve this state of affairs, as the eventual social impact of graduates is hard to evaluate and has little effect on their alma mater (with some notable exceptions). We would therefore be excited to fund effort... (read more)
EA Founders Camp
Effective altruism, empowering exceptional people
The EA community is scaling up, and funding ambitious new projects. To support continued growth of new organisations and projects, we would be excited to fund an organisation to run EA Founders Camps. These events would provide an exciting, sparky environment for (1) Potential founders to meet co-founders, (2) Founders to hear about and generate great ideas for impactful projects and organisations, (3) Founders to get key training tailored to their project area, (4) Founders to build a support network of other new and existing founders, (5) Founders to connect with funders and advisers.
Find promising candidates for “Cause X” with an iterative forecast-guided thinktank
Epistemic institutions
How likely is it that the EA community is neglecting a cause area that is more pressing than current candidates? We are fairly confident in the importance of the community’s current community areas, but we think it’s still important to keep searching for more candidates.
We’d be excited to fund organisations attacking this problem in a structured, rigorous way, to reduce the chance that the EA community is missing huge opportunities.
We propos... (read more)
Help high impact academics spend more time doing research
Empowering exceptional people
Top academic researchers are key drivers of progress in priority areas like biorisk, global priorities research and AI research. Yet even top academics are often unable to spend as much time as they want to on their research.
We’d be excited to fund an organisation providing centralised services to maximise research time for top academics, while minimising the overheads of setting up these systems for academics. It might focus on:
(1) Funding and negotiating teaching ... (read more)
Space's preferences and objectives research
Space Governance, Artificial Intelligence, Epistemic Institutions, Values and Reflective Processes, Great Power Relations, Research That Can Help Us Improve
In order to govern space well, one needs to understand its preferences and objectives: for example, that of dark energy and dark matter. These can be then weighted by an AI approved under the veil of ignorance by all entities, and solutions that maximize the weighted sum, while centralizing wellbeing and systemic stability, selected, and supported by any space ... (read more)
Systemic change marginal cost-effectiveness program estimation and evaluation
Effective Altruism, Research That Can Help Us Improve, Artificial Intelligence
Instead of focusing on single, the ones which are measurable and highlighted by academia, outcomes, one can focus on advancing systemic change (institutionalizing safe positive systems) by selecting programs with the highest (and lowest) marginal cost-effectiveness, considering impact costs development. Then, impact can be increased by 1) advising resource shifts from low to high cost-effectiveness progr... (read more)
Machine olfaction for disease detection
Biorisk and Recovery from Catastrophe
Dogs can be trained to recognize the smell of Covid-19 and many other diseases. However, this takes a lot of time. It might be possible in the very near future to build robotic noses (machine olfaction), that work as well as a dog's. This would mean that once one neural net has been trained to recognize a new pathogen, the software could easily be distributed around the globe. Sensors in public places could then pick up in real time whether someone infectious was close by. This wou... (read more)
Research institute focused on civilizational lock-in
Values and Reflective Processes, Economic Growth, Space Governance, Effective Altruism
One source of long-term risks and potential levers to positively shape the future is the possibility that certain values or social structures get locked in, such as via global totalitarianism, self-replicating colonies, or widespread dominance of a single set of values. Though organizations exist dedicated to work on risks of human extinction, we would like to see an academic or independent institute focused on other events that could have an impact on the order of millions of years or more. Are such events plausible, and which ones should be of most interest and concern? Such an institute might be similar in structure to FHI, GPI, or CSER, drawing on the social sciences, history, philosophy, and mathematics.
Consulting on best practices around info hazards
Epistemic Institutions, Effective Altruism, Research That Can Help Us Improve
Information about ways to influence the long-term future can in some cases give rise to information hazards, where true information can cause harm. Typical examples concern research into existential risks, such as around potential powerful weapons or algorithms prone to misuse. Other risks exist, however, and may also be especially important for longtermists. For example, better understanding of ways social structures and values can get locked in may help powerful actors achieve deeply misguided objectives.
We would like to support an organization that can develop a set of best practices and consult with important institutions, companies, and longtermist organizations on how best to manage information hazards. We would like to see work to help organizations think about the tradeoffs in sharing information. How common are info hazards? Are there ways to eliminate or minimize downsides? Is it typically the case that the downsides to information sharing are much smaller than upsides or vice versa?
Modern Public Forums
Values and Reflective Processes, Epistemic Institutions, Effective Altruism
Violence begins when conversations stop. We'd love to see a renaissance of ancient Greek agoras or Roman fora which offered their citizens a public space where they could gather, study and discuss current events as well as everything else that is timelessly important for the future of humanity. In modern times, such places have become increasingly scarce and social media do not constitute a suitable replacement since many critical layers of human communicati... (read more)
Comprehensive, personalized, open source simulation engine for public policy reforms
Epistemic Institutions, Economic Growth, Values and Reflective Processes
Policy researchers apply quantitative modeling to estimate the impacts of immigration reform on GDP, child benefits on fertility, safety net reform on poverty, carbon pricing on emissions, and other policies. But these analyses are typically narrow, impersonal, inflexible, and closed-source, and the public can rarely access the models that produce them.
We'd like to see a general simulation engine—built with open source code and freely available to researchers and the public—to estimate the impact of a wide variety of public policy reforms on a wide variety of outcomes, using a wide variety of customizable parameters and assumptions. Such a simulation engine could power analyses like those above, while opening up policy analysis to more intricate reforms, presented as a technology product that estimates impacts on society and one's own household.
A common technology layer for public policy analysis would promote empiricism across institutions from government to think tanks to the media. Exposing households to society-wide and pers... (read more)
Tradable impact certificates
Effective Altruism, Research That Can Help Us Improve, Economic Growth
Issuing and trading impact certificates can popularize and normalize impact investment and profitable strategic research among the world's economic influencers. Then, economic growth will have an approximately good direction, only the relative popularization of impact certificates management/incentivization would remain.
Commercial marketing analysis
Artificial Intelligence, Epistemic Institutions, Economic Growth
What tricks to manipulate humans does AI use? For example, why are the glossy balls used increasingly more often in unrelated advertisement? The color gradients to captivate attention (day&night), physical or mental space intrusion narrated as giving one power to defend themselves from such issues or offend others, racial and gender hierarchical power stereotypes in conjunction with images that narrate positive relationships, etc. AI would love it, since analys... (read more)
EA community's trading bot
Artificial Intelligence, Effective Altruism
If you have the capital to invest while being able to influence the market and you are just aligned with EA, why would you not get a trading bot. EAs who are the world's top experts on AI can code it, possibly using the knowledge of their respective institutions, and of course impact is generated. It saves time just think about it.
Effects of humanitarian development on peace and conflict
Great Power Relations, Values and Reflective Processes, Effective Altruism, Biorisk and Recovery from Catastrophe
Is it not that conflict stems from suboptimal institutions, such as those which value aggression and disregard, for the lack of better alternatives known, so can be prevented by general humanitarian development? It can be that when people are more able to contribute and benefit from others' upskilling rather than competing for scarce resources because it is challenging to increase efficien... (read more)
Wellbeing determinants' understanding
Research That Will Help Us Improve
Without understanding the fundamentals of individuals' wellbeing, you cannot build institutions based in and optimizing for wellbeing, even if you have a lot of attention and prediction capacity: you do not know what to advocate for or research. So, you should fund a team of neuroscientists, sociologists, and anthropologists, to provide an interdisciplinary interperspective understanding of what, fundamentally, makes individuals happy. This should be understood fundamentally (e. g. safe... (read more)
Just looked at the website and the following probably fits under talent-search / innovative educational experiments. Apologies for the formatting (this is from a private doc of ideas some time ago, and I currently don't have the time to reformat it / I'm also travelling with spotty internet).
Project 1:
Title:
Longtermist movement building via "cash transfers" (i.e. grants/fellowships) to talented (high-school) students (from developing countries) to support them to work on the world's most pressing problems.
Idea:
Identify talented (e.g. top 0.01%) high ... (read more)
Yes this sounds plausible. I'm generally excited to think about ways humanity can survive and/or flourish after civilizational collapse and other large-scale disasters.
EA storytelling
Research That Can Help Us Improve, Values and Reflective Processes, Effective Altruism
The stronger the stories that EA tells are, the more people will be convinced to do something about EA in their own lives. We’re interested in funding people with a proven track record in storytelling, including generating viral content, to create EA stories that could reach millions of people.
(Potentially extends existing Project Ideas ‘A fund for movies and documentaries’ and 'Critiquing our approach'.)
Project ideas from this page that are relevant to this idea:
EA-themed Superhero Graphic Novel / Shounen Anime / K Drama (jknowak)
Solving institutional dysfunction
Values and Reflective Processes
Thousands of institutions have potential to do more good, but are hampered by dysfunctions such as excess bureaucracy, internal politics, and misalignment of the values they and their employees hold with their actions. Often these dysfunctions are well-known by their employees, but still persist.
We're excited to fund proposals to study institutional dysfunction and investigate solutions, as well as tools to monitor dysfunctions that lead to poor EA outcomes, and to empower employees to solve t... (read more)
Research into why people don't like EA
Research That Can Help Us Improve
Many people have heard of EA and weren’t convinced. We want to understand why, so that we can find approaches to convince them. If we can win more people over to EA, we can directly increase the impact that EA has in the world.
We’re excited to fund proposals to research why people do and don’t like EA, and the approaches that are most effective in winning people over to EA.
(Potentially extends existing Project Idea 'Critiquing our approach'.)
AI Safety Academic Conference
Technical AI Safety
The idea is to fund and provide logistical/admin support for a reasonably large AI safety conference along the lines of Neurips etc. Academic conferences provide several benefits: 1) Potentially increasing the prestige of an area and boosting the career capital of people who get accepted papers. 2) Networking and sharing ideas, 3) Providing feedback on submitted papers and highlighting important/useful papers. This conference would be unusual in that the work submitted shares approximately t... (read more)
Promote ways that suppress status seeking
Great Power Relations, Economic Growth
Status seeking is associated with massive economic inefficiencies (waste production, economic inequality,..). The zero sum game nature of status seeking also puts a toll on individual well being and consequently on suboptimal ways the societies function.
In the political domain, status seeking can lead to wars (as the recent developments illustrate).
The EA community should invest into institutions/research/solutions leading to diverting from status seeking.
Screen and record all DNA synthesis
Biorisk and Recovery from Catastrophe
Screening all DNA synthesis orders for potentially serious hazards would reduce the risk that a dangerous biological agent is engineered and released. Robustly recording what DNA is synthesized (necessarily in an encrypted fashion) would allow labs to prove that they had not engineered an agent causing an outbreak. We are interested in funding work to solve technical, political and incentive problems related to securing DNA synthesis.
Meta note: there are already some cool EA-aligned projects related to this, such as SecureDNA from the MIT Media Lab and Common Mechanism to Prevent Illicit Gene Synthesis from NTI/IBBIS. Also, this one is not an original idea of mine to an even greater extent than the others I've posted.
The Billionaire Nice List
Philanthropy
A regularly updated list of how much impact we estimate billionaires have created. Billionaires care about their public image, people like checking lists. Let's attempt to create a list which can be sorted by different moral weights and incentivises billionaires to do more good.
Group psychology in space
Space governance
When human colonies are established in outer space, their relationship with Earth will be very important for their well-being. Initially, they’re likely to be dependent on Earth. Like settler colonies on Earth, they may grow to desire independence over time. Drawing on history and research on social group identities from social psychology, researchers should attempt to understand the kind of group identities likely to arise in independent colonies. As colonies grow they’ll inevitably form independent group identities, but depending on relationships with social groups back home, these identities could support links with Earth or create antagonistic relationships with them. Attitudes on Earth might also vary from supportive, exclusionary, or even prejudiced. Better understanding intergroup relations between Earth powers and their settler colonies off-world could help us develop equitable governance structures that promote peace and cooperation between groups.
Fund publicization of scientific datasets
Epistemic institutions
Scientific research has made huge strides in the last 10 years towards more openness and data sharing. But it is still common for scientists to keep some data proprietary for some length of time, particularly large datasets that cost millions of dollars to collect like, for instance, fMRI datasets in neuroscience. More funding for open science could pay scientists when their data is actually used by third parties, further incentivizing them to make data not only accessible but useable. Op... (read more)
Research raising sanity waterline
It seems that teaching the general public rationality tools may cause more polarisations. That is because many ideas seem to be used primarily for argument-winning instead of truth-seeking.
There is a risk that some ideas will make people less rational when teaching rationality. For example, Eliezer Yudkowsky wrote an article Knowing About Biases Can Hurt People.
Scott Alexander uses the term Symmetric and Asymmetric Weapons for a similar idea: Some thinking-tools are more useful for winning arguments tha... (read more)
Create an organization doing literature reviews and research on demand
Values and Reflective Processes, Effective Altruism
Create a research organization that will offer literature reviews and research to other EA organizations. They will focus on questions that are not theory-heavy and can be approached by a generalists without previous deep knowledge of the field. Previous examples of such research are publications of AI Impacts or literature reviews by Luke Muehlhauser.
Besides research itself, this is useful also for:
- it frees up the time of senior researchers
- It can be a good training place for junior researchers.
- it may enable a larger infusion of valuable ideas from academia.
Research differential technological progress and trajectory changes
Research That Can Help Us Improve, Values and Reflective Processes
The idea of Differential technological progress (DTP) may be a crucial consideration for many at-first-glance good ideas like:
- improving scientific publishing
- increasing GDP
- increasing average intelligence
But given its importance, there hasn`t been much research and publications on GTP.
Central question for research is how to use DTP to prioritize interventions. Examples of subquestion to research are:
- when in the past there were intentional trajectory changes.
- what subgoals seem to be good when DTP is considered.
- and so on.
Find good ways to distribute books to people with high potential
Epistemic Institutions, Effective Altruism
This project has two parts:
1) find people with high potential, especially students.
2) find a good way to distribute books on world problems to them.
Ad 1: Examples:
- students in low and medium income countries may have a higher demand for English books
- participants on STEM olympiads
- people with SAT scores > x
- students in selective schools
Ad 2: It is important to do it in a nice, non-preaching way.
One possible implementation is a book club that sends out a book every two months, with regular online meetups for its readers.
Create and curate educational materials on EA-related topics
Effective Altruism
EA Fellowship and EA Handbook took existing resources and curated them into a good introduction to EA. Do something similar with different formats and subjects.
I.e., create:
- Fellowships
- Reading lists.
- record existing courses in academia
- and so on
With a goals to:
- make it easy to take up new fields.
In fields like:
- Rationality
- Bioweapons
- Forecasting
- and so on.
Buying and building products and services that influence culture
Movement building
Mass media producers such as news services, computer games and books and movies studios, etc, heavily influence culture. Culture in turn creates and influences norms for collective values (e.g., trust in various groups and institutions) and behaviours (e.g., prosocial or antisocial behaviour). Collective values and behaviour then influence social outcomes. We'd therefore welcome work to build or acquire mass media producers and use these to promote relevant values and behaviou... (read more)
Advocacy for digital minds
Artificial Intelligence, Values and Reflective Processes, Effective Altruism
Digital sentience is likely to be widespread in the most important future scenarios. It may be possible to shape the development and deployment of artificially sentient beings in various ways, e.g. through corporate outreach and lobbying. For example, constitutions can be drafted or revised to grant personhood on the basis of sentience; corporate charters can include responsibilities to sentient subroutines; and laws regarding safe artificial intelligence can be tailored to consider the interests of a sentient system. We would like to see an organization dedicated to identifying and pursuing opportunities to protect the interests of digital minds. There could be one or multiple organizations. We expect foundational research to be crucial here; a successful effort would hinge on thorough research into potential policies and the best ways of identifying digital suffering.
Lobbying architects of the future
Values and Reflective Processes, Effective Altruism
Advocacy often focuses on changing politics, but the most important decisions about the future of civilization may be made in domains that receive relatively less attention. Examples include the reward functions of generally intelligent algorithms that eventually get scaled up, the design of the first space colonies, and the structure of virtual reality. We would like to see one or more organizations focused on getting the right values considered by influential decision-makers at institutions like NASA and Google. We would be excited about targeted outreach to promote consideration of aligned artificial intelligence, existential risks, the interests of future generations, and nonhuman (both animal and digital) minds. The nature of this work could take various forms, but some potential strategies are prestigious conferences in important industries, retreats including a small number of highly-influential professionals, or shareholder activism.
Assessment companies
Epistemic Institutions, Empowering Exceptional People
... (read more)Much better narratives of the future and understanding of “Utopia”
Many efforts to discuss “utopia” are unproductive, and often the word is disliked. This is despite most people caring deeply about the future and how it is shaped. Improving communication of the future is important for practical reasons, like improving public understanding of longtermist projects. Also, limited understanding of preferences over even the medium term future could unduly influence work and limit progress toward better outcomes more broadly. This project includes research and de... (read more)
Cheap, lifesaving treatments
Epistemic institutions; Artificial Intelligence; Economic Growth; Effective Altruism; Research That Will Help Us Improve
Hundreds of existing, low-cost, and widely available generic drugs could be repurposed as effective treatments for additional indications. Yet this major opportunity to improve outcomes for patients suffering from cancer and other diseases while lowering healthcare costs is being ignored due to a market failure. We are interested in funding innovative solutions for bringing repurposed generic drugs to widesprea... (read more)
Materials Informatics
For centuries, we have identified some of the most important materials in modern society by chance. Some of these materials include steel, copper, rubber, etc.
With the current grand challenges of today's world, the discovery and scaling of new advanced materials are necessary to create the impact. (After all, everything around us is materials).
I'd like to see more funding on materials informatics and guidance/regulation in materials informatics so we're not creating any advanced materials or nanomaterials that could cause a catastrophe.
High Quality Outward-Facing Communications Organization
This creates a new communications organization that deeply understands outside media and attitudes, and reports events to the community. The organization will expertly provide content and services tailored to EAs and their projects on demand. This organization is a servant, an expression of the community and respects Truth. Carefully created, this organization should be invaluable as EA grows many times and into new domains and competencies.
Imagine a new megaproject. How do we talk about a giant n... (read more)
Facilitate interdisciplinarity in governmental applications of social science
Values and Reflective Processes, Economic Growth
At the moment, governmental applications of social science (where, for example, economists who use the paradigm of methodological individualism are disproportionately represented) could benefit from drawing on other fields of social science that can fill potential blind spots. The theory of social norms is a particularly relevant example. Also, behavioral scientists and psychologists could potentially be very helpful in improving the... (read more)
Training Course for Professional AI Ethicists on Longterm Impacts
Artifical Intelligence
Most AI Ethicists focus on the short-term impacts of AI rather than the longer term impacts. Many might be interested in a free professional development course covering this topic. Such a course should cover a variety of perspectives, including that of prominent AI Safety skeptics.
EA Berkeley Hostel
Effective Altruism
Every week, EAs pass through Berkeley and someone needs to pay around $200 a night to house them or scramble to find a couch they can crash on. This becomes increasingly complicated when someone finds a trial run offer and needs to stay another week than expected or even find a job offer and suddenly need to rush to find housing. Currently, there exists NO hostel (or even a hotel room that costs less than a couple hundred bucks) even close to Berkeley, much less an EA hostel. A hostel in Berkeley would allow flexibility ... (read more)
EA services consultancy network or organisation (early draft)
Movement building and resolving coordination problems
Considerable need for support for small projects on tech, design etc. Many effective charities lacking key ingredients for improvements. Many good ideas never get off the ground due to lack of technical expertise. Can do surveys of movement leaders and also scale up as needed when there is more demand. Incl:
Tech support organisation
Associated media and PR services for EA organisations to publicise work via media
Content creation for SEO and medi... (read more)
EA Micro Schools
Effective Altruism
We would be excited to fund projects that make it easier to start up an EA-aligned, accredited private school.
As EA matures, there will be more and more parents. Kids of self-identified EAs are likely to be smart and neurodivergent, and may struggle with the default schooling system. They're also likely to grow into future adult EAs. Remote work options will free up location choice, and there could be major community-building gains if parents can easily find their ideal school in an EA hub.
Variation: develop an EA stream o... (read more)
Using the EA survey to answer key research questions.
Research & movement building
We would like to support work by EA researchers to preregister hypothesises and measures to test (with ethics approval) i) in the EA survey (maybe as a non-mandatory final section) and ii) with the public to compare the results. For instance, this could to explore how different demographics, personality types and identities (e.g., identification as social justice activist/climate change activist) interact with different moral views or arguments for key EA behaviours ... (read more)
A replication lab or project to replicate and expand key EA research
Movement building & conceptual dissemination
EA outreach and strategy is supported by a growing pool of social psychology research exploring EA related topics (e.g., appeals to change dietary or donation choice, understand moral views or related interventions etc). However, much social psychology research doesn’t replicate or varies depending on audience when retested. It's therefore possible that some key findings and theories that guide EA are more or less valid or robust than c... (read more)
EA movement building evaluation support
Movement building
Effective social movement building requires us to understand what is working well and why. However, there is very limited information on how to track EA groups performance, and on how different approached perform on in achieving key outcomes. We would like to support work to address this, for instance, to help with standardisation of EA group metrics and the creation of simple tracking systems (e.g., distribution of a single sheet and related data visualisation program for tracking attendees across all groups).
An annual reports on cryptocurrency activity and philanthropy
Public influence & attention economy
Lots of money has been invested in cryptocurrency, and it seems likely that this will continue to be the case. The growth in the market has led to many new millionaires, some of whom are quite atypical and young relative to high wealth individuals in other areas. Cryptocurrency philanthropic norms appear to differ from the main population of donors and are not as well established. Thus, identifying, and publicising key trends and opportunities in this area ... (read more)
Bountied Rationality Website
Effective Altruism
Oftentimes great ideas fail to find funding through a grant because those who come up with a great proposal are not the right people to complete the proposal. An inducement prize platform separates who comes up with ideas (proposers) and those who complete the ideas (bounty hunters), thereby allowing the best ideas to be elevated based on the quality of the idea itself. It also makes it easier to find others working on the same project because there can be a "competitors and collaborators" tab that shows who el... (read more)
Better understanding the needs of organisational leaders
Coincidence of wants problems
In EA, organisational leaders and potential workers often don't have good information about each other’s needs and offerings (See EA needs consultancies). The same is true for researchers who might like to do research for organisations but don't know what to do. We would like to fund work to help to resolve this. This could involve collecting advanced market commitments for funders (e.g., org group x would pay up to x for y hours of design time next year, on average). It could involved identifying unknowns for key decision makers in EA in relevant areas (e.g., instructional decision-making, longtermism, or animal welfare) which could be used to develop a research agendas and kickstart research.
Understanding public awareness and opinion of key EA values, the EA movement, and/or key organisations
Movement building & conceptual dissemination
What the public thinks of EA is quite relevant to many key outcomes, including movement building. We would therefore like to fund work to understand public trends on topics such as key values (e.g., longtermism, cosmopolitanism or resource maximisation), attitudes towards activist movement 'brands' (e.g., EA, vegan activism, extinction rebellion), and awareness and attitude towards key EA organisations ... (read more)
Better understanding the role of behaviour science and systems thinking in producing key EA outcomes
Social change and movement building
Behaviour and systems change are core to all EA outcomes. We would therefore like to support research to provide a better understanding of the causes of EA relevant behaviour (e.g., career change, donation, involvement in EA or social movement), at both psychological and structural levels.
---
See this for some examples of ideas potentially relevant to AI governance or safety
Better understanding social movements
Movement building & Conceptual dissemination
People involved with social movements are important collaborators for the EA movement. However, there is relatively little high quality survey work to understand how these groups differ and overlap. We would therefore like to fund research to regularly survey members of social movements to better understand them. For instance, this could involve understanding i) aggregations of behaviours and attitudes (e.g., what different identities, demographics/geographies/groups... (read more)
Creating more EA relevant credentials
Movement building
EA wants to equip young people with knowledge and motivation to improve the long-term future by providing high quality online educational resources for anyone in the world to learn about effective altruism and longtermism. Most young people follow established education paths (e.g., school, university, and professional courses) and seek related credentials during this time. There are relatively few credentialed courses or activities which provide exposure to EA ideals and core capabilities. We would therefore like to fund more of these. For instance, these might include talent based scholarships (e.g., a ‘rising social impact star award’), cause related Olympiads (e.g., AI safety), MOOCs/university courses (e.g., on causes or, key skill sets, with an EA tie-in), and EA themed essay writing competitions (.e.g., asking high school students to write about 'the most effective ways to improve the world’ and giving awards to
the best ones).
Create EA focused communication initiatives
Movement building & Conceptual dissemination
Optimising EA Movement building and coordination requires confident and effective communicators for compelling and high fidelity conceptual dissemination. To help to improve communication across member of the EA community, we would welcome applications for courses focused on helping EAs to communicate better, for instance, modelled on the toastmasters program or the Dale Carnegie course.
Converting key EA research outputs into academic publications
Conceptual dissemination
Academic publications are considered to be significantly more credible than other types of publications. However, many EA aligned organisations such as Rethink Priorities produce valuable research that is never published. To help address this, we would like to fund academic publication support organisations, to help organisation which are unaffiliated with universities to get ethics approval, write grants, produce academic research outputs etc.
Creating EA aligned research labs
Conceptual dissemination
Academic publications are considered to be significantly more credible than other types of publications. Many academics with outsized impacts lead publication labs. These hire many junior researchers to help maximise the return on the knowledge and experience of a more senior researcher. We would like to support attempts to found and scale up academic research labs aligned with relevant cause areas.
Creating more EA aligned journals or conferences
Movement building
Academic publications are considered to be significantly more credible than other types of publications. However, the academic publication system is highly misaligned with key EA values (e.g., efficiency and intellectual novelty/impartiality). We would therefore like to encourage initiatives to start, influence or acquire influential academic journals or conferences to enable EA to have better academic impacts towards our desired outcomes.
Just FYI, here is copy explaining a relate... (read more)
Supporting Longitudinal studies of Effective Altruists
Movement building
One significant part of the EA movement is helping individuals to have maximal impact across their lifecycle. However, EA lacks evidence for how different choices, circumstances and lifestyles affect individual impacts. To address this we would like to support longitudinal studies to understand, for instance, how important factors such as age, career, happiness, mental health and actions (e.g., taking pledges, attending events, undergoing career changes etc) interact and change perceived impacts and EA involvements over lifespans and how these differ between current, and former EAs etc.
New EA incubation and scaling funds and organisations
Movement building, coordination, coincidence of wants problems, & scaling
Charity entrepreneurship, Y Combinator, Rocket Internet and similar have had notable and disproportionate economic and social impacts and accelerated the dissemination of innovative ideas. The EA community has also called for more founders. We would therefore like to support EA and social impact funds that initiate, or scale relevant initiatives such as charities (e.g., tax-deductible EA charity funds, long term future fund equivalents or research institutes)
Collective financing for EA products
Movement building, coordination, coincidence of wants problems
As shown by crowdfunding platforms, collective financing has many benefits. For instance, it allows individuals to collectively fund projects that they could not fund as individuals and for projects to start and scale when they would not otherwise exist. We would therefore like to fund projects to support collective financing with the EA community. For instance, this could involve allowing individuals to commit to providing a project or service (e.g., a ... (read more)
Better recruitment and talent scouting networks
Movement building, coordination, coincidence of wants problems
Decentralised social good communities face significant coordination problems: Many talented social actors and influencers are either unaware of key knowledge or unable to find a clear fit for their skills. This is particularly true in less developed countries, where relevant networks are relatively nascent. To address this, we’d like to support the work that develops the global network of recruiters and talent scouts. For instance, these organisatio... (read more)
Developing GCR scenario response teams and plans
Global catastrophic risks
As Covid-19 demonstrated, groups are unable to efficiently mobilise and coordinate to deal with potential Global Catastrophic Risks (GCRs) or large scale events without prior preparation. This leads to extensive inefficiencies, risks and social costs. Organisations address such unpreparedness by simulating key risks and training to handle them. We would similarly like to fund relevant institutions and organisations teams to simulate GCR related outcomes (e.g, nuclear attacks, wars or pandemic outbreaks) to develop and practice responses and disseminate best practice.
EA community housing network
Movement building & coordination
Social movement building requires key members of the community to have regular rewarding interactions. To catalyse social movement building, we would like to establish more EA organisations and institutions across the world more travel arrangements between them. For instance, this could be modelled on approaches such as the “International House” student accommodation, which provides cheap accommodation for students and works to instil cosmopolitan values.
...
A late update is that I ... (read more)
Cotton Bot
Economic growth
Problem: In 2021, a mere 30% of the world’s cotton harvest was gathered by machinery. This means
that over 60% of the 2021 worldwide supply of cotton was harvested using the same methods as
American slaves in the 1850’s. A significant amount of the hand harvesting includes forced labor.
Solution: The integration of existing technologies can provide a modular, robust, swarming team of
small-scale, low-cost harvesters. Thoughtful system design will ensure the harvesters are simple to
operate and maintain while still containing leadi... (read more)
Research Coordination Projects
Research that can help us improve
At the root of many problems that are being discussed are coordination problems. People are in prisoners' dilemmas, and keep defecting. This is the case in the suggestion to buy a scientific journal: if the universities coordinated they could buy the journal, remove fees, improve editorial policies, and they would be in a far better situation. Since they don't coordinate, they have to pay to access their own research.
Research into this type of coordination problem has revealed two general strat... (read more)
Combined conferences
Effective altruism, Epistemic institutions, Values and reflective processes
Fund teams that have roots in both EA and in other relevant fields and communities to put on conferences that bring together those communities. For example, it could be valuable to put on a conference for EA and RadicalxChange, given that there is a lot of overlap in interests but significant differences in approach. This could help bring in new ideas into EA, especially as a conference is a good way to build relationships and have lengthy, careful discussions. O... (read more)
Increase the number of STEM-trained people, in EA and in general
Economic growth, Research that can help us improve
Research and efforts to increase the numberof quantitatively skilled people in general, and targeted EA movement-building efforts to them could potentially be very impactful. (e.g., AI alignment research, biorisk research, scientific research in general) Incentivizing STEM education at the school and university levels, facilitating immigration of STEM degree holders, and offering STEM specific guidance via 80,000 Hours and other organizations could potentially be very impactful.
On-demand Software Engineering Support for Academic AI Safety Labs
AI safety work, e.g. in RL and NLP, involves both theoretical and engineering work, but academic training and infrastructure does not optimize for engineering. An independent non-profit could cover this shortcoming by providing software engineers (SWE) as contractors, code-reviewers, and mentors to academics working on AI safety. AI safety research is often well funded, but even grant-rich professors are bottlenecked by university salary rules and professor hours which makes hiring competent... (read more)
Audio/video databases of people's experiences of problems
Values and reflective processes, Effective altruism, Research that can help us improve, epistemic institutions
Grantmakers and policymakers are usually far removed from the problems that people face in their daily lives, especially from the problems of people who are more marginalised. Part of the solution to this should be that grantmakers and policymakers make sure to talk to a variety of people to involve them in decisionmaking. However, databases of audio and video interviews with people could als... (read more)
Regulating AI consciousness.
Artificial intelligence, Values and reflective process
The probability that AIs will be capable of conscious processing in the incoming decades is not negligible. With the right information dynamics, some artificial cognitive architecture could support conscious experiences. The global neural workspace is an example of a leading theory of consciousness compatible with this view. Furthermore, if it turns out that conscious processing improves learning efficiency then building AI capable of consciousness might become an effective path toward more generally capable AI. Building conscious AIs would have crucial ethical implications given their high expected population. To decrease the chance of bad moral outcomes we could follow two broad strategies. First, we could fund policy projects aiming to work with regulators to ban or slow down research that poses a substantial risk to building conscious AI. Regulations slowing the arrival of conscious AIs could be in place until we gain more moral clarity and a solid understanding of machine consciousness. For example, philosopher Thomas Metzinger advocated a moratorium on synthetic phenomenology in ... (read more)
Intellectual coaching
Empowering exceptional people, Effective Altruism
Many people with the potential to do good research and writing work hit blockers that are a complex mix of psychological blockers and intellectual issues. For example uncertainty and fear around what to work on, lack of confidence in one's ability. It's difficult to find someone to help address this kind of problem. Therapists and mainstream coaches don't have a good understanding of research and EA work. But within EA most of the coaching available is focussed on career choice or produc... (read more)
EA Marketing Agency
Improve Marketing in EA Domains at Scale
Problem: EAs aren’t good at marketing, and marketing is important.
Solution: Fund an experienced marketer who is an EA or EA-adjacent to start an EA marketing agency to help EA orgs.
Top ML researchers to AI safety researchers
Pay top ML researchers to switch to AI safety
Problem: <.001% of the world’s brightest minds are working on AI safety. Many are working on AI capabilities.
Solution: Pay them to switch. Pay them their same salary, or more, or maybe a lot more.
X-risk Art Competitions
Fund competitions to make x-risk art to create emotion
Problem: Some EAs find longtermism intellectually compelling but not emotionally compelling, so they don’t work on it, yet feel guilty.
Solution: Hold competitions where artists make art explicitly intended to make x-risk emotionally compelling. Use crowd voting to determine winners.
AGI Early Warning System
Anonymous Fire Alarm for Spotting Red Flags in AI Safety
Problem: In a fast takeoff scenario, individuals at places like DeepMind or OpenAI may see alarming red flags but not share them because of myriad institutional/political reasons.
Solution: create an anonymous form - a “fire alarm” (like an whistleblowing Andon Cord of sorts) where these employees can report what they’re seeing. We could restrict the audience to a small council of AI safety leaders, who then can determine next steps. This could, in theory, provide days to months of additional response time.
Teaching buy-out fund
Allocate EA Researchers from Teaching Activities to Research
Problem: Professors spend a lot of their time teaching instead of researching. Many don’t know that many universities offer “teaching buy-outs”, where if you pay a certain amount of money, you don’t have to teach. Many also don’t know that a lot of EA funding would be interested in paying that.
Solution: Make a fund that's explicitly for this, to make it so more EAs know. This is the 80/20 of promoting the idea. Alternatively, funders can just advertise this offering in other ways.
Academic AI Safety Journal
Start an Academic Journal for AI Safety Research
Problem: There isn’t one. There should be. It would boost prestige and attract more talent to the field.
Solution: Fund someone to start one.
This has come up a few times before and is controversial.
Pros:
Cons:
FWIW I take the first con to be decisive against it. Higher status takes a long time to build, and better peer review is (sadly) a mirage.
New non-academic intellectual communities
Empowering exceptional people, Values and reflective Processes
The pathologies of academia are well known, and there are many people who would like to engage with and contribute to research but once they are outside of academia they don't have the structures to do so. Recently there have been some new projects springing up to fill this gap, such as:
- InterIntellect, where people can host and take part in online salons on any topic. The founder of this (Anna Gat) was supported by an emergent ventures grant.
- The Cat
... (read more)Alignment Forum Writers
Pay Top Alignment Forum Contributors to Work Full Time on AI Safety
Problem: Some of AF’s top contributors don’t actually work full-time on AI safety because they have a day job to pay the bills.
Solution: Offer them enough money to quit their job and work on AI safety full time.
Bounty Budgets
Like Regranting, but for Bounties
Problem: In the same way that regranting decentralizes grantmaking, so do the same thing for bounties. For example, give the top 20 AI safety researchers up to $100,000 to create bounties or RFPs for, say, technical research problems. They could also reallocate their budget to other trusted people, creating a system of decentralized trust.
In theory, FTX’s regrantors could already do this with their existing budgets, but this would encourage people to think creatively about using bounties or RFPs.
Bounties are great because you only pay out if it's successful. If hypothetically each researcher created 5 bounties at $10,000 each that’d be 100 bounties - lots of experiments.
RFPs are great because it puts less risk on the applicants but also is a scalable, low-management way to turn money into impact.
Examples: 1) I’ll pay you $1,000 for every bounty idea that gets funded
2) Richard Ngo
EA Forum Writers
Pay top EA Forum contributors to write about EA topics full time
Problem: Some of the EA Forum’s top writers don’t work on EA, but contribute some of the community’s most important ideas via writing.
Solution: Pay them to write about EA ideas full time. This could be combined with the independent researcher incubator quite well.
AI Safety “school” / More AI safety Courses
Train People in AI Safety at Scale
Problem: Part of the talent bottleneck is caused by there not being enough people who have the relevant skills and knowledge to do AI safety work. Right now, there’s no clear way to gain those skills. There’s the AGI Fundamentals curriculum, which has been a great success, but aside from that, there’s just a handful of reading lists. This ambiguity and lack of structure lead to way fewer people getting into the field than otherwise would.
Solution: Create an AI safety “school” or a bunch more AI safety courses. Make it so that if you finish the AGI Fundamentals course there are a lot more courses where you can dive deeper into various topics (e.g. an interpretability course, values learning course, an agent foundations course, etc). Make it so there’s a clear curriculum to build up your technical skills (probably just finding the best existing courses, putting them in the right order, and adding some accountability systems). This could be funded course by course, or funded as a school, which would probably lead to more and better quality content in the long run.
EA Productivity Fund
Increase the output of top longtermists by paying for things like coaching, therapy, personal assistants, and more.
Problem: Longtermism is severely talent constrained. Yet, even though these services could easily increase a top EAs productivity by 10-50%, many can’t afford them or would be put off by the cost (imposter syndrome or just because it feels selfish).
Solution: Create a lightly-administered fund to pay for them. It’s unclear what the best way would be to select who gets funding, but a very simple decision metric could be to give it to anybody who gets funding from Open Phil, LTFF, SFF, or FTX. This would leverage other people’s existing vetting work.
Translate EA content at scale
Reach More Potential EAs in Non-English Languages
Problem: Lots of potential EAs don’t speak English, but most EA content hasn’t been translated
Solution: Pay people to translate the top EA content of all time into the most popular languages, then promote it to the relevant language communities.
Incubator for Independent Researchers
Training People to Work Independently on AI Safety
Problem: AI safety is bottlenecked by management and jobs. There are <10 orgs you can do AI safety full time at, and they are limited by the number of people they can manage and their research interests.
Solution: Make an “independent researcher incubator”. Train up people to work independently on AI safety. Match them with problems the top AI safety researchers are excited about. Connect them with advisors and teammates. Provide light-touch coaching/accountability. Provide enough funding so they can work full time or provide seed funding to establish themselves, after which they fundraise individually. Help them set up co-working or co-habitation with other researchers.
This could also be structured as a research organization instead of an incubator.
Multilingual web searching and browsing
Effective altruism, epistemic institutions
Despite the capability of automated translation, there is no smooth way to browse the web in multiple languages. It would be useful to have search engines return results from any language, with the results automatically translated into English. When you click on them, you then go to a web page automatically translated into English and can continue browsing in English. This seems important for EA because EA research currently relies primarily on English resources, and this coul... (read more)
Nonprofit Growth Research Think Tank/Consultancy
EA Ops, Effective Altruism
Most EA organisations and projects will be faced (at several times during their organisational lifecycle) with changes to their organisations due the growth of their teams.
If handled poorly a team can grow with many "growing pains" such as processes, policies, financial systems, (project) management and organisational/team structures that are not fit to the new status quo.
We'd love to see an organization that guides other EA organisations on their path to growth by identifying the right strategies and blind spots to manage the change phase in a period of growth.
EA-oriented research search engines
Effective altruism
EA researchers and people in similar roles such as grantmakers and policy analysts face a difficult search challenge. They are often trying to find high-quality resources that synthesise expert consensus in fields that are unfamiliar to them. Google often returns results that are too low-quality and popularly-oriented, but google scholar returns results that are too specific or which are only tangentally related to EA/policy/grantmaker interests. An improved search engine would return quality synthesis r... (read more)
Job application support for underrepresented groups
Increasing diversity in EA, Effective Altruism
Underrepresented groups usually face additional (or exacerbated) challenges in job applications: language barriers, impostor-syndrome, smaller networks, etc. that affect their application success. There are organisations within the EA ecosystem that provide career coaching but none provides dedicated, on demand support with job applications.
We'd love to see an organisation that provides ongoing support to people from underrepresented groups in job applications including: finding the right opportunities, preparing application documents, preparing for interviews, etc. so they are more likely to land high impact roles.
Vetting and matchmaking organization of consultants and contractors for EA founders
Empowering Exceptional People, Effective Altruism
Founders of new projects, charities and other EA-aligned organisations can have an extremely high impact. These individuals tend to suffer more from issues such as overwhelm, burnout, etc. which can easily lead them to have much less impact both short and long-term. A potential intervention against this is decreasing the decision-making overload by helping them outsource some of their decision-making.
We'd love to see an organization that offers vetting and matchmaking for independent consultants and contractors in several relevant areas of decision-making for these people so they can tap into knowledge and expertise faster with less effort and cognitive load.
This service can be considered and expansion of this idea by aviv.
Leadership and management auditing
Effective Altruism
It is uncertain at what cost to employees' well-being EA organisations achieve impact. A sustainable ecosystems of EA organizations that has long-term impact should have a foundation of evidence-based leadership and management that doesn't harm employees or volunteers (or at least tries to avoid this).
We'd love to see an organisation that evaluates the leadership and management practices of EA organisations and its effect on the well-being of their employees at all levels of the organisation as well as make recommendations for improvement.
Synthesis book fund/prize
Senior academics or practitioners have the accumulated experience and knowledge to be able to write grand syntheses of their subjects, or to put forward grand theories, without those just being wild speculation. This fund would proactively support and/or retroactively reward work of this type. To make this kind of work more likely, the fund could seek out academics that seem in a particularly good place to create a work of this type and encourage them to do this. In addition, the fund could support the writing of both a... (read more)
Credence Weighted Citation Metrics
Epistemic Institutions
Citation metrics (total citations, h-index, g-index, etc.) are intended to estimate a researcher's contribution to a field. However, if false claims get cited more then true claims (Serra-Garcia and Gneezy 2021), these citation metrics are clearly not fit for purpose.
I suggest modifying these citation metrics by weighing each paper by the probability that it will replicate. If each paper i has ci citations and probability of replicating pi, we can modify each formula as follows: instead of measuring t... (read more)
Funding AI policy proposals to slow down high-risk AI capability research.
AI alignment, AI policy
We want AI alignment research to catch up and surpass AI capability research. Among others, AI capability research requires a friendly political environment. We would be interested in funding AI policy proposals that would increase the chance of obtaining effective regulations slowing down highly risky AI capability R&D. For example, some regulations could impose large language models to pass a thorough safety audit before deployment or scaling in parameter... (read more)
Bonuses/prizes/support for critically situated or talented workers
Empowering Exceptional People
Work that advances society should be rewarded and compensated at fair market value. Unfortunately, rewards are often incommensurate, delayed or altogether unrealized. We'd be excited to see a funding process that 1) identifies work that’s under appreciated by or insulated from the market and 2) provides incentives for workers/teams to stay put and complete said work.
EA often focuses on building new organizations to solve problems, but talented people are al... (read more)
Self-Improving Healthcare
Biorisk and Recovery from Catastrophe, Epistemic Institutions, Economic Growth
Our healthcare systems aren't perfect. One underdiscussed part of this is that we learn almost nothing from the vast majority of treatment that happens. I'd love to see systems that learn from the day-to-day process of treating patients, systems that use automatic feedback loops and crowd wisdom to detect and correct mistakes, and that identify, test and incorporate new treatments. It should be possible to do this. Below is my suggestion.
I suggest we allo... (read more)
Organization to push for mandatory liability insurance for dual-use research
Biorisk and Recovery from Catastrophe
Owen Cotton-Barratt for the Global Priorities Project in 2015:
... (read more)EA ops: "Immigration Tech"
I have an idea for a cloud based, AI-powered SaaS platform to help governments handle immigration. Think KYC meets immigration
Today the immigration process is disjointed and fragmented amongst different countries and in most cases it's cumbersome, overly bureaucratic. That means that difficulties for immigrants, particularly in clear Human Rights cases, as well as for countries, who may be losing out on highly skilled migrants.
The idea is a platform that connects between potential immigrants and potential host countries. Instead of an immigrant applying individually to a number of countries, he would upload his relevant documentation to the platform that will then be shared with his countries of choice. Another model could be for interested countries to directly reach out to the potential immigrant of their own accord.
Part of the work of the platform would be to perform the relevant KYC work to authenticate the request as legitimate - thereby saving time and resources for national immigration departments, particularly when a request is lodged to multiple countries.
Obviously the idea is still in it's early stages and there are a number of detail... (read more)
Responsible AI Incubator
AI Safety
Creating an incubator to encourage new startups to invest in the responsible use of AI (including longer-term safety issues) by making this a requirement of investment. In addition to influencing companies, this could enhance the credibility of the field and help more AI safety researchers to become established.
Downsides: This could accelerate AI timelines, but the fund would only have to offer slightly better terms in order to entice startups to join.
Promote Ethical Corporate Behavior
EA to purchase 5% of Blackrock and 5% of Vanguard shares. To be clear, I don't mean 5% of their index funds, but rather 5% of the underlying fund management companies.
EA's investment of circa $10 billion can be leveraged into a board seat on companies that manage circa $20 trillion in assets. EA could lobby these companies to apply a corporate ethics tests on all their index funds. E.g. excluding coal and promoting other EA priorities.
EA to create an incubator to fund social enterprises with a high social return on investment.
This will help improve the visibility of the EA brand. It will also help connect ideas to improve the world with capital.
Lobby big tech companies to create AI Safety departments to monitor the growth of machine learning technology and implement proactive risk mitigation.
Create a suite of online and in person EA qualifications to help attract new people into the movement and unskill existing members.
The suite of online qualifications could follow a similar model to the Khan academy. Short, interactive courses led by gifted teachers and delivered online. These courses would cover foundational EA materials.
EA could also partner with universities to deliver formal courses in areas such as existential risk or AI safety.
Establish a virtual EA co-working space in the metaverse or on another platform to allow EA's from every country to meet and create new ideas together.
Economic growth
Work with developing countries to buy an area of land to form an EA special economic area. This can be a place where EA can congregate and innovate in IT and other fields. It can also be a place where EA can demonstrate new policies, technologies and pioneer new ways of thinking.
EA could expand on this idea to build communities in remote places that are likely to survive extinction events. It will provide a good opportunity to test technology that will be used in any space colonies.
Blockchain for people to prove their ID. Often in a disaster people's identity documents are lost or taken. This Blockchain will allow people to prove who they are and will also allow direct disaster relief payments to be made via the Blockchain.
Incentivize researchers to prioritize paradigm shifts rather than incremental advances
Economic growth, Research That Can Help Us Improve
There's a plausible case that societal under-innovation is one of the largest causes (if not the largest cause) of people's suboptimal well-being. For example, scientific research could be less risk-averse/incremental and more pro-moonshots. Interdiscplinary research on how to achieve society's full innovation potential, and movement-building targeted at universities, scientific journals, and grant agencies to incentivize scientific moonshots could potentially be very impactful.
Generous prizes to attract young top talent to EA in big countries
Effective altruism
Prizes are a straightforward way to attract top talent to engage with EA ideas. They also require relatively low human capital or expertise and therefore are conceivably scalable for different countries. Through a nationwide selection process optimized for raw talent, ability to get things done, and altruistic alignment, an EA prize could quickly make the movement become well-known and prestigious in big countries. High school graduates and early university students would probably be the best target audience. The prize could come with a few strings attached, such as participating in a two-week-long EA fellowship, or with more intense commitments, such as working for a year on an EA-aligned project. Brazil and India are probably the best fit, considering their openness to Western ideas and philanthropic investment (in comparison to China and Russia). Other candidates may include the Philippines, where EA groups have been relatively successful, Indonesia, Argentina, Nigeria, and Mexico.
Research into increasing the “surface area” of important problems
Artificial Intelligence, Biorisk and Recovery from Catastrophe, Epistemic Institutions, Values and Reflective Processes, Economic Growth, Great Power Relations, Space Governance, Effective Altruism
The idea here is that 80,000 Hours seems to follow an approach along the lines of (1) What are the biggest problems? (2) What are the obvious ways to make progress on these problems? (3) How can we get people to implement these obvious ways?
If we hold the first question constant, we can instead ask:... (read more)
Making AI alignment research among the most lucrative career path in the world.
AI alignment
Having the most productive researchers in AI alignment would increase our chances to develop competitive aligned models and agents. As of now, the most lucrative careers tend to be in top AI companies. They attract many bright graduate students and researchers. We want this to change and enable AI alignment research to become the most attractive career choice for excellent junior and senior engineers and researchers. We are willing to fund AI alignment workers with wages higher than top AI companies' standards. For example, wages could start around 250k$/year and grow with productivity and experience.
Prevent stable global totalitarian regimes through uncensorable broadcasts
Great Power Relations, Epistemic Institutions
Human civilization may get caught in a stable global totalitarian regime. Current and past totalitarian regimes have struggled with influences from the outside. So it may be critical to make sure now that future global totalitarian regimes will also have influences from the outside.
North Korea strikes me as a great example of a totalitarian regime straight out of 1984. Its systematic oppression of its citizens is so sophisticated that I could well imagine a world-wide regime of this sort to be stable for a very long time. Even as it exists today, it’s remarkably stable.
The main source of instability is that there’s a world all around North Korea, and especially right to its south, that works so much better in terms of welfare, justice, prospecity, growth, and various moral preferences that are widely shared in the rest of the world.
There may be other sources of instability – for example, I don’t currently understand why North Korea’s currency is inflated to worthlessness – but if not, then we, today, are to a hypothetical future global totalitarian state what the r... (read more)
Research to determine what human cultures minimize the risks of major catastrophes
Great Power Relations, Values and Reflective Processes, Artificial Intelligence
I posit that human cultures differ and that there’s a chance that some cultures are more likely to punish in minor ways and more likely to adapt to new situations peacefully while other may be more likely to wage wars. This may be completely wrong.
But if it is now, we could investigate what processes can be used to foster the sort of culture that is less likely to immanentize global catastrophes, a... (read more)
Safety of comprehensive AI services
Artificial Intelligence
I imagine that comprehensive AI services (CAIS) could face similar problems to intelligence agencies. Ideally, an intelligence agency would only hire those people who are maximally trusted, but then they could hire hardly anyone. Instead they split the information that any one person can see such that (1) that person can’t do much harm with the one piece of the full picture that they have and (2) if it leaks or the person exploits their knowledge in illegitimate ways, the higher-ups can trace the le... (read more)
CEA for the developing world
Effective Altruism
The main EA movement building organization, CEA, focuses primarily on talented students in top universities of developed countries. This seems to be due to a combination of geographical and cultural proximity, quantity of English speakers, and ease of finding top talent. However, there is a huge amount of untapped talent in developing countries that may be more easily reached through dedicated organizations optimized for being culturally, linguistically, and geographically close to such talent, such as a CEA for India or Brazil. Such an organization would develop its own goals and strategies tailored to their respective regions, such as prioritizing nationwide prizes over group-by-group support, hiring local EA talent to lead projects, and identifying and partnering with regionally influential universities and institutions. This project would not only contribute to increasing diversity in EA, but also foster organizational competition by allowing different movement building strategies, and better position the EA movement for unexpected geopolitical power shifts.
Authoritative Statements of EA Views
Epistemic Institutions
In academia, law, and government, it would be helpful to have citeable statements of EA relevant views presented in an authoritative and unbiased manner. Having such material available lends gravitas to proposals that help address related problems and provides greater justification in taking those views for granted.
(This is a variation on 'Expert polling for everything' focused on providing authority of views to non-experts. The Cambridge Declaration on Consciousness is a good example.)
An ecosystem of organizations to initiate a “Hasty Reflection”
Values and Reflective Processes, Epistemic Institutions, Effective Altruism, Research That Can Help Us Improve
The Long Reflection appears to me to be robustly desirable. It only suffers from being more or less unrealistic depending on how it is construed.
In particular, I feel that two aspects of it are in tension: (1) delaying important, risky, and irreversible decisions until after we’ve arrived at a Long Reflection–facilitated consensus on them, and (2) waiting with the Long Reflection itself until after we’ve achieved existential security.
I would expect, as a prior, that most things happen because of economic or political necessity, which is very hard to influence. Hence the Long Reflection either has to ramp up early enough that we can arrive at consensus conclusions and then engage in the advocacy efforts that’ll be necessary to improve over the default outcomes or else risk that flawed solutions get locked in forever. But the first comes at the risk of diverting resources from existential security. This indicates that there is some optimal trade-off point between existential security and timely conclusions. (From ... (read more)
A start-up accelerator for pledge-signing EAs and longtermists.
Economic Growth, Effective Altruism, Empowering Exceptional People
Y-combinator/Entrepreneur First meets Founders Pledge. A top-tier start-up accelerator where applicants sign a pledge to donate a significant amount of exit proceeds/profits to doing the most good they can from an effective altruist/longtermist perspective. Build start-ups and network with your value aligned peers!
Funding the AI alignment institute, a Manhattan project scale for AI alignment.
Artificial intelligence
Aligning AI with human interests could be very hard. The current growth in AI alignment research might be insufficient to align AI. To speed up alignment research, we want to fund an ambitious institute attracting hundreds to thousands of researchers and engineers to work full-time on aligning AI. The institute would enable these researchers to work with computing resources competitive with top AI industries. We could also slow down risky AI capability res... (read more)
Fund/Create training for mental health workers
Effective Altruism
A limiting reagent in health care in Canada right now is that there aren’t enough psychologists/psychiatrists/ mental health workers. People don’t have access to these services and end up in the emergency department and crashing the health system in other ways. Mental health is fundamental for participation in societal roles, and highly conscientious people are at risk and children are waiting years for assessments (like for ADHD) which can change the course of their lives.
Psychiatry is one of the least well paid medical specialties, it takes many years to train psychologists and psychiatrists.
I propose looking at funding the training of the mental health workforce, as well as lobbying to have mental health services to be included as essential health care services.
Solve Type 2 Diabetes
Biorisk and Recovery from Catastrophe
Type 2 Diabetes, caused by insulin resistance, is one of the top 10 causes of disability (DALYs) and also is root cause for ischemic heart disease and stroke, which are also in the top 10. People with diabetes are immune compromised and have worse outcomes from infection (as we saw in Covid). Several treatments to reverse diabetes are known, and there are groups like Virta Health doing good work in this space, but some treatments are prohibitively expensive (like GLP-1 agonists). Prevention and nutr... (read more)
A few people have mentioned retroactive public goods funding. I'd suggest broadening the scope a bit:
... (read more)A project to investigate and prioritize project proposals such as all of these
Research That Can Help Us Improve, Effective Altruism, Empowering Exceptional People
Even long lists of project proposals like this one can miss important projects. The proposals (including my own) are also rarely concrete enough to gauge their importance or tractability.
Charity entrepreneurs are currently mostly on their own when it comes to prioritizing between project proposals and making them more concrete. There may be great benefits to specialization and economies of scale h... (read more)
A think tank to investigate the game theory of ethics
Values and Reflective Processes, Effective Altruism, Research That Can Help Us Improve, Space Governance, Artificial Intelligence
Caspar Oesterheld’s work on Evidential Cooperation in Large Worlds (ECL) shows that some fairly weak assumptions about the shape of the universe are enough to arrive at the conclusion that there is one optimal system of ethics: the compromise between all the preferences of all agents who cooperate with each other acausally. That would solve ethics for all practical purposes. It... (read more)
Scoring scientific fields
Epistemic Institutions
Some fields of science are uncontroversially more reliable than others. Physics is more reliable than theoretical sociology, for example. But other fields aren't that easy to score. Should you believe the claims of a random sleep research paper? Or a paper from personality psychology? Efficacy is just as important, as a scientific field with low efficacy is probably not worth engaging with at all.
A scientific field can be evaluated by giving it a score along one or more dimensions, where a lo... (read more)
Scandinavian-like parental leave (25 weeks +) in EA organizations
Leading the way with a policy combating demographic decline, while supporting talent selection and diversity in EA community
Paid parental leave creates an incentive to have (more) kids - or rather, it takes away part of the large financial incentive not to have kids. My concrete suggestion is to fund Scandinavian-like parental leaves for employees in specified EA organizations. This would open up more access to the large pool of talented family oriented persons. Further, having an unusually beneficial parental leave benefit could inspire other organizations to follow, and thus help combat demographic decline. The idea should be quite easy to pilot, implement and scale, and the results relatively easy to measure.
Monitoring Nanotechnologies and APM
Nanotechnologies, and a catastrophic scenario linked to it called “Grey goo”, have received very little attention recently (more information here ), whereas nanotechnologies keep moving forward and that some think that it’s one of the most plausible ways of getting extinct.
We’d be excited that a person or an organization closely monitors the evolution of the field and produces content on how dangerous it is. Knowing whether there are actionable steps that could be taken now or not would be very valuable for both funders and researchers of the longtermist community.
Making Impactful Science More Reputable
There are two things that matter in science: reputation and funding. While there is more and more funding available for mission-driven science, we’d be excited to see projects that would try to increase the reputation of impactful science. We think that increasing the reputation of impactful work could over time increase substantially the amount of research done on most things that society care about.
Some of the ways we could provide more reputation to impactful research:
- Awarding prizes to past and present researchers
... (read more)An Organisation that Sells its Impact for Profit
Empowering Exceptional People, Epistemic Institutions
Nonprofits are inefficient in some respects: they don't maximize value for anyone the way for-profits do for their customers. Moreover, they lack market valuations, so successful nonprofits scale too slowly while unsuccessful ones linger too long. One way to address this is to start an organisation that only accepts funding that incentivizes impact. Its revenue would come from: (1) Selling Impact Cerificates, (2) Prizes, and/or (3) Grants (but only if they value the work at a similar level to the impact certificates). Such an organization could operate on an entirely for-profit basis. Funding would be raised from for-profit investors. Staff would be paid in salary plus equity. The main premise here is that increased salaries are a small price to pay for the efficiencies that can be gained from for-profit markets. Of course, this can only succeed if the funding mechanisms (1-3) become sufficiently popular, but given the increased funding in longtermist circles, this now looks increasingly likely.
See also Retrospective grant evaluations, Retroactive public goods funding, Impact ... (read more)
Continuous sampling for high-risk laboratories
Biorisk and Recovery from Catastrophe
We would be excited to fund efforts to test laboratory monitoring systems that would provide data for biosafety and biosurveillance. The 1979 Sverdlovsk anthrax leak happened because a clogged air filter had been removed from the bioweapons laboratory's exhaust pipe and no one informed the night shift manager. What if, by default, ventilation ducts in high-containment laboratories were monitored to detect escaping pathogens? Establishing a practice of continuous sampling wou... (read more)
Ethics Education
Values and Reflective Processes
Over the next century, leaders will likely have to make increasingly high-stakes ethical decisions. In democratic societies, large numbers of people may play a role in making those decisions. And yet, ethics is seldom thoroughly taught in most educational curricula. While it may be covered briefly in secondary school and is covered in detail at university for those who attend and choose to study it, many accomplished people do not have even a superficial understanding of the most important ethical theories and... (read more)
Experimental Wargames for Great Power War and Biological Warfare
Biorisk and Recovery from Catastrophe, Epistemic Institutions
This is a proposal to fund a series of "experimental wargames," on great power war and biological warfare. Wargames have long been a standard tool of think tanks, the military, and the academic IR world since the early Cold War. Until recently, however, these games were largely used to uncover unknown unknowns and help with scenario planning. Most such games continue to be unscientific exercises. Recent work on "experimental wa... (read more)
Quick start kit for new EA orgs
EA ops
Stipe atlas for longtermist orgs. Rather than figuring out the best tools, registrations, and practices for every new org, figure out the best default options and provide an easy interface to start up faster.
Creative Arms Control
Biorisk and Recovery from Catastrophe
This is a proposal to fund research efforts on "creative arms control," or non-treaty-based international governance mechanisms. Traditional arms control -- formal treaty-based international agreements -- has fallen out of favor among some states, to the extent that some prominent policymakers have asked whether we've reached "The End of Arms Control."[1] Treaties are difficult to negotiate and may be poorly suited to some fast-moving issues like autonomous weapons, synthetic biology, and cyber... (read more)
A Project Candor for Global Catastrophic Risks
Biorisk and Recovery from Catastrophe, Values and Reflective Processes, Effective Altruism
This is a proposal to fund a large-scale public communications project on global catastrophic risks (GCRs), modeled on the Eisenhower administration's Project Candor. Project Candor was a Cold War public relations campaign to "inform the public of the realities of the 'Age of Peril'" (see Unclassified 1953 Memo from Eisenhower Library). Policymakers were concerned that the public did not yet understand that the threats from nuclear weapons and the Soviet Union had inaugurated a new era in human history: the Age of Peril. Today, at the precipice, the Age of Peril continues with possible risks from engineered pandemics, thermonuclear exchange, great power war, and more. Voting behavior and public discourse, however, do not seem attuned to these risks. A new privately-funded Project Candor would communicate to the public the nature of the threats, their probabilities, and what we can do about them. This proposal is related to "a fund for movies and documentaries" and "new publications on the most pressing issues," but differs in that it would be a unified and coordinated campaign across multiple media.
Expected value calculations in practice
Invest in creating the tools to approximate expected value calculations for speculative projects, even if hard.
Currently, we can’t compare the impact of speculative interventions in a principled way. When making a decision about where to work or donate, longtermists or risk-neutral neartermists may have to choose an organization based on status, network effects, or expert opinion. This is, obviously, not ideal.
We could instead push towards having expected value calculations for more things. In the same way that GiveWell did something similar for global health and development, we could try to do something similar for longtermism/speculative projects. Longer writeup here.
Offer paid sabbatical to people considering changing careers
Empowering Exceptional People
People sometimes are locked-in in their non-EA careers because while working, they do not have time to:
Create an organization that will offer paid sabbaticals to people considering changing careers to more EA-aligned jobs to help this transition. During the sabbatical, they could be members of a community of people in a similar situation, with coaching available.
Agree. I think that having an Advance Market Commitment system for this makes sense. E.g., FTX says 'We will fund mid-career academics/professionals for up to x months to do y. ' My experience is that most of the high value people I know who are good professional are sufficiently time poor and dissuaded by uncertainty that they won't spend 2-5 hours to apply for something they don't know they will get. The barriers and costs are probably greater than most EA funders realise.
An alternative/related idea is to have a simple EOI system where people can submit a fleshed out CV and a paragraph and then get a AMC on an application - e.g., We think that there is a more than 60% chance that we would fund this and would therefore welcome a full application.
A fast and widely used global database of pandemic prevention data
Biorisk
Speed is of the essence for pandemic prevention when emergence occurs. A fast and widely used global database could potentially be very impactful. It would be great if events like the early discovery of potential pandemic pathogens, doctors' diagnoses of potential pandemic symptoms, etc. regularly and automatically gets uploaded to the database, and high-frequency algorithms can use this database to predict potential pandemic outbreaks faster than people can do.
Global (baseline) Education Curriculum
Getting people aligned, avoiding division, humans on this planet are in the same team.
By creating a basic program and common understanding it will be much easier to implement any of the global policies required to handle climate change.
Some of the proposed subjects:
- Literacy, numeracy
- English. Alternatively: Latin and Esperanto are not really competitors, Chinese too difficult
- Health, human body, food, nutrition
- Nature, earth sciences, environment
- Making, engineering, tinkering
- Communication, relationships, culture, to
... (read more)One Device Per Human
Similar to: https://en.wikipedia.org/wiki/One_Laptop_per_Child
Allowing people from all over the world to vote on global issues.
(this is assuming we have global governance)
Provide personal assistants for EAs
Empowering Exceptional People
Many senior EAs spend way too much with busywork because it is hard to get a good personal assistant. This is currently so because:
All these factors would be removed if an agency managed personal assistants.
Look for UFOs
Space Governance
In recent years, there has been an upsurge in reports by the military on sightings of UFOs including detecting the same object with multiple modalities at once (examples: 1, 2).
Avi Loeb proposes to create a network of high-resolution sensors, (just as the military have). But compared to the military their results will not be classified and can be openly analyzed by scientists. The cost of doing this is in the order of millions of dollars.
Knowing if there are aliens has many consequences, including for the ... (read more)
Givewell for AI alignment
Artificial intelligence
When choosing where to donate to have the largest positive impact on AI alignment, the current best resource appears to be Larks annual literature review and charity comparison on the EA/LW forums. Those posts are very high-quality but they’re only published once a year and are ultimately the views of one person. A frequently updated donation recommendation resource contributed to by various experts would improve the volume and coordination of donations to AI alignment organisations and projects.
T... (read more)
Write encyclopedias (esp. Wikipedia), then translate them (esp. to Russian and Chinese)
Epistemic Institutions
Create a team of people who will write articles on Wikipedia on subjects related to EA. Why this is important is described here.
Besides writing articles on English Wikipedia, they can also:
- Create good illustrations (somehow, medical articles tend to have much beter pictures than other areas)
- Translate these articles to other languages (especially Russian and Chinese)
- Topics that are not notable enough for Wikipedia can be described in a sep
... (read more)Open-source intelligence agency
Great Power Relations
Create an organization that will collect and analyze open-source intelligence information on critical topics (e.g. US nuclear arsenal, more examples below) and publish it online.
Many documents on US nuclear arsenal and military activities were obtained through Freedom of Information Act. Still, they were never analyzed properly because it is a lot of tedious work that journalists do not have the capacity or incentive to do. Standard open-source intelligence gathering methods can provide even more information. As a result, there is only a limited public understanding of important sources of x-risk.
Possible subjects of investigation:
Website for coordinating independent donors and applicants for funding
Empowering exceptional people, effective altruism
At EAG London 2021, many attendees indicated in their profiles that they were looking for donation opportunities. Donation autonomy is important to many prospective donors, and increasing the range of potential funding sources is important to those applying for funding. A curated website which allows applicants to post requests for funding and allows potential donors to browse those requests and offer to fully or partially fund applicants, seems like an effective solution.
Create an independent organization working along the lines of Implementation Support Unit of Biological Weapons Convention
Biorisk and Recovery from Catastrophe
Biological Weapons Convention, forbidding the development of biological weapons, was signed in 1972 by most countries. But compliance is verified by Implementation Support Unit (BCW ISU), with the budget in the range $1-2m and only roughly four employees. At the same time, it seems there is a fair probability that Russia has an active biological weapon development program.
Cr... (read more)
Research scholarships / funding for self-study
Empowering exceptional people
The value of a full-time researcher in some of the most impactful cause areas has been estimated as being between several hundred thousand to several million dollars per year, and research progress is now seen by most as the largest bottleneck to improving the odds of good outcomes in these areas. Widespread provision of scholarships / funding for self-study could enable far more potential researchers to gain the necessary experience, knowledge, skills and qualifications to ma... (read more)
Find a niche to create a subsidized prediction market
Epistemic Institutions
One of the problems of current forecasting is that it isn’t getting attention from decision-makers. One way to jumpstart this is to create a subsidized market in some well-chosen area that will work well and thus publicly prove and legitimize the use of prediction markets.
One suitable idea is Robin Hanson’s idea with fire-CEO market:
... (read more)Researching valence for AI alignment
Artificial Intelligence, Values and Reflective Processes
In psychology, valence refers to the attractiveness, neutrality, or aversiveness of subjective experience. Improving our understanding of valence and its principal components could have large implications for how we approach AI alignment. For example, determining the extent to which valence is an intrinsic property of reality could provide computer-legible targets to align AI towards. This could be investigated experimentally: the relationship between experiences and their neural correlates & subjective reports could be mapped out across a large sample of subjects and cultural contexts.
Nuclear arms reduction to lower AI risk
Artificial Intelligence and Great Power Relations
In addition to being an existential risk in their own right, the continued existence of large numbers of launch-ready nuclear weapons also bears on risks from transformative AI. Existing launch-ready nuclear weapon systems could be manipulated or leveraged by a powerful AI to further its goals if it decided to behave adversarially towards humans. We think understanding the dynamics of and policy responses to this topic are under-researched and would benefit from further investigation.
Researching the relationship between subjective well-being and political stability
Great Power Relations, Values and Reflective Processes
Early research has found a strong association between a society's political stability and the reported subjective well-being of its population. Political stability appears to be a major existential risk factor. Better understanding this relationship, perhaps by investigating natural experiments and running controlled experiments, could inform our views of appropriate policy-making and intervention points.
Prevent community drainage due to value drift
Effective Altruism, Movement building
Most Effective Altruists are still young and will have the greates impact with their careers (and spend the greatest amounts of money) in several decades. However, people also change a lot and for some this leads to a decrease of engagement or even full drop-out. Since there is evidence, that drop out rates might be up to 30% throughout the career of higly engaged EAs, this is some serious loss of high impact work and well directed money.
Ways of tackling this prob... (read more)
EA content translation service
Effective Altruism, Movement Building
(Maybe add to #30 - diversity in EA)
EA-related texts are often using academic language needed to convey complex concepts. For non-native speakers reading and understanding those texts takes a lot more time than reading about the same topic in their native language would. Furthermore, today many educated people in important positions, especcially in non-western countries, do not at or only poorly speak English. (This is likely part of the reason that EA currently mainly exists in English speaking countries and almost exclusively consists of people speaking English well.)
To make EA widely known and easy to understand there needs to be a translation service enabling e.g. 80k, important Forum posts or the Precipice to be read in different languages. This would not only make EA easier to understand - and thus spread ideas further - but also likely increase epistemic diversity of the community by making EA more international.
A social media platform with better incentives
Epistemic Institutions, Values and Reflective Processes
Social media has arguably become a major way in which people consume information and develop their values, and the most popular platforms are far from optimally set up to bring people closer to truthfulness or altruistic ends. We’d love to see experiments with social media platforms that provide more pro-social incentives and yet have the potential to reach a large audience.
Institutions as coordination mechanisms
Artificial Intelligence, Biorisk and Recovery from Catastrophe, Great Power Relations, Space Governance, Values and Reflective Processes
A lot of major problems - such as biorisk, AI governance risk and the risks of great power war - can be modeled as coordination problems, and may be at least partially solved via better coordination among the relevant actors. We’d love to see experiments with institutions that use mechanism design to allow actors to coordinate better. One current example of such an institution is NATO: Article 5 is a coordination mechanism that aligns the interests of NATO member states. But we could create similar institutions for e.g. biorisk, where countries commit to a matching mechanism - where “everyone acts in a certain way if everyone else does” - with costs imposed to defectors to solve a tragedy of the commons dynamic.
Experiments with and within video games
Values and Reflective Processes, Empowering Exceptional People
Video games are a powerful tool to reach hundreds of millions of people, an engine of creativity and innovation, and a fertile ground for experimentation. We’d love to see experiments with and within video games that help create new tools to address major issues. For instance, we’d love experiments with new governance and incentive systems and institutions, new ways to educate people about pressing problems, games that simulate actual problems and allow players to brainstorm solutions, and games that help identify and recruit exceptional people.
Representation of future generations within major institutions
Values and Reflective Processes, Epistemic Institutions
We think at least part of the issues facing us today would be better handled if there was less political short-termism, and if there were more incentives for major political and non-political institutions to take into account the interests of future generations. One way to address this is to establish explicit representation of future generations in these institutions through strategic advocacy, which can be done in many ways and has been piloted in the past few decades.
Scaling successful policies
Biorisk and Recovery from Catastrophe, Economic Growth
Information flow across institutions (including national governments) is far from optimal, and there could be large gains in simply scaling what already works in some places. We’d love to see an organization that takes a prioritized approach to researching which policies are currently in place to address major global issues, identifying which of these are most promising to bring to other institutions and geographies, and then bringing these to the institutions and geographies where they are most needed.
A better overview of the effective altruism community
Effective Altruism
The effective altruism movement has grown large enough that it has become hard for any individual to have a good overview of ongoing projects and existing organizations. There is currently no central repository on what is happening across different causes and parts of the movement, which means many opportunities for coordination may be left on the table. We would like to see more initiatives like the yearly EA survey and a more detailed version of Ben Todd’s recent post that research and provide an overview of what is happening across the effective altruism movement.
A public EA impact investing evaluator
Effective Altruism, Empowering Exceptional People
Charity evaluators that publicly share their research - such as GiveWell, Founders Pledge and Animal Charity Evaluators - have arguably not only helped move a lot of money to effective funding opportunities but also introduced many people to the principles of effective altruism, which they have applied in their lives in various ways. Apart from some relatively small projects (1) (2) (3) there is currently no public EA research presence in the growing impact investing sector, which is both large in the amount of money being invested and in its potential to draw more exceptional people’s attention to the effective altruism movement. We’d love to see an organization that takes GiveWell-quality funding opportunity research to the impact investing space and publicly shares its findings.
More public EA charity evaluators
Effective Altruism
There are dozens of EA fundraising organizations deferring to just a handful of organizations that publish their research on funding opportunities, most notably GiveWell, Founders Pledge and Animal Charity Evaluators. We would like to see more professional funding opportunity research organizations sharing their research with the public, both to increase the quality of research in the areas that are currently covered - through competition and diversity of perspectives and methodologies - and to cover important areas that aren’t yet covered such as AI and EA meta.
Formulate AI-super-projects that would be both prestigious and socially beneficial
Artificial Intelligence, Great Power Relations
There are already some signs of race dynamics between the US and China in developing TAI. Arguably, they are at least partly motivated by concerns of national prestige. If race dynamics speed up, it might be beneficial to present a set of prestigious AI-projects that the US and other countries can adopt. These projects should have the following features:
- Be highly visible and impressive for a wide audience
- Contribute to safer
... (read more)Quantify the overall suffering from different conditions, and determine whether there's misallocation of resources in biomedical research.
I suspect there's a big gap between the distribution of resources allocated to the study of different diseases and what people actually suffer from the most. Among other factors that lead to non-optimal allocation, I'd guess that life-threatening diseases are overstudied whereas conditions that may really harm people's well-being, but are not deadly, are understudied. For example, I'd guess that chronic pain is understud... (read more)
Increase diversity with more ‘medium term’ plans to enable participation when travel is required
Community Building and Diversity, Values and Reflective Processes
I’m new here and it seems like many opportunities are planned with short notice. This can work well for people with lots of flexibility, but may discourage participation from people who are mid-career/working and people with families. I propose that organizations within EA encourage diversity by lengthening some planning horizons. Funding a stable hub with enough runway to have a 6 month planning horizon would be helpful for professionals and parents like my family.
A “Red Team” to rigorously explore possible futures and advocate against interventions that threaten to backfire
Research That Can Help Us Improve, Effective Altruism, Epistemic Institutions, Values and Reflective Processes
Motivation. There are a lot of proposals here. There are additional proposals on the Future Fund website. There are additional proposals also on various lists I have collected. Many EA charities are already implementing ambitious interventions. But really we’re quite clueless about what the future will bring.
This week alone I’ve discussed with friends and acquaintances three decisions in completely different contexts that might make the difference between paradise and hell for all sentient life, and not just in the abstract in the way that cluelessness forces us to assign some probability to almost any outcome but in the sense were we could point to concrete mechanisms along which the failure might occur. Yet we had to decide. I imagine that people in more influential positions than mine have to make similar decisions on almost a daily basis and on hardly any more information.
As a result, the robustness of an intervention has been the key criterion for prioritiza... (read more)
Create and distribute civilizational restart manuals
A number of "existential risks" we are worried about may not directly kill off everybody, but would still cause enough deaths and chaos to make rebuilding extremely difficult. Thus, we propose that people design and distribute "civilizational restart manuals" to places that are likely to survive biological or nuclear catastrophes, giving humanity more backup options in case of extreme diasters.
The first version can be really cheap, perhaps involving storing paper copies of parts of Wikipedia plus 10 most important books sent to 100 safe and relatively uncorrelated locations -- somewhere in New Zealand, the Antarctica research base, a couple of nuclear bunkers, nuclear submarines, etc.
We are perhaps even more concerned about great moral values like concern for all sentient beings surviving and re-emerging than preserving civilization itself, so we would love for people to do further research and work into considering how to preserve cosmopolitan values as well.
My comment from another thread applies here too:
Predicting Our Future Grants
Epistemic Institutions, Research That Can Help Us Improve
If we had access to a crystal ball that allowed us to know exactly what our grants five years from now otherwise would have been, we can make substantially better decisions now. Just making the grants we'd otherwise have made five years in the future can save a lot of grantmaking time and money, as well as cause many amazing projects to happen more quickly.
We don't have a crystal ball that lets us see future grants. But perhaps high-quality forecasts can be the next best thing. Thus, we're extremely excited about people experimenting with Prediction-Evaluation setups to predict the Future Fund's future grants with high accuracy, helping us to potentially allocate better grants more quickly.
A think tank to develop proof of stake for international conflicts
Artificial Intelligence, Great Power Relations, Space Governance
International conflicts pose a risk already, and that’ll only get worse when AI arms races start among countries. Yet, establishing a central world government is hard and bears the risk that it may be taken over by a dictator.
Currently we’re implementing an algorithm that puts at stake the lives of millions of citizens, and where almost anyone can slash the stake of almost anyone else. Instead we could put a lot of weath at stak... (read more)
A Longtermist Nobel Prize
All Areas
The idea is to upgrade the Future of Life Award to be more desirable. The prizemoney would be increased from $50k to$10M SEK (roughly $1.1M) per individual to match the Nobel Prizes. Both for prestige, and to make sure ideal candidates are selected, the selection procedure would be reviewed, adding extra judges or governance mechanisms as needed. This would not immediately mean that longtermism has something to match the prestige of a Nobel, but it would give a substantial reward and offer top longtermists something to strive for.
(A variation on a suggestion by DavidMoss)
Infrastructure to support independent researchers
Epistemic Institutions, Empowering Exceptional People
The EA and Longtermist communities appear to contain a relatively large proportion of independent researchers compared to traditional academia. While working independently can provide the freedom to address impactful topics by liberating researchers from the perversive incentives, bureaucracy, and other constraints imposed on academics, the lack of institutional support can impose other difficulties that range from routine (e.g. difficulties accessing pay-walled publications) to restrictive (e.g. lack of mentorship, limited opportunities for professional development). Virtual independent scholarship institutes have recently emerged to provide institutional support (e.g. affiliation for submitting journal articles, grant management) for academic researchers working independently. We expect that facilitating additional and more productive independent EA and Longtermist research will increase the demographic diversity and expand the geographical inclusivity of these communities of researchers. Initially, we would like to determine the main needs and limitations independent... (read more)
Refinement of project idea #8, Pathogen sterilization technology
Add: ‘We’d also be interested in the development of therapeutic techniques that could treat infections using these (e.g. relying on physical principles) or similar approaches.’
Stratospheric cleaning to mitigate nuclear winters
Recovery from Catastrophes
Proposals to recover from a nuclear winter have primarily focused on providing alternative means of food production until agriculture recovers. A complementary strategy would be to develop technologies to remove stratospheric soot, which could reduce the duration and severity of the nuclear winter if used soon after nuclear strikes while smoke remains concentrated above a relatively small geographic area. Stratospheric cleaning could also prove useful in the event of supervolcano e... (read more)
You may be interested in this. I considered some pretty speculative things to prevent or mollify a supervolcanic eruption, but the volume of the stratosphere is so enormous that I think cleaning it would be very challenging.
Pipeline for podcasts
Effective altruism
Crowdsourced resources, networks, and grants may help facilitate EAs and longtermists' creation of high-impact, informative podcasts.
The Petrov Prize for wise decision-making under pressure
Epistemic Institutions, Values and Reflective Processes
On September 26, 1983, Stanislav Petrov singlehandedly averted a nuclear war when he decided to wait for more evidence before reporting an apparent launch of ICBMs aimed at the Soviet Union. The incident was later determined to be a false alarm caused by an equipment malfunction. While Petrov's story is one of the most dramatic examples ever of impactful decision-making under pressure, there are plenty of other people and organizations throughout ... (read more)
Reduce meat consumption
Biorisk, Moral circle expansion
Research and efforts to reduce broad meat consumption would help moral circle expansion, pandemic prevention, and climate change mitigation. Perhaps messaging from the pandemic-prevention angle (in addition to the climate change angle and the moral circle expansion angle) may help.
Eliminate all mosquito-borne viruses by permanently immunizing mosquitoes
Biorisk and Recovery from Catastrophe
Billions of people are at risk from mosquito-borne viruses, including the threat of new viruses emerging. Over a century of large-scale attempts to eradicate mosquitoes as virus vectors has changed little: there could be significant value in demonstrating large-scale, permanent vector control for both general deployment and rapid response to novel viruses. Recent research has shown that infecting mosquitoes with Wolbachia, a bacterium, out-competes viruses (including dengue, yellow fever and Zika), preventing the virus from replicating within the insect, essentially immunizing it. The bacterium passes to future generations by infecting mosquito eggs, allowing a small release of immunized mosquitoes to gradually and permanently immunize an entire population of mosquitoes. We are interested in proposals for taking this technology to massive scale, with a particular focus on rapid deployment in the case of novel mosquito-borne viruses.
Epistemic status: Wolbachia impact on dengue fever has been demonstrated in a large RCT and about 10 city-level pilots. Impact on ot... (read more)
Campaign to eliminate lead globally
Economic Growth
Lead exposure limits IQ, takes over 1M lives every year and costs Africa alone $130B annually, 4% of GDP: an extraordinary limit on human potential. Most lead exposure is through paint in buildings and toys. The US banned lead paint in 1978 but 60% of countries still permit it. We would like to see ideas for a global policy campaign, perhaps similar to Bloomberg’s $1B tobacco advocacy campaign (estimated to have saved ~30M lives), to push for regulations and industry monitoring.
Epistemic status: The “prize” feels very large but I am not aware of proven interventions for lead regulations. 30 minutes of Googling suggests the only existing implementer (www.leadelimination.org) might be too small for this level of funding so there may not be many applicants.
Conflict of interest: I work for a small, new non-profit focused on $B giving. We are generally focused on projects with large, existing implementers so have not pursued lead elimination policy beyond initial light research
Pilot emergency geoengineering solutions for catastrophic climate change
Research That Can Help Us Improve
Toby Ord puts the risk of runaway climate change causing the extinction of humanity by 2100 at 1/1000, a staggering expected loss. Emergency solutions, such as seeding oceans with carbon-absorbing algae or creating more reflective clouds, may be our last chance to prevent catastrophic warming but are extraordinarily operationally complex and may have unforeseen negative side-effects. Governments are highly unlikely to invest in massive geoengineering solutions until the last minute, at which point they may be rushed in execution and cause significant collateral damage. We’d like to fund people who can:
Epistemic status: there seems to be rea... (read more)
Research into reducing general info-hazards
Biorisk
Researching and diseminating knowledge on how to generally reduce info-hazards could potentially be very impactful. An ambitious goal would be to have an info-hazard section in the training of journal editors, department chairs, and biotech CEOs in relevant scientific fields (although perhaps such a training would also be an info-hazard!)
Simultaneously reliable and widely trusted media
Epistemic institutions
Eeliable (in the truthseeking sense) media seems to not be widely trusted, and widely trusted media seems to not be reliable. Research and efforts to simultaneously achieve both could potentially be very impactful, for political resolution of a broad range of issues. (Ambitious idea: Can EAs/longtermists establish a media competitor?)
Normalize broad ownership of hazmat suit (and of N-day supply of non-perishable food and water)
Biorisk
If everyone either wore a hazmat suit all the time or stayed at home for 14 days (especially in the early stages of the COVID-19 pandemic), the pandemic would have been over. Normalize, fund, and advocate for broad ownership of hazmat suits and of non-perishable food and water, for preventing future pandemics. This may be more feasible in developing countries than developed countries, but in principle foreign aid/EA can make it feasible for developed countries as well.
Global Mini-public on AI Policy and Cooperation
Artificial Intelligence (Governance), Epistemic Institutions, Values and Reflective Processes, Great Power Relations
We'd like to fund an organization to institutionalize regular (e.g. yearly) global mini-publics to create recommendations on AI policy and cooperation; ideally in partnership with the key academic journals (and potentially the UN, major corporations, research instituions, etc.) . Somewhat analogous to globalca.org which focuses on gene editing (https://www.science.org/doi/10.1126/science.ab... (read more)
Reducing risks from laboratory accidents
Biorisk and Recovery from Catastrophe
Some life sciences research, such as gain-of-function work with potential pandemic pathogens, poses serious risks even in the absence of bad actors. What if we could eliminate biological risks from laboratory accidents? We'd like to see work to reduce the likelihood of accidents, such as empirical biosafety research and human factors analysis on laboratory equipment. We'd also like to see work that reduces the severity of accidents, such as warning systems to inform scientists if a pathogen has not been successfully deactivated and user-friendly lab strains that incorporate modern biocontainment methods.
Influencing culture to align with longtermism/EA
Effective altruism
"Everything is downstream of culture." So, basic research and practical efforts to make culture more aligned with longtermism/EA could potentially be very impactful.
Increasing social norms of moral circle expansion/cooperation
Moral circle expansion
International cooperation on existential risks and other impactful issues is largely downstream of social norms of, for example, whether foreigners are part of one's moral circle. Research and efforts to encourage social norms of moral circle expansion and cooperation to include out-group members could potentially be very impactful, especially in relevant countries (e.g., US and China) and among relevant decision-makers.
Global cooperation/coordination on existential risks
AI, Biorisk
Negative relationships between, for example, US and China are detrimental to pandemic prevention efforts, to the detriment of all people. Research on and efforts to facilitate fast, effective, and transparent global cooperation/coordination on pandemic prevention can be very impactful. Movement building on the sheer importance of this (especially among the relevant scientists and governmental decision-makers) would be especially impactful. Perhaps pandemic prevention can be "carved out" in U.S.-China relations? This also applies to other existential risks.
Reducing antibiotic resistance
Biorisk
If say a plague bacterium (maybe there are better examples) became resistant to all available antibiotics and started spreading, it could cause a pandemic like the Black Death. Research on how to behaviorally reduce antibiotic use (e.g., reduce meat consumption, convince meat companies to not use antibiotics, reduce overprescription) and how to develop new antibiotics (AI could help), and advocacy of reducing antibiotic use could potentially be high impact.
Reducing vaccine hesitancy
Biorisk
Even if we have extremely quick development of vaccines for pandemic pathogens, vaccine hesitancy can limit the impact of vaccines. Research and efforts to reduce vaccine hesistancy in general could potentially be high-impact.
Movement-building/research/pipeline for content creators/influencers
Effective altruism
Content creators/influencers have (if popular) a lot of outreach potential and earning-to-give potential. We should investigate the possibility of investing in movement-building or a pipeline into this field. Practical research on how to be a successful influencer is also likely to be broadly applicable for movement-building in general.
Incremental Institutional Review Board Reform
Epistemic Institutions, Values and Reflective Process
Institutional Review Boards (IRBs) regulate biomedical and social science research. In addition to slowing and deterring life-saving biomedical research, IRBs interfere with controversial but useful social science research, eg, Scott Atran was deterred from studying Jihadi terrorists; Mark Kleiman was deterred from studying the California prison system, and a Florida State University IRB cited public controversy as a reason to deter research. We would like to see a group focused on advocating for plausible reforms to IRBs that allow more social science research to be performed. Some plausible examples:
Concrete steps to these goals could be:
- sponsoring a prize for the first university that allowed use of Prof. Omri Ben-Shahar’s electronic checklist tool;
- setting up a journal for “Deterred Social Science Resea
... (read more)Replacing Institutional Review Boards with Strict Liability
Biorisk, Epistemic Institutions, Values and Reflective Process
Institutional Review Boards (IRBs) regulate biomedical and social science research. As a result of their risk-averse nature, important biomedical research is slowed or deterred entirely; eg, the UK human challenge trial was delayed by several months because of a protracted ethics review process and an enrollment delay in a thrombolytics trial cost thousands of lives. In the US, a plausible challenge to IRB legality can be mounted on Firs... (read more)
Re: Expert polling for everything (already listed on ftxfuturefund.org/projects)
Some questions that I think it would be very valuable to get the answers for:
1. Year with 10% chance of AGI?
2. P(doom|AGI in that year)?
3. What would it take for you to work on AGI Alignment ($ amount, other)?
1 & 2 because I think that, for AGI x-risk timelines, 10% chance (by year X) estimates should be the headline, not 50%.
And 3 should be asked specifically to the topmost intelligent/qualified/capable people in the world, as an initial investigation into this project ide... (read more)
Burying caches of basic machinery needed to rebuild civilisation from scratch
Recovery from Catastophe
Should the worst happen, and a global catastrophe happens, we want to be able to help survivors rebuild civilisation as quickly and efficiently as possible. To this end, burying caches of machinery that can be used to bootstrap development is a useful part of a civilisation recovery toolkit. Such a cache could be in the form of a shipping container filled with heavy machines of open source design, such as a wind turbine, an engine, a tractor with back hoe, an oven, basic computers and CNC fabricators, etc. Written instructions would also be included of course! Along with a selection of useful books. First we aim to put together a prototype of such a cache and test it in various locations with people of various skill levels, to see how well they fare at "rebuilding" in simulated catastrophe scenarios. Learning from this, we will iterate the design until at least 10% of simulations are successful (to what is judged to be a reasonable level). We ultimately aim to bury 10,000 such caches at strategic locations around the world. Some will be in well known locations (for the case of sudde... (read more)
Targeted social media advertising to give away high-value books
Effective Altruism, Values and Reflective Processes, Epistemic Institutions
Books are a high-fidelity means of spreading ideas. We think that high-value books are those that promote the safeguarding and flourishing of humanity and all sentient life, using evidence and reason. Many of the most valuable books have come out of the Effective Altruism (EA) movement over the last decade. We are keen for more people who want to maximize the good they do to read them. Offering those most likely to be interested in EA ideas free high-value books via targeted adverts on social media could be a highly cost effective means of growing the EA movement in a values-preserving manner. Examples of target demographics are people interested in charity and volunteering, technology, or veg*anism. Examples of books that could be offered are The Life You Can Save, Doing Good Better, The Precipice, Human Compatible, The End of Animal Farming. Perhaps a list of books could be offered, with people being allowed to chose any one.
Pipeline for writing books
Effective altruism
It's plausible that more EAs/longtermists should be writing books on the interesting subjects they are experts in, but they currently do not because of a lack of experience or other types of friction. Crowdsourced resources, networks, and grants may help facilitate this. Books written by EAs would have at least two benefits: (a) dissemination of knowledge, and (b) earning-to-give opportunities (via royalties).
This is an interesting idea; it definitely seems plausible that EAs (who often have a lot of unique knowledge!) might be underrating the benefits of writing books. Could you expand a little on what you are thinking here? (I'd also be interested to hear from anyone else with relevant experience.) How hard is it to publish a book? If you try, do you have a high chance of getting rejected? How do people usually do marketing and get people to read their stuff?
Maybe this is too cynical of me (or too internet-centric), but I doubt the main benefits would come from earning royalties (not likely to be very profitable relative to other things skilled EAs could be doing!) or spreading knowledge (just read the blog posts!). But I think trying to publish more EA books might help greatly with:
- Prestige and legibility (just like how academic papers are considered more legit than blog posts by academics and governments). It might be easier for, say, the US Democratic Party to get behind an EA-inspired pandemic-prevention plan, foreign-aid revamp, or predction-markets-y institutional-reform agenda if they could point to a prestigious book rather than
... (read more)Facilitating relocation
Economic growth, Effective altruism
People are over-averse to moving, even if it moving leads to much better opportunities (e.g., when a volcano destroyed a fraction of nearby houses, their inhabitants who were forced to move ended up better off on earnings and education, conditional on being young; see this paper). Research and incentivization can help reduce this over-aversion.
It is plausible that even EAs underconsider relocation.If so, it means a lot of impactful value may be achieved by convincing and facilitating EAs' relocation to high-impact career opportunities.
Normalizing regular wear of PPE
Biorisk
Containing a potential pandemic is extremely high-impact. If a high proportion of people regularly wore PPE, this could make the difference in determining whether or not the outbreak is stopped before it becomes a pandemic. Regularly wearing masks is much more doable than regularly wearing hazmat suits, although the political polarization of masks in certain countries is a barrier. Even so, preventing a fraction of future pandemics (which can on expectation be achieved by regular mask-wearing in a fraction of the world's countries) is still quite high impact. Applying the theory of social norms and of prestige may help normalize the regular wear of PPE. Convincing and publicizing prestigious individuals' regular mask-wearing and associating regular mask-wearing to morality may potentially be helpful on this front (in America, this may only work in certain types of communities) .
Adversarial collaborations on important topics
Epistemic Institutions
There are many important topics, such as the level of risk from advanced artificial intelligence and how to reduce it, among which there are reasonable people with very different views. We are interested in experimenting with various types of adversarial collaborations, which we define as people with opposing views working to clarify their disagreement and either resolve the disagreement or identify an experiment/observation that would resolve it. We are especially excited about combining adversarial collaborations with forecasting on any double cruxes identified from them. Some ideas for experimentation might be varying the number of participants, varying the level of moderation and strictness of enforced structure, and introducing AI-based aids.
Existing and past work relevant to this space include the Adversarial Collaboration Project, SlateStarCodex's adversarial collaboration contests, and the Late 2021 MIRI Conversations.
Platform Democracy Institutions
Artificial Intelligence (Governance), Epistemic Institutions, Values and Reflective Processes, Great Power Relations
Facebook/Meta, YouTube/Google, and other platforms make incredibly impactful decisions about the communications of billions. Better choices can significantly impact geopolitics, pandemic response, the incentives on politicians and journalists, etc. Right now, those decisions are primarily in the hands of corporate CEO’s—and heavily influenced by pressure from partisan and authoritarian governments aiming to entrench their own power. There is an alternative: platform democracy. In the past decade, a new suite of democratic processes have been shown to be surprisingly effective at navigating challenging and controversial issues, from nuclear power policy in South Korea to abortion in Ireland.
Such processes have been tested around the world, overcome the pitfalls of elections and referendums, and can work at platform scale. They enable the creation of independent ‘people’s mandates’ for platform policies—something invaluable for the impacted populations, well-meaning governments which are unable to act on speech, and ... (read more)
Operations and Execution Support for Impact
Empowering Exceptional People, Effective Altruism
The skill of running operations for building and growing a non-profit organization is often very different from doing the "core work" of that org. Figuring out operational details can suck energy away from the core work, leaving many promising people deciding not to start new orgs even when it is appropriate and necessary for scaling impact. We'd like to see an organization that could provide a sort of recruiting and matchmaking service which identifies promis... (read more)
Bridging-based Ranking for Recommender Systems
Artificial Intelligence, Epistemic Institutions, Values and Reflective Processes, Great Power Relations
Recommender systems are used by platforms like FB/Meta, YouTube/Google, Twitter, TikTok, etc. to direct the attention of billions of people every day. These systems, due to a combination of psychological, sociological, organizational, etc. factors are currently most likely to reward content producers with attention if they stoke division (e.g. outgroup animosity). Because attention is a currency that can be converted into money, power, and status, this "bias toward division” has impacts groups at every scale; from local school boards to Congress to geopolitics.
Ensuring that recommender systems can mitigate this bias is crucial to functional democracy, to cooperation on catastrophic risks (e.g. AGI, pandemics, climate change), and simply to reducing the likelihood of escalating wars. We urgently need more research on how to better design recommender systems; we need to create open source implementations that do the right thing from the start which can be adopted by cash-strapped startups; and we need a mix of pressure and support to ensure these improvements will be rapidly deployed at platform scale.
Polis lobbying
Political
Pol.is is a tool for mapping coalitions (mentioned in this 80,000 Hours podcast). Rather than running standard polls on issues, large Polis pols could be run, as Taiwan does. These would seek to build solutions which hold broad support before taking them to lobbyists.
Optimal strategies for existential security
Research That Can Help Us Improve
If we don't achieve existential security (a persistent state of negligible x-risk), an existential catastrophe is destined to happen at some point, wiping out humanity's longterm potential. Despite the incredible importance of achieving existential security, there is a lack of a consensus within the EA community on how best to do so, which is partly down to a lack of high-quality, in-depth research on this question. Instead, most research has focused on reducing specific existentia... (read more)
Calculating the cost-effectiveness of research into foundational moral questions
Research That Can Help Us Improve
All actions aiming at improving the world are either implicitly or explicitly founded on a moral theory. However, there are many conflicting moral theories and little consensus regarding which theory, if any, can be considered the correct one (this issue is also known as Moral Uncertainty). Further adding to the confusion are issues such as whom to include as moral agents (animals? AIs?) and Moral Cluelessness. These issues make it extremely dif... (read more)
Research on solving the wicked problem of underinvestment into interdisciplinary research
Economic Growth, Research That Can Help Us Improve
"Interdisciplinary research is widely considered a hothouse for innovation, and the only plausible approach to complex problems such as climate change," but are systematically underfunded and underconsidered (Bromham et al., 2016). Thinking of this problem as a wicked problem and researching how to systematically solve it (at the university, department, publication journal, and grant agency levels) could potentially be impactful.
EA influencers
Effective Altruism
More awareness of EA = more talent and money for EA
Pay A-list influencers, with followings independent of EA, to promote EA content and themes. Concentrate on influencers popular with GenZ.
Risks: lack of message fidelity
DNA banks and backup of Svalbard Global Seed Vault
Biorisk and Recovery from Catastrophe
Arguably, the most important information that the world has generated is the diversity of codes for life. Technologies are available to allow all these to be stored quickly and at low cost in DNA banks. Seed banks currently provide security for the world’s food supply. In the event of a catastrophe, it may be important to have multiple seed banks for redundancy.
Replicate the Project Ideas Competition for other types of communities than EAs
Research That Can Help Us Improve
People have contributed a lot of really insightful and promising ideas here. Given that "there are no wrong ideas in brainstorming" and that there may be systematic blind spots for effective altruists/longtermists' paradigm, perhaps doing this broad-idea-crowdsourcing exercise in other types of communities could get us new, potentially promising ideas.
Research to solve global coordination problems
Epistemic Institutions, Values and Reflective Processes
In Scott Alexander's Meditations on Moloch, Scott argues that a number of humanity's major problems (corruption, environmental extraction, arms races, existential risks from emerging technologies, etc) occur because agents are unable to coordinate for a positive global outcome. Our current major coordination mechanisms of free markets, international institutions and democracy are inadequate to solve this problem. Research needs to be done to design better c... (read more)
Reducing amount of time productive people spend doing paperwork
Economic Growth, Research That Can Help Us Improve
One example is productive researchers working in high-impact fields who are forced to write copious paperwork for grants. Another is filing taxes. Funding various approaches to reduce this problem, such as research on optimal streamlining of grant decision processes, nonprofits/volunteers/crowdsourced advice for helping fill out paperwork like taxes, improving pipelines into lab managers/personal assistants to high-productivity researchers, etc. could potentially be impactful.
Longtermism movement-building/election/appointment efforts, targeted at federal and state governments
Effective altruism
Increasing knowledge of and alignment with longtermism in government by targeted movement-building and facilitating the election/appointment of sympathetic people (and of close friends and family of sympathetic people) could potentially be very impactful. If longtermism/EA becomes a social norm in, say, Congress or the Washington 'blob', we could benefit from the stickiness of this social norm.
Studying stimulants' and anti-depressants' long-term effects on productivity and health in healthy people (e.g. Modafinil, Adderall, and Wellbutrin)
Economic Growth, Effective Altruism
Is it beneficial or harmful for long-term productivity to take Modafinil, Adderall, Wellbutrin, or other stimulants on a regular basis as a healthy person (some people speculate that it might make you less productive on days where you're not taking it)? If it's beneficial, what's the effect size? What frequency hits the best trade-off between building up tolerance vs short-term productivity gains? What are the long-term health effects? Does it affect longevity?
Some people think that taking stimulants regularly provides a large net boost to productivity. If true, that would mean we could relatively cheaply increase the productivity of the world and thereby increase economic growth. In particular, it could also increase the productivity of the EA community (which might be unusually willing to act on such information), including AI and biorisk researchers.
My very superficial impression is that many academics avoid researching the use of drugs in healthy people and that there is a bias against taking medic... (read more)
Backup communication systems
Biorisk and Recovery from Catastrophe
In the event of GCRs, conflicts or disasters, communication systems are key to sensemake and coordinate effectively. They prevent chaos and further escalation of conflicts. Today, there are many threats to the global communication infrastucutre including EMPs, widespread cyber attacks, and solar flares.
Building in reciprocal altruism into exercise, via a nonprofit with a mobile app
Effective altruism
Regular exercise likely has a very large positive impact on health and well-being. A lot of Americans do not do sufficient regular exercise, which is probably a major reason for suboptimal quality of life and subsequently suboptimal productivity.
One reason why people don't like regular exercise from going to the gym is that it feels artificial or unpleasant, and feels like a waste of time and energy. In a sense, this viewpoint is correct; moving heavy objects ... (read more)
Secure full-stack open-source computing for information security
Artificial Intelligence, Biorisk, Research that will help us improve
Much of our sensitive research and weaponry, like AI, biolabs, nuclear weapons, etc, are built upon insecure infrastructure. Think of a scenario in the future where one hacker could hack and control fleets of self-driving cars and essentially have a swarm of missiles. Real information security would need to build the full stack of computing from the hardware, OS, compilers, to application layers. It would also ideally be open-source and inspectable to ensure security.
Enlightenment at scale (provocative title :-) )
Values and Reflective Processes (?), X-risk (?)
A strong meditation practice promises enticing benefits to the meditator---less suffering, more control over ones attention and awareness, more insight, more equanimity. Brahmavihara practice promises the cultivation of loving-kindness, compassion, and empathetic joy. The world would be a much better place if everybody suffered less, had more equanimity, and felt strong compassion and empathy with other beings. But meditation is hard! Becoming a skilled meditator, and reaping these benefits, requires probably thousands of hours of dedicated practice. Most people will just not put in this amount of effort. But maybe it doesn't need to be this way. The field of meditation teaching seems underdeveloped, and innovative methods that make use of technology (e.g. neurofeedback) seem largely unexplored. We are interested in supporting scalable solutions that bring the benefits of meditation to many people.
Note:
- I don't actually know if meditation really has these benefits; this would needed to be established first (there should be quite some research on this by now). It seems plausible to me that m
... (read more)Sub-extinction event drills, games, exercises
Civilizational resilience to catastrophes
Someone should build up expertise and produce educational materials / run workshops on questions like
Differentially distributing these materials/workshops to people who live in geographical areas likely to survive at all could help rebuilding efforts in worlds where massive sub-extinction events occur.
Longtermist risk screening and certification of institutions
Artificial Intelligence, Biorisk and Recovery from Catastrophe
Companies, nonprofits and government institutions participate and invest in activities that might significantly increase global catastrophic risk like gain-of-function research or research that might increase the likelihood of unaligned AGI. We’d like to see an organisation that evaluates and proposes policies and practices that should be followed in order to reduce these risks. Institutions that commit to following these practices and submit themselves to independent audits could be certified. This could help investors and funders to screen institutions for potential risks. It could also be used in future corporate campaigns to move companies and investors into adopting responsible practices.
Develop organizations like the Institute for Advanced Study, but for longtermism
Effective altruism
The Global Priorities Institute in the UK is one example. It could be very impactful to develop similar research organizations in other locations, such as the US and the EU. (Perhaps they exist already and I just don't know about them!)
Addendum: Even the GPI could be more interdisciplinary like the IAS. e.g., branch out in addition to economics and philosophy.
Targeting movement-building efforts at top universities' career offices
Effective altruism
Wouldn't it be great if top universities' career offices were aligned with EA and with longtermism? Maybe they can use material from 80,000 hours in their help of their universities' students. An ambitious endgame is that all top universities' career offices are aligned with EA/longtermism, or at least highly aware of the paradigm and of resources like 80,000 hours, so that they can directly convince and/or facilitate students' pursuit of high-impact career options.
A public longtermism pledge/petition
Effective altruism
One way to increase the solidarity of EAs and longtermists, and to increase the gravitas with which longtermism is associated, is to have a public pledge or petition that people can sign. Public intellectuals, academic faculty, and prestigious individuals can be recruited to sign and publicly highlighted if they agree to sign. This would facilitate longtermism becoming a social norm. That this could have high impact is demonstrated, for example, by the substantial underconsideration of the risks of nuclear war (from the Russia-Ukraine war) by many public intellectuals' public comments at the moment.
Research on predicting interest in EA/longtermism
Effective altruism, Research that will help us improve
In order to help movement-builders better target their efforts, research on how to identify people who are more likely than average to be receptive to EA/longtermism could be quite impactful. Facilitating this research in the behavioral sciences can be done by funding fellowships, grants, and collaboration opportunities on the topic.
Research on predicting talent
Effective altruism, Economic growth, Research that will help us improve
The prediction of which people (e.g., prospective students, prospective employees, people to whom movement-builders target their efforts) are likely to have high potential is extremely important. But it is plausible that the current way in which these predictions are made are incomplete, cognitively biased, and substantially suboptimal. Research into identifying general or field-specific talent could be very impactful. This can be done by funding fellowships, grants, and collaboration opportunities on the topic.
'Bunker' survival research grants
Biorisk and Recovery from Catastrophe
Grants for investigation of what skills and tools/materials would be needed in ideal emergency kits to improve chances of survival/health. For example, what should you have in your bunker? Training in basic medical skills - like wilderness first aid, how to keep people mentally well under these conditions, which micronutrients should be stocked, PPE. The greater the proportion of the population that has these things on hand, may increase chances of survival.
Rationalism But For Group Psychology
Epistemic Institutions
LessWrong and the rationalist community have done well to highlight biases and help individuals become more rational, as well as creating a community around this. But most of the biggest things in life are done by groups and organizations.
We'd like to see a project that takes group psychology / organizational psychology and turns it into a rationalist movement with actionable advice to help groups be less biased and help groups achieve more impact, like how the original rationalist movement did so with individuals. We imagine this would involve identifying useful ideas from group psychology / organizational psychology literature and popularizing them in the rationalist community, as well as trying to intentionally experiment. Perhaps this could come up with better ideas for meetings, how to hire, how to attract talent, better ways to help align employees with organizational goals, better ways to keep track of projects, etc.
Disclaimer: This is just my personal opinion and not the opinion of Rethink Priorities. This project idea was not seen by anyone else at Rethink Priorities prior to posting.
Targeting movement-building efforts at top universities' administration and admissions
Effective altruism
Currently, the admissions officers of top (say, US) universities select and recruit high-potential students (modulo things like Harvard's Z list), and EA thereby uses targeted efforts to persuade and facilitate these high-potential students to go into high-impact careers. Yet, most graduates of top universities still do not do so, and a significant proportion of them go into zero-sum or negative-sum careers due to sticky social norms.
One solution m... (read more)
Headhunter Office: targeted recruitment of aligned MDs, and other mid-career professionals
Effective Altruism, Community Growth and Diversity
I am a physician, and I have several conversations a week with bright, altruistic, and burned out colleagues. These professionals are often in a position to earn to give, and also can be entrepreneurial and adept at navigating complex systems and could be future organizational leaders or 'founder types'. Currently, there are cosmetic MLM groups and others recruiting from this group of physicians looking to make their lives more fulfilling and meaningful while still earning an income - there is literally an MD Exit strategy facebook group.
I propose an EA headhunter office to recruit for the community. For example, recruiting physicians explicitly, using some of the successful techniques that pharma uses like having physicians recruit their peers. Perhaps there are similar aligned mid-career professionals in law, public administration, engineering, etc.
New academic publishing system
Research that will help us improve, Epistemic Institutions, Empowering Exceptional People
It is well-known that the incentive structure for academic publishing is messed up. Changing publish-or-perish incentives is hard. However, one particular broken thing is that some journals operate on a model where they rent out their prestige to both authors (who pay to have their works accepted) and readers (who pay to read), extracting money from both while providing little value except their brand. This seems like a situation that coul... (read more)
Institute/Grants for improving the science of indoor air quality
Biorisk
During Covid we learned that ‘air is the new poop’ in terms of hygiene. Improving indoor air quality can prevent respiratory pathogen transmission both in the case of a pandemic and for general health. A granting agency could support advances in indoor air quality and their implementation such as in airplanes and classrooms.
EA Health Institute/Chief Wellness Officer
Empowering Exceptional People, Effective Altruism, Community Building
Optimizing physical and mental health can improve cognitive performance and decrease burnout. We need EAs/longtermists to have the health resilience to weather the storm - physical fitness, sleep, nutrition, mental health. An institution could be created to assist EA aligned organizations and individuals. Using best practices from high performance workplace health, both personal and organizational, and innovative new ideas, a wellness team could help EAs have sustainable and productive careers. This could be done through consulting, coaching, preparation of educational materials or retreats. From a community growth perspective, EA becomes more attractive to some when one doesn’t have to sacrifice health for deeply meaningful work.
(Disclosure -I'm a physician/physician wellness SME - helping with this could be a good personal fit)
Support for EAs having children
Empowering Exceptional People, Effective Altruism
Children of EAs are much more likely to become EAs (100-1000x?) and future generations of EAs may have a large impact. Having children usually means a pause in work which is poorly compensated and difficult to time. I propose an institute to support EAs wishing to have children. EAs could be could be supported with fertility costs, including egg freezing, and be given grants for parental leave which improve parental and child health outcomes. There are many trade offs in parenting, which could be discussed in an EA parenting forum. Building a community could benefit these EA parents and their children.
Evidence for 100-1000x estimate? What is the base rate for children following their parents? When I've seen this discussed before, the conclusion is usually that memetic transfer of EA is much easier than genetic transfer of EA.
Regular prizes/awards for EA art
Effective Altruism
Works of art (e.g. stories, music, visual art) can be a major force inspiring people to do something or care about something. Prizes can directly lead to work (see for example the creative writing contest), but might also have an even bigger role in defining and promoting some type of work or some quality in works. Creating a (for example) annual prize/award scheme might go a long way towards defining and promoting an EA-aligned genre (consider how the existence of Hugo and Nebula awards helps define and promote science fiction). The existence of a prestigious / high-paying prize for the presence of specific qualities in a work is also likely to draw attention to those qualities more broadly; news like "Work X wins award for its depiction of [thoughtful altruism] / [the long-term future] / [epistemic rigor under uncertainty]" might make those qualities more of a conversation topic and something that more artists want to depict and explore, with knock-on effects for culture.
Wild animal suffering in space
Space governance, moral circle expansion.
Terraforming other planets might cause animals to come to exist in these planets, either because of intentional or unintentional behaviors. These animals might live net negative lives.
Also, we cannot rule out the possibility that there are already wild "animals" (or any form of sentient beings) who might be suffering from net negative lives in other planets. (this does not relate directly to the Fermi Paradox, which is highly intelligent lives, not lives per se)
Relevant research include:
Prosocial social platforms
Epistemic institutions, movement-building, economic growth
The existing set of social media platforms is not particularly diverse, and existing platforms also often create negative externalities: reducing productive work hours, plausibly lowering epistemic standards, and increasing signalling/credentialism (by making easily legible credentials more important, and in some cases reducing the dimensionality of competition, e.g. LinkedIn reducing people to their most recent jobs and place of study, again making the competition for cred... (read more)
Megastar salaries for AI alignment work
Artificial Intelligence
Aligning future superhuman AI systems is arguably the most difficult problem currently facing humanity; and the most important. In order to solve it, we need all the help we can get from the very best and brightest. To the extent that we can identify the absolute most intelligent, most capable, and most qualified people on the planet – think Fields Medalists, Nobel Prize winners, foremost champions of intellectual competition, the most sought-after engineers – we aim to offer them salaries competitive with top sportspeople, actors and music artists to work on the problem. This is complementary to our AI alignment prizes, in that getting paid is not dependent on results. The pay is for devoting a significant amount of full time work (say a year), and maximum brainpower, to the problem; with the hope that highly promising directions in the pursuit of a full solution will be forthcoming. We will aim to provide access to top AI alignment researchers for guidance, affiliation with top-tier universities, and an exclusive retreat house and office for fellows of this program to use, if so desired.
Preventing factory farming from spreading beyond the earth
Space governance, moral circle expansion (yes I am also proposing a new area of interest.)
Early space advocates such as Gerard O’Neill and Thomas Heppenheimer had both included animal husbandry in their designs of space colonies. In our time, the European Space Agency, the Canadian Space Agency, the Beijing University of Aeronautics and Astronautics, and NASA, have all expressed interests or announced projects to employ fish or insect farming in space.
This, if successful, might multiply the suffering of farmed animals by many times of the numbers of farmed animals on earth currently, spanned across the long-term future. Research is needed in areas like:
- Continuous tracking of the scientific research on transporting and raising animals in space colonies or other planets.
- Tracking, or even conducting research on the feasibility of cultivating meat in space.
- Tracking the development and implementation of AI in factory farming, which might enable unmanned factory farms and therefore make space factory farming more feasible. For instance, the aquaculture industry is hoping that AI can help them overcome major difficultie
... (read more)Build LMIC university capacity
Economic growth, Empowering exceptional people
Universities in LMICs often have limited access to funding. Additional funding could enable many good outcomes including:
Funding could be focussed on issues of concern to EAs, such as pandemics, or could be unrestricted to boost overall university capacity. As well as funding universities, funds could be provided for networks, independent labs, access to journals, travel and conferences, spinout companies etc.
Wikipedia research/infrastructure/support
Epistemics
Wikipedia is a hugely valuable public resource. Internally however, there are slow processes and aging mechanisms, as in many institutions. Run a research and lobbying organisation to help wikipedia maximise its value to the world.
EA-themed Superhero Graphic Novel / Shounen Anime / K Drama
Effective Altruism Meta, Community Building
I really like to think about that Superman fanfic where he tried to aim for 'most good'. Many existing superhero stories could be rewritten so the main protagonists tries to maximize their impact. I know non-fiction movies/documentaries were mentioned but I think the 3 types of media I mentioned have the potential to become really popular (are consumed by vast number of teenagers and (young) adults globally. It's a risk (it could be a flop), but I think one we could take. I am pretty confident a big enough budget can 'buy quality' so it would be better than average story.
Funding Stress/Penetration Tests of vital orgs/infrastructure
Cyber Risks, Cybersecurity
Most orgs don't spend enough on ensuring their infrastructure is safe from hackers and we should ensure that labs working on AI safety, biorisk companies, EA orgs etc. are safe from malicious hackers.
Internal market for (EA) recruitment
Effective Altruism Operations, Economic Growth
Open source tool that would allow companies/orgs to set up internal (prediction) markets where all employees could bet on which candidate would be the best fit and be awarded points/real money for every month they stayed at the company.
Redefine humanity & assisting its transition
Artificial intelligence, values and reflective processes
As humanity inevitably evolves into coexistence with AI – the adage “if a man will not work, he shall not eat” needs to be redefined. Apart from AI’s early displacement effects already apparent (cue autonomous driving/trucking industry etc), humanity’s productivity function will continue rising due to the intrinsic nature of AI (consider 3D printing normal/lux goods at economies of scale), so much so that even plentitude becomes a potential problem. (In the usual then followed citation of ‘what about the African kids’ – kindly note this is a separate distribution problem) Ultimately – we should be contributing towards smoothing the AI transition curve and managing initial displacement by AI followed by proactively managing integration.
Increasing Earth’s probability of survival
Space governance
Currently as we transition from a Kardashev Type 0 to Type 1 civilization, our probability of encountering/alerting other civilisations increases exponentially. This is somewhat ironic that we may fall upon our impetus. Citing dark forest theory that the end game is such that ‘lacking assurances, the safety option for any species is to annihilate other life forms before they have a chance to do the same’, means that humanity is immediately on the defensive. (Applying a chronological framework and assuming linearity of time) As of such – we should fund ways to increase our probability of survival (by either deterrence mechanisms, signalling non threat or camouflage) such that we may evolve uninterrupted. (This also assumes we don’t kill ourselves first, the probability of which is sadly also non zero)
Just throwing out crazy suggestions (I’m sensing that’s what the thematic is here) would be something like hyper gravity generation device that bends observable light emitted from our planet, so much so that when observed – we would look like a blackhole.
Combatting DeepFake
Epistemic Institutions, Artificial Intelligence
As AI advances –Numerous high quality deepfake videos/images are being produced at an alarmingly increasing rate. Delving into the question ‘What happens when we can’t trust our eyes and ears anymore?’, This immediately raises obvious signals that it will affect many industries such as journalism, military, celebrities, government etc. Proactively funding a superior ML anti-deepfake bot for commercial use is important such that images/videos can be properly verified. The end game will likely come down to some degree of superior computing power since both are ML based algos – hence the advantage here would be first mover and/or altruistic (think along the same example of free antivirus software) in nature.
Anti-Pollution of the universe
Space governance
As we take one small step for man, our giant leap for humanity leaves footprints of toxicity that we justify as ‘negative externalities’. There are currently 20,000 catalogued objects comprised of rocket shards, collision debris and inactive satellites which cause major traffic risks in orbit around our planet whilst also likely polluting the universe. As Boeing, OneWeb, SpaceX etc increase their launches, we similarly add to the congestion and space collision probabilities. (read as disasters waiting to happen). There are currently NO debris removal methods. If we’ve learnt anything from our current micro history of mankind on Earth – it’s that the nature/universe around us is important since we’re intricately linked and that there are costs to our polluting behaviour in the pursuit of ‘territory/energy etc’. Hence when we’re playing at the macro cosmic level – it is even more imperative that we get this framework/relationship/thought process right.
Pragmatic forecasting training
Epistemic institutions
There is a big jump between reading Superforecasting and actually doing forecasting, especially at work. One problem is that the book is written as a popular book, and so doesn't cover the specifics you need - e.g. what techniques should you use to combine data to get a base rate? It would be useful to have something more textbooky which teaches specific techniques and gives lots of worked examples and exercises. Furthermore, there are many additional challenges of implementing forecasting in a policy or funder environment such as:
It would be valuable to have a consultancy helping organisations such as funders and government departments implement forecasting in a real-world context. This consultancy could then over time build up a course or textbook that teaches what they have learned to a wider audience.
DIY decentralized nucleic acid observatory
Biorisk and Recovery from Catastrophes
As part of the larger effort of building an early detection center for novel pathogens, a smaller self-sustaining version is needed for remote locations. The ideal early-detection center would not only have surveillance stations in the largest hubs and airports of the world, but also in as many medium sized ones as possible. For this it is needed to provide a ready-made, small and transportable product which allows meta-genomic surveillance of wastewater or air ventilation. One solution would be designing a workflow utilizing the easily scalable and portable technology of nanopore sequencing and combining it with a workflow to extract nucleic acids from wastewater. The sharing of instructions on how to build and use this method could lead to a "do it yourself" (DIY) and decentralized version of a nucleic acid observatory. Instead of staffing a whole lab at a central location, it would be possible to only have one or two personnel in key locations who use this product to sequence samples directly and only transmit the data to the larger surveillance effort.
Credible expert Q&A forums
Epistemic institutions
Decisionmakers (e.g. funders and policymakers) tend to use a mixture of desk research, interviews with experts, and workshops with experts to inform their decisions. Online forums where questions can be asked of experts could be a useful part of this process. Forums are useful compared with desk research as information can be sought that may not be covered in existing sources. They are useful compared with interviews and workshops as they require less organisational overhead to get expert input and what i... (read more)
AI alignment: Evaluate the extent to which large language models have natural abstractions
Artificial Intelligence
The natural abstraction hypothesis is the hypothesis that neural networks will learn abstractions very similar to human concepts because these concepts are a better decomposition of reality than the alternatives. If it were true in practice, it would imply that large NNs (and large LMs in particular, due to being trained on natural language) would learn faithful models of human values, as well as bound the difficulty of translating between the model and human ontologies in ELK, avoiding the hard case of ELK in practice. If it turns out that the natural abstraction hypothesis is true at relevant scales, this would allow us to sidestep a large part of the alignment problem, and if it is false then this allows us to know to avoid a class of approaches that would be doomed to fail.
We'd like to see work towards gathering evidence on whether natural abstractions holds in practice and how this scales with model size, with a focus on interpretability of model latents, and experiments in toy environments that test whether human simulators are favored in practice. Work towar... (read more)
Refinement of idea #33, "A fund for movies and documentaries":
I'd like to see filmmakers (including screenwriters and directors) working on EA-inspired films collaborate with social scientists and other subject-matter experts to ensure that their films realistically depict EA issues (such as x-risks) and social dynamics. These collaborations can help filmmakers avoid pitfalls like those committed by Don't Look Up and The Ministry for the Future.[1]
- ^
... (read more)From this review: "But while here and there an offhand reference to some reluctant group or other is made, they are, in Ministry, always feckless. The initial disaster undermines India’s Hindu nationalist party, rather than strengthening it. Further disasters are met with turns to socialism. The anti-fossil fuel terrorism that is portrayed (and both criticized and seen as necessary by varying characters) does not provoke anti-environmental terrorism in response. One particular striking example is about two-thirds of the way through the novel, when a small American town is evacuated in the name of half-Earth. While not welcomed, this evacuation is accepted in a way that is all but impossible to imagine, at least while we, looking up from
Pro-immigration advocacy outside the United States
Economic Growth
Increasing migration to rich countries could dramatically reduce poverty and grow the world economy by up to 150%. Open Philanthropy has long had pro-immigration reform in the U.S. as a focus area, but the American political climate has been very hostile to and/or polarized on immigration, making it harder to make progress in the U.S. However, other high-income countries might be more receptive to increasing immigration, and would thus be easier places to make progress. For example, according to a 2018 Pew survey, 81% of Japanese citizens support increasing or keeping immigration levels about the same. It would be worth exploring which developed countries are most promising for pro-immigration advocacy, and then advocating for immigration there.
What this project could look like:
Related posts:
- Which countries are most receptiv
... (read more)Extinction-level events outside of biorisk and nuclear catastrophes
Biorisk and Recovery from Catastrophe
In order to prepare for worst-case catastrophes, we need to anticipate them. Biological weapons and nuclear catastrophes are two well-identified threats to humanity's long term survival, as is climat change. However, there may be emerging risks that are yet to be addressed by policymakers or the EA community.
We'd be interested in convincing works highlighting credible, large-scale risks that are overlooked by most forecasters and the EA community, as well as any recovery strategies that are applicable.
Longtermist democracy / institutional quality index
Values and Reflective Processes, Epistemic Institutions
Several indices exist to quantify the degree of liberal democracy in all countries and territories around the world, like Freedom in the World and the EIU's Democracy Index. These indices are convenient for describing and comparing the state of liberal democracy in different countries, because they distill the various complicated aspects of a state's political system into one or more numbers that are easy for a layperson to understand.
We propose a "democracy index" that emphasizes the qualities of political systems that are most relevant to making the long-term future go well. Such qualities could include voting systems, free and fair elections, voter competence, and capacity for long-term planning in government - and the set of qualities used could be based on research such as this post. This index would help make analysis of countries and territories' political systems more accessible to EAs/longtermists who aren't political scientists, since it would distill them down to a few easy-to-understand numbers. It would also help the longtermist community track progress towards bet... (read more)
Research on solving wicked problems
Economic growth, Values and Reflective Processes
It seems that many (almost all?) of the outstanding problems we effective altruists wish to solve are wicked problems. A better general understanding on how wicked problems could be solved may potentially be very impactful. This can be done by establishing relevant fellowships, grants, and collaboration opportunities to facilitate research on this topic.
Targeted practical statistical training
Economic Growth, Values and Reflective Processes
"Human cognition is characterized by cognitive biases, which systematically lead to errors in judgment: errors that can potentially be catastrophic (e.g., overconfidence as a cause of war). For example, a strong case can be made that Russia's invasion of Ukraine has been an irrational decision of Putin, a consequence of which is potential nuclear war. Overconfidence is a cause of wars and of underpreparation for catastrophes (e.g., pandemics, as illustrated by the COVID-19 pandemic).
One way to reduce detrimental and potentially catastrophic decisions is to provide people with statistical training that can help empower beneficial decision-making via correct calibration of beliefs. (Statistical training to keep track of the mean past payoff/observation can be helpful in a general sense; see my paper on the evolution of human cognitive biases and implications.) At the moment, statistical training is provided to a very small percentage of people, and most provisions of statistical training are not laser-focused on the improvement of practical learning/decision-making capabilities, but for other indir... (read more)
Optimal 90-second pitches for EA/longtermism
Effective altruism
Longtermism is nuanced; a full discussion requires a large amount of time. It is possible that more people than currently may be interested in learning more about the movement if they are presented with a short but compelling pitch that is suited for the quickness of many people's lifestyle. (I've stated a spontaneous and very suboptimal pitch for EA on at least one occasion, which I regret.)
Optimized 90-second or so pitches may potentially help the movement's outreach. Persuasive pitches (each focused on each of a myriad of topics/angles that the listener may be interested in) can be selected by community contests/focus groups and posted online, both for viewing and for informing movement builders' efforts.
This list should have karma hidden and entries randomised. I guess most poeple do not read and vote all the way to the bottom. I certainly didn't the first time I read it.
I agree; something like Reddit's contest mode would be useful here. I've sorted the list by "newest first" to avoid mostly seeing the most upvoted entries.
EA follower bounties
EA community building
Offer a fixed rate for subscribers to EA accounts on different platforms. Ask forum users to note all the accounts above a certain size they can think of which they think post quality EA content and remunerate all according to the same standard, per platform. Alternatively only pay midsized accounts of those for whom it's not paid already or on platforms we would like more coverage on.
Metaculus Competitor
Forecasting
Prediction markets don't incentivise long term questions and the Good Judgement Open has slow question creation. This leaves Metaculus as the only place to forecast questions over long time horizons. This is too important a problem to have a single organisation solving. At least one more forecasting organisation should exist to try and build the infrastructure necessary to take forecasts, improve individual forecasting, display track records and make 5 - 1000 year forecasts.
Fund Sentinel, a nationwide pandemic early response system (originally suggested by alexrjl)
Biowarfare
Fund the biosecurity program explained on this podcast. Any time anyone gets sick you sequence a sample. Any unknown genetic material gets sequenced again at a higher level. This allows for rapid response to new pathogens.
Subsidise catastrophic risk-related markets on prediction markets
Prediction markets and catastrophic risk
Many markets don't exist because there isn't enough liquidity. A fund could create important longtermist markets on biorisk, AI safetry and nuclear war by pledging to provide significant liquidity once created. This would likely still only work for markets resolving in 1-10 years, due to inflation, but still*.
*It has been suggested to run prediction markets which use indices rather than currency. But people have shown reluctance to bet on ETH markets, so might show reluctance here too.
FTX, which itself runs prediction markets, might be particularly well-suited for prediction-market interventions like this. I myself think that they could do a lot to advance people's understanding of prediction markets if in addition to their presidential prediction market, they also offered a conditional prediction market of how an indicator like the S&P 500 would do 1 week after the 2024 election, conditional on the Republicans winning vs the Democrats winning. Conditional prediction markets for important indicators on big national elections would provide both directly useful info in addition to educating people about prediction markets' potential.
Eliminate disease-bearing mosquitos (originally suggested by David Manheim)
Malaria
Act on the long-running plan to design and release mosquitos to outcompete those which spread malaria thereby avoiding infection.
Purchase a top journal
Metascience
Journals give bad incentives to academics - they require new knowledge to be written in hard to understand language, without pre-registration at great cost and sometimes focused on unimportant topics. Taking over a top journal and ensuring it incentivised high quality work on the most important topics would begin to turn the scientific system around.
We could, of course, simply get the future fund to pay for this. There is, however, an alternative that might be worth thinking about.
This seems like the kind of thing that dominant assurance contracts are designed to solve. We could run a Kickstarter, and use the future fund to pay the early backers if we fail to reach the target amount. This should incentivise all those who want the journals bought to chip in.
Here is one way we could do this:
- Use a system like pol.is to identify points of consensus between universities. This should be about the rules going forward if we buy the journal. For example, do they all want pre-registration? What should the copyright situation be? How should peer-review work? How should the journal be ran? etc
- Whatever the consensus is, commit to implementing it if the buyout is successful
- Start crowdsourcing the funds needed. To maximise the chance of success, this should be done using a DAC (dominant assurance contract). This works like any other crowdfunding mechanism (GoFundMe, Kickstarter, etc), except we have a pool of money that is used to pay the early backers if we fail to meet the goal. If the standard donation size we're asking the unis for i
... (read more)Approval Voting in the UK
Politics
The Centre for Election Science has done good work pushing approval voting in the US. In the UK there aren't ballot initiatives, but both political parties could allow approval voting in their constituencies. If they did then it would be easier to push at a national level.
Nuclear Funding Shortfall
Nuclear Risk
There has been a significant shortfall in nuclear risk funding. The most effective elements of this could be covered by the fund.
Politician forecasting stipend
Politics, better epistemics
Many people think politicians are underpaid. Many think they have a poor grasp of the likelihood of future events. Offer every Senator and Representative a yearly sum to make public predictions about future public statistics. The forecasting would help them correct their own errors and provide a valuable source of information on who makes good decisions about the future and who doesn't.
EA/AI Hiring Round
Effective Altruism/ AI Safety
Meet with a variety of organisations and design an short set of questions to best predict good candidates for roles. Allow anyone to take this test every 3 months and apply for a broad range of positions eg all EA ops roles in their city or all AI safety roles. Hire more, higher quality candidates.
Automated Open Project Ideas Board
The Future Fund
All of these ideas should be submitted to a board where anyone can forecast their value in dollars lives saved per $ as rated by a trusted research organisation, say Rethink Priorities. The forecasts can be reputation or prediction markets. Then that research organisation checks 1% of the ideas and scores them. These scores are used to weight the other forecasts. This creates a scalable system for ranking ideas. Then funders can donate to them as they see fit.
Superforecasting team
Global catastrophic risks
We know that top forecasters exist, but few are currently employed to predict around long term risks. These forecasters should be supported by developers to help maximise their accuracy and output. Multiple organisations could employ 100s or 1000s of top forecasters to analyse developing situations and suggest outcomes that are most likely to resolve them for the interests of all consciousness.
Funding private versions of Longtermist Political Institutions to lay groundwork for government versions
Some of the seemingly most promising and tractable ways to reduce short-termist incentives for legislators are Posterity Impact Assessments (PIA) and Futures Assemblies (see Tyler John's work). But, it isn't clear just how PIAs would actually work, e.g. what would qualify as an appropriate triggering mechanism, what evaluatory approaches would be employed to judge policies, how far into the future policies can be evaluated. It seems like it would be relatively inexpensive to fund an organization to do PIAs in order to build a framework which a potential in-government research institute could adopt instead of having to start from scratch. The precedent set by this organization seems like it would also contribute to reducing the difficulty of advocating for longtermist agency/research institutes within government.
Similarly, it would be reasonably affordable to run a trial Futures Assembly wherein a representative sample of a country's population is formed to deliberate over how and to what extent policy makers should consider the interests of future persons/generations. This would provide a precedent for potential government funded versions as well as a democratically legitimate advocate for longtermist policy decisions.
Basically, EAs could lay the groundwork for some of the most promising/feasible longtermist political institutions without first needing to get legislation passed.
Targeted facilitation of high-impact career pivots for ex-academics
Effective altruism
Effective altruists/longtermists have targeted their movement-building efforts to young people (undergraduate and high-school students), an effective strategy given that young people are more likely to be in the process of career exploration and investments in them will be long-lasting.
Another effective movement-building strategy may be to help Ph.D. graduates, postdocs, etc. who are pivoting out of academia. Ex-academics are likely to have difficult-to-obtain, and often impactful/generalizable skills, and are likely undervalued by the hypercompetitive academic job market (due to academics' strong, social-norm-based preference for academic jobs and consequent oversupply). Ex-academics are likely to be in the process of career exploration. Targeted outreach, fellowships, and careering coaching by student organizations and EA movement-building experts may help direct more of these ex-academics to high-impact career pivots.
Participatory longtermism
Values and reflective processes, Effective Altruism
Most longtermist and EA ideas come from a small group of people with similar backgrounds, but could affect the global population now and in the future. This creates the risk of longtermist decisionmakers not being aligned with that wider population. Participatory methods aim to involve people decisionmaking about issues that affect them, and they have become common in fields such as international development, global health, and humanitarian aid. Although a lot could be learned from existing participatory methods, they would need to be adapted to issues of concern to EAs and longtermists. The fund could support the development of new participatory methods that fit with EA and longtermist concerns, and could fund the running of participatory processes on key issues.
Additional notes:
Tools for improved transmission of tacit knowledge
Biorisk and recovery from catastrophe
Many scientific and technological skills require learning through apprenticeship under a more experienced practitioner, and can't easily be described in writing. If a global catastrophe breaks the transmission of skills from masters to apprentices, it may be difficult to recover those skills. This would make recovery from catastrophe difficult. But there may be ways of improving the recording of these skills, such as through video or methods of observing expert performan... (read more)
SEP for every subject
Epistemic institutions
Create free online encyclopedias for every academic subject (or those most relevant to longtermism) written by experts and regularly updated. Despite the Stanford Encyclopedia of Philosophy being widely-known and well-loved there are few examples from other subjects. Often academic encyclopedias are both behind institutional paywalls and not accessible on sci-hub (e.g. https://oxfordre.com/). This would provide decisionmakers and the public with better access to academic views on a variety of topics.
Movement-building targeted at existential-risk-relevant fields' international scientific communities
Biorisk, Artificial Intelligence
Scientists of the Manhattan Project built the first nuclear bombs, the development and use of which normalized nuclear proliferation. Contrast this with bioweapons, which in principle could also have been normalized if not for the advocacy of scientists like Matthew Meselson, which led to a lasting international agreement to not develop bioweapons (Biological Weapons Convention).
Targeted efforts to build the movement of reducing catastrophic/existential risks (and longtermism in general) specifically in the international scientific communities of fields that are highly relevant to certain existential risks, whose lasting cooperation would be crucial for the non-realization of these risks, could potentially be very impactful. Some potential approaches include funding of fellowships/grants/collaboration opportunities, creating scientific societies/conferences, and organizing advocacy/outreach/petitions.
Causal microfoundations for behavioral science
Artificial Intelligence, Values and Reflective Processes
The science of human behavior is afflicted by a replication crisis. By some estimates, over half of the empirical literature does not replicate. A significant cause of this problem is undertheorization. Without a cumulative theoretical framework from which to work, researchers often lack meaningful hypotheses to test, and so instead default to their personal, often culturally biased folk intuitions. Their resulting interpretations of studies’ data thus frequently fail to replicate and generalize (See the seminal paper of Michael Muthukrishna and my advisor Joe Henrich.)
Finding the correct causal microfoundations for behavioral science can provide a deeper understanding of precisely when we can extrapolate empirical findings out-of-sample. This could be especially helpful for making externally valid predictions in historically unprecedented situations (e.g., regarding emergent technologies or anthropogenic catastrophic/existential risks), for which much of the relevant data required for empirically estimating policy counterfactuals may not yet exist.
One area where the correct causal... (read more)
Massive US-China exchange programme
Great power conflict, AI
Fund (university) students to live in the other country in a host family: between US-China, Russia-US, China-India, potentially India-Pakistan. This is important if one thinks that personal experience make it less likely that individuals incentivise or encourage escalation, war and certain competitive dynamics.
Making Future Grantmaking More Optimal
Effective altruism
Focus Groups Exploring Longtermism / Deliberative Democracy for Longtermism
Epistemic Institutions, Values and Reflective Processes
Right now longtermism is being developed within a relatively narrow set of stakeholders and participants relative to the broad set of people (and nonhumans) that would be affected by the decisions we make. We'd like to see focus groups that attempt to engage a more diverse group of people (diversity across many axes including but not limited to race, gender, age, geography, and socioeconomic status) and attempt to explain longtermism to them and explore what visions they have for the future of humanity (and nonhumans). Hopefully through many iterations we can find a way to go across what is likely rather large initial inferential distance to explore how a broader and more diverse group of people would think about longtermism once ideally informed. This can be related to and informed by engaging in deliberative democracy. This also could be helping to initiate what longtermists call "the long reflection".
Disclaimer: This is just my personal opinion and not the opinion of Rethink Priorities. This project idea was not seen by anyone else at Rethink Priorities prior to posting.
Broadening statistical education
Economic Growth, Values and Reflective Processes
Human cognition is characterized by cognitive biases, which systematically lead to errors in judgment: errors that can potentially be catastrophic (e.g., overconfidence as a cause of war). For example, a strong case can be made that Russia's invasion of Ukraine has been an irrational decision of Putin, a consequence of which is potential nuclear war. Overconfidence is a cause of wars and of underpreparation for catastrophes (e.g., pandemics, as illustrated by the COVID-19 pande... (read more)
Mental health treatment to prevent anthropogenic catastrophic/existential risks
Biorisk and Recovery from Catastrophe
Issues of mental health can be very harmful to the well-being of the self and others. The degree to which this harm can occur can, when combined with technology, even result in catastrophic/existential risks. (The Russian invasion of Ukraine, the cause of which may be the mental state of Putin, can plausibly lead to nuclear war. Another example is engineered pandemics.) Given the disproportionately anthropogenic skew of catastrophic/existential risks, research/funding/advocacy for mental health treatment (general or targeted) may help prevent such risks.
Reminds me of some of the proposals here: https://forum.effectivealtruism.org/posts/LpkXtFXdsRd4rG8Kb/reducing-long-term-risks-from-malevolent-actors
Regulatory markets of AI safety
Artificial Intelligence
A political think tank to refine and push for regulatory markets of AI safety in as many countries as possible. Jack Clark, Gillian K. Hadfield: “We propose a new model for regulation to achieve AI safety: global regulatory markets. We first sketch the model in general terms and provide an overview of the costs and benefits of this approach. We then demonstrate how the model might work in practice: responding to the risk of adversarial attacks on AI models employed in commercial drones.”
It is probably h... (read more)
Proportional prizes for prescient philanthropists
Effective Altruism, Economic Growth, Empowering Excetional People
A low-tech alternative to my proposal for impact markets is to offer regular, reliable prizes for early supporters of exceptionally impactful charities. These can be founders, advisors, or donors. The prizes would not only go to the top supporters but proportionally to almost anyone who can prove that they’ve contributed (or where the charity has proof of the contribution), capped only at a level where the prize money is close to the cost of the administrative overhead.
Donors may be rewarded in proportion to the aggregate size of their donations, advisors may be rewarded in proportion to their time investment valued at market rates, founders may be rewarded in proportion to the sum of both.
If these prizes are awarded reliably, maybe by several entities, they may have some of the same benefits as impact markets. Smart and altruistic donors, advisors, and charity serial entrepreneurs can accumulate more capital that they can use to support their next equally prescient project.
A better open-source human-legible world-model, to be incorporated into future ML interpretability systems
Artificial intelligence
[UPDATE 3 MONTHS LATER: Better description and justification is now available in Section 15.2.2.1 here.]
It is probable that future powerful AGI systems will involve a learning algorithm that builds a common-sense world-model in the form of a giant unlabeled black-box data structure—after all, something like this is true in both modern machine learning and (I claim) human brains. Improving our ability, as humans, to look inside an... (read more)
Putting Books in Libraries
Effective Altruism
The idea of this project is to come up with a menu of ~30 books and a list of ~10000 libraries, and to offer to buy for each library, any number of books from the menu. This would ensure that folks interested in EA-related topics, who browse a library, discover these ideas. The books would be ones that teach people to use an effective altruist mindset, similar to those on this list. The libraries could be ones that are large, or that that serve top universities or cities with large English-speaking populations.
The case for the project is that if you assume that the value of discovering one new EA contributor is $200k, and that each book is read once per year (which seems plausible based on at least one random library) then the project will deliver far greater than the financial costs, of about $20 per book. The time costs would be minimised by doing much of the correspondence with libraries over the space over a short period of weeks to months. It also can serve as a useful experiment for even larger-scale book distributions, and could be replicated in other languages.
I really like this project idea! It's ambitious and yet approachable, and it seems that a lot of this work could be delegated to virtual personal assistants. Before starting the project, it seems that it would be valuable to quickly get a sense of how often EA books in libraries are read. For example, you could see how many copies of Doing Good Better are currently checked out, or perhaps you could nicely ask a library if they could tell you how many times a given book has been checked out.
In terms of the cost estimates, how would targeted social media advertising compare? Say targeting people who are already interested in charity and volunteering, or technology, or veg*anism, and offering to send them a free book.
I like this idea, but I wonder - how many people / students actually use physical libraries still? I don't think I've used one in over 15 years. My impression is that most are in chronic decline (and many have closed over the last decade).
Towards Better Epistemology in Medicine
Epistemic Institutions, Values and Reflective Processes
Medicine is a field subject to an incentive landscape that can, among other issues, encourage pathological risk aversion in treatment and research, which holds back patients getting the care
with the greatest expected value to them and limits our ability as a society to adapt
to new and changing health issues such as global pandemics. Medical professionals are often trained in a narrow set of epistemic norms that lead to slow updates on new evidence, overreliance on individual decisionmaking, and difficulty communicating about complex tradeoffs. The unavoidable closeness to moral and ethical issues, as well as difficulties in reasoning about decisions that hold lives directly in the balance, exacerbate the problem.
We're interested in projects that address these problems, perhaps including the following:
- Literature and media that promotes truth-seeking and expected-value-thinking norms
in medicine, whether explicit in non-fiction or training material, or in fictional settings
- Resources that seek to aggregate medical evidence relevant to a specific condition or
clinical appli... (read more)
Impact markets to smooth out retroactive funding
Effective Altruism, Empowering Excetional People, Economic Growth, Epistemic Institutions
Yonatan Cale already made the case for retroactive funding, i.e. that it’s easier to tell what has succeeded than what will succeed. The questions of what will succeed, in turn, can be answered by a market.
Investors will try to predict which charities will succeed to the point of receiving retroactive funding. A retroactive funder can make larger grants in proportion to their reduction in uncertainty (5–10x), time savings from having to do less vetting (~ 2x), and delay (~ 1.5x). Hence investors with enough foresight can even make a profit and turn the prediction of retro fund decisions into their business model. Promising charities can bootstrap rapidly with these early financial injections, successful serial charity entrepreneurs can accumulate more and more capital to reinvest into their next charity venture, and funders save time because they have to do only a fraction of the vetting.
We – Kenny Bambridge, Matt Brooks, Dony Christie, Denis Drescher, and a number of advisors – are actively working toward this goal. I’ve been thinking about the m... (read more)
Rule of Law Fund
Values and Reflective Processes and Economic Growth
A strong rule of law helps ensure equity, human rights, property rights, contract enforcement, and due process. Many countries are still developing their legal systems. Between 2010 and 2020 twenty-four different countries ratified a constitution. The legal systems that evolve today will have a lasting impact on future generations.
This fund would offer funding for organizations and individuals engaged in legal scholarship and litigation that align with the Future Fund’s guiding principles, with a specific focus on strengthening the rule of law in countries with less developed legal institutions.
Unified, quantified world model
Epistemic Institutions, Effective Altruism, Values and Refelctive Processes, Research That Can Help Us Improve
Effective altruism started out, to some extend, with a strong focus on quantitative prioritization along the lines of GiveWell’s quantitative models, the Disease Control Priorities studies, etc. But they largely ignore complex, often nonlinear effects of these interventions on culture, international coordination, and the long-term future. Attempts to transfer the same rigor to quantative models of the long-term future (such as Tarsney’s set of models in The Epistemic Challenge to Longtermism) are still in their infancy. Otherwise effective altruist prioritization today is a grab bag of hundreds of considerations that interact in complex ways that (probably) no one has an overview over. Decision-makers may forget to take half of them into account if they haven’t recently thought about them. That makes it hard to prioritize, and misprioritization becomes more and more costly with every year.
A dedicated think tank could create and continually expand a unified world model that (1) is a repository of all considerations that affect altruistic decisi... (read more)
Monitoring and advocacy to make Zoonotic Risk Prediction projects safer
Biorisk and recovery from catastrophe
Following COVID-19, a great deal of funding is becoming available for "Zoonotic Risk Prediction" projects, which intend to broadly sample wildlife pathogens, map their evolutionary space for pandemic potential, and publish rank-ordered lists of the riskiest pathogens. Such work is of dubious biodefence value, presents a direct risk of accidental release in the field and lab, and the resulting information is a clear biosecurity infohazard.
We would be excited to fund projects to collect, monitor, and report on the activities of these projects. ZRP projects have multiple components- field sampling, computational modelling, and lab characterization - each of which carry distinct risks and leaves an information trail. Monitoring and reporting on open source information associated with ZRP projects could disincentivize the riskiest aspects of this work, target resources for event surveillance and early warning of accidental release, and provide material for advocacy efforts.
There is some overlap with portions of the BWC project, but I think this is best tackled as a separate body of work/by a different team (due to radically different OPSEC, deception, and scrutiny profiles). I've thought about this a fair bit and am happy to discuss offline.
Improving ventilation
Biorisk
Ventilation emerged as a potential intervention to reduce the risk of COVID and other pathogens. Additionally, poor air quality is a health concern in its own right, negatively affecting cognition and cognitive development. Despite this, there still does not seem to be commonly accepted wisdom about what kind of ventilation interventions ought to be pursued in offices, bedrooms, and other locations.
We'd like to see a project that does rigorous research to establish strong ventilation strategies in a variety of contexts and explores their effectiveness on various ventilation issues. Once successful ventilation strategies are developed, assuming it would be cost-effective to do so, this project could then aim to roll out ventilation and campaign/market for ventilation interventions either as a for-profit, non-profit, or hybrid.
Disclaimer: This is just my personal opinion and not the opinion of Rethink Priorities. This project idea was not seen by anyone else at Rethink Priorities prior to posting.
Longtermist Policy Lobbying Group
Biorisk, Recovery from Catastrophe, Epistemic Institutions, Values and Reflective Processes
Many social movements find a lot of opportunity by attempting to influence policy to achieve their goals . While longtermism can and should remain bi-partisan, there may be many opportunities to pull the rope sideways on policy areas of concern.
We'd like to see a project that attempts to carefully understand the lobbying process and explores garnering support for identified tractable policies. We think while such a project could scale to be very large once successful, anyone working on this project should really aim to start small and tred carefully, aiming to avoid issues around the unilateralist curse and ensuring to not make longtermism into an overly partisan issue. It's likely that longtermist lobbying might also be best done as lobbying for other clear areas related to longtermism but as a distinct idea, such as lobbying for climate change mitigation or lobbying for pandemic preparedness.
Disclaimer: This is just my personal opinion and not the opinion of Rethink Priorities. This project idea was not seen by anyone else at Rethink Priorities prior to posting.
Landscape Analysis: Longtermist Policy
Biorisk, Recovery from Catastrophe, Epistemic Institutions, Values and Reflective Processes
Many social movements find a lot of opportunity by attempting to influence policy to achieve their goals - what ought we do for longtermist policy? Longtermism can and should remain bi-partisan but there may be many opportunities to pull the rope sideways on policy areas of concern.
We'd like to see a project that attempts to collect a large number of possible longtermist policies that are tractable, explore strategies for pushing these policies, and also use public opinion polling on representative samples to understand which policies are popular. Based on this information, we could then suggest initiatives to try to push for.
Disclaimer: This is just my personal opinion and not the opinion of Rethink Priorities. This project idea was not seen by anyone else at Rethink Priorities prior to posting.
Never Again: A Blue-Ribbon Panel on COVID Failures
Biorisk, Epistemic Institutions
Since effective altruism came to exist as a movement, COVID was the first big test of a negative event that was clearly within our areas of concern and expertise. Despite many high-profile warnings, the world was clearly not prepared to meet the moment and did not successfully contain COVID and prevent excess deaths to the extent that should've been theoretically possible if these warnings had been properly heeded. What went wrong?
We'd like to see a project that goes into extensive detail about the global COVID response - from governments, non-profits, for-profit companies, various high-profile individuals, and the effective altruism movement - and understands what the possibilities were for policy action given what we knew at the time and where things fell apart. What could've gone better and - more importantly - how might we be better prepared for the next disaster? And rather than try to re-fight the last war, what needs to be done now for us to better handle a future disaster that may not be bio-risk at all?
Disclaimer: This is just my personal opinion and not the opinion of Rethink Priorities. This project idea was not seen by anyone else at Rethink Priorities prior to posting.
Minor note about the name: "Never Again" is a slogan often associated with the Holocaust. I think that people using it for COVID might be taken as appropriation or similar. I might suggest a different name.
https://en.wikipedia.org/wiki/Never_again
Are you thinking of EAs running this themselves? We already have an informal sense of what some top priorities are for action in biosafety/pandemic-preparedness going forwards (ramp up investment in vaccines and sterilizing technology, improve PPE, try to ban Gain of Function research, etc), even if this has never been tied together into a unified and rigorously prioritized framework.
I think the idea of a blue-ribbon panel on Covid failures could have huge impact if it had (in the best-case) official buy-in from government agencies like the CDC, or (failing that) at least something like "support from a couple prestigious universities" or "participation from a pair of senators that care about the issue" or "we don't get the USA or UK but we do get a small European country like Portugal to do a Blue Ribbon Covid Panel". In short, I think this idea might ideally look more like "lobby for the creation of an official Blue Ribbon Panel, and also try to contribute to it and influence it with EA research" rather than just running it entirely as an internal EA research project. But maybe I am wrong and a really good, comprehensive EA report could change a lot of minds.
Pandemic preparedness in LMIC countries
Biorisk
COVID has shown us that biorisk challenges fall on all countries, regardless of how prepared and well-resourced the countries are. While there certainly are many problems with pandemic preparedness high-income countries that need to be addressed, LMIC countries face even more issues in helping detect, identify, contain, mitigate, and/or prevent currently known and novel pathogens. Additionally, even after high income countries successfully contain a pathogen it may continue to spread within LMIC countries opening up risk of further more virulent mutations.
We'd like to see a project that works with LMIC governments to understand their current pandemic prevention plans and understand their local context. This project would especially focused on novel pathogens that are more severe than currently known pathogens -- and help provide the resources and knowledge needed to upgrade their plans to match the best practices of current bio-risk experts. Such a project would likely benefit from a team that contains expertise working with LMIC countries. An emergency fund and expert advice can also be provisioned to be ready to go when pathogens are... (read more)
Securing offices and schools against SARS-3
Biorisk
The COVID-19 pandemic has demonstrated failures of our scientific, political, and epistemic institutions, but also of our physical structures. We believe that accurate and high-quality designs of offices and schools to be secure against pathogen spread of airborne viruses can be a) directly useful, b) potentially generalize well to future pandemics, and c) provide the necessary training ground for building more robust and ambitious projects in the future, including large-scale civilizational refuges.
We picked offices and schools to limit the threat model and surface area, but we're in theory excited about designs that can contain pathogen spread in any well-trafficked built environment.
Investment strategies for longtermist funders
Research That Can Help Us Improve, Epistemic Institutions, Economic growth
Because of their non-standard goals, longtermist funders should arguably follow investment strategies that differ from standard best practices in investing. Longtermists place unusual value on certain scenarios and may have different views of how the future is likely to play out.
We'd be excited to see projects that make a contribution towards producing a pipeline of actionable recommendations in this regard. We think this is mostly a matter of combining a knowledge of finance with detailed views of the future for our areas of interest (i.e. forecasts for different scenarios with a focus on how giving opportunities may change and the associated financial winners/losers). There is a huge amount of room for research on these topics. Useful contributions could be made by research that develops these views of the future in a financially-relevant way, practical analysis of existing or potential financial instruments, and work to improve coordination on these topics.
Some of the ways the strategies of altruistic funders may differ include:
- Mission-correlated investing
... (read more)I have had a similar idea, which I didn't submit, relating to trying to create investor access to tax-deductible longtermist/patient philanthropy funds across all major EA hubs. Ideally these would be scaled up/modelled on the existing EA long term future fund (which I recall reading about but can't find now, sorry)
Edit - found it and some ideas - see this and top level post.
Reflection Retreats
Effective Altruism
There are certain points in our life when the decisions we make can greatly affect the trajectory of our lives. This could include deciding what degree to study, graduating or making a major career change. These retreats would bring together a bunch of EAs together (possibly some non-EAs too) to reflect on these decision and start making applications/plans, ect.
Longtermism Policy Lab
Epistemic Institutions, Values and Reflective Processes, Great Power Relations, Space Governance, Research That Can Help Us Improve
Despite the growing recognition of the importance of long-term perspectives, governance remains oriented around short-term incentives. More coordination and collaboration between researchers and policymakers, practicioners and industry professionals is needed to translate research into policy. The Longtermism Policy Lab will bridge this gap, working with societal partners and governments at all levels (local to global) to undertake policy experiments. The Lab will also contain a research component, establishing and pursuing an ambitious interdisciplinary Longtermism research agenda, including an emphasis on research that doesn't fit well within either academia or traditional research institutes. We want to see this organisation serve as a direct link between longtermism as a governance approach and its implementation within all levels of governance across the globe.
Centralising Information on EA/AI Safety
Effective Altruism, AI Safety
There are many list of opportunities available in EA/AI Safety and many lists of what organisations exist. Unfortunately these lists tend to get outdated. It would be extremely valuable to have a single list that is up to date and filterable according to various criteria. This would require someone being paid to maintain these part-time.
Another opportunity for centralisation would be to create an EA link shortener with pretty URLs. So for example, you'd be able to type in ea.guide/careers to see information on careers or ea.guide/forum to jump to the forum.
Notes: I own the URL ea.guide so I'd be able to donate it.
(Per Nick's post, reposting)
Large-scale randomized controlled trials
Values and Reflective Processes; Epistemic institutions; Economic Growth
RCTs are the gold standard in social science research but are frequently too expensive for most researchers to run, particularly in the United States. We are interested in large-scale funding of RCTs that are usually impossible due to a lack of funding.
(Per Nick's post, reposting)
Private-sector ARPA models
All
Many of the technological innovations of the last fifty years have their genesis in experiments run by DARPA. ARPA models are characterized by individual decision-makers taking on risky bets within defined themes, setting ambitious goals, and mobilizing top researchers and entrepreneurs to meet them. We are interested in funding work to study these funding models and to create similar models in our areas of interest.
(Per Nick's post, reposting)
Practitioner research
All
Universities are primarily filled with professors trained in similar ways. Although universities sometimes have “professors of the practice,” these positions are often reserved for folks nearing retirement. We are interested in funding ways for practitioners to spend time conducting and publishing “research” informed by their lived real-world experiences.
(Per Nick's note, reposting)
Cross-university research
Values and Reflective Processes, Research That Will Help Us Improve, Epistemic Institutions, Empowering Exceptional People
Since 1978, more than 30 scientists supported by the Howard Hughes Medical Institute have won the Nobel prize in medicine. We are interested in funding other cross-institutional collections of researchers and financial support beyond the biosciences, focusing on economic growth, public policy, and general social sciences.
(Per Nick's note, reposting)
Political fellowships
Values and Reflective Processes, Empowering Exceptional People
We’re like to fund ways to pull people who don’t run for political office to run for political office. It's like a MacArthur. You get a call one day. You've been selected. You'd make a great public servant, even if you don't know it. You'd get some training, like DCCC and NRCC, and when you run, you get two million spent by the super-PAC run by the best. They've done the analysis. They'll provide funding. They've lined up endorsers. You've never thought about politics, but they've got your back. Say what you want to say, make a difference in the world: run the campaign you don't mind losing. And if you win, make it real.
(Per Nick's note, reposting)
Development of cross-disciplinary talent
Economic Growth, Values and Reflective Processes, Empowering Exceptional People,
The NIH successfully funded the creation of interdisciplinary graduate programs in, for example, computational biology and Ph.D./MD programs. Increasingly, the returns to studying in one discipline, artificially constructed, cannot solve our most pressing problems. We are interested in funding the development of fluent individuals in two or more fields — particularly people with expertise in technology and soci... (read more)
(Per Nick's note, reposting)
Market shaping and advanced market commitments
Epistemic institutions; Economic Growth
Market shaping is when an idea can only be jump-started by committed demand or other forces. Operation Warp Speed is the most recent example of market-shaping through advanced market commitments, but it has been used several times for other vaccine development. We are interested in funding work to understand when market-shaping makes sense, ideas for creating and funding market-shaping methods, and specific market-shaping or advanced market commitments in our areas of interest.
(I drafted this then realized that it is largely the same as Zac's comment above - so I've strong upvoted that comment and I'm posting here in case my take on it is useful.)
Crowding in other funding
We're excited to see ideas for structuring projects in our areas of interest that leverage our funds by aligning with the tastes of other funders and investors. While we are excited about spending billions of dollars on the best projects we can find, we're also excited to include other funders and investors in the journey of helping these projects scale in the best way possible. We would like to maximize the chance that other sources of funding come in. Some projects are inherently widely attractive and some others are only ever likely to attract (or want) longtermist funding. But, we expect that there are many projects where one or more general mechanisms can be applied to crowd in other funding. This may include:
(Per Nick's note, reposting)
Replication funding and publication
Epistemic Institutions
The replication crisis is a foundational problem in (social) science. We are interested in funding publications, registries, and other funds focused on ensuring that trials and experiments are replicable by other scientists.
(Per Nick's note, reposting)
Longitudinal studies
Epistemic Institutions; Economic Growth
We are interested in funding long-term, large-scale data collection efforts. One of the most valuable research tools in social science is the collection of cross-sectional data over time, whether on educational outcomes, political attitudes and affiliations, health access, and outcomes. We are interested in funding research projects that intend to collect data over twenty years. The projects require significant funding to ensure follow-up data collection.
Civic sector software
Economic Growth, Values and Reflective Processes
Software and software vendors are among the biggest barriers to instituting new public policies or processes. The last twenty years have seen staggering advances in technology, user interfaces, and user-centric design, but governments have been left behind, saddled with outdated, bespoke, and inefficient software solutions. Worse, change of any kind can be impractical with existing technology systems or when choosing from existing vendors. This fact prevents public servants from implementing new evidence-based practices, becoming more data-driven, or experimenting with new service models.
Recent improvements in civic technology are often at the fringes of government activity, while investments in best practices or “what works” are often impossible for any government to implement because of technology. So while over the last five years, there has been an explosion of investments and activity around “civic innovation,” the results are often mediocre. On the one hand, governments end up with little more than tech toys or apps that have no relationship to the outcomes that matter (e.g. poverty alleviation, service deli... (read more)
(For context, I was the Chief Data Officer of the California State Government and CTO of Newark, NJ when Cory Booker was Mayor).
I actually think the way to do this is to partner with one city and build everything they need to run the city. The problem is that people can't use piecemeal systems very well. It would just take a huge initial set of capital -- like exactly the type of capital that could be provided here.
Social sector infrastructure
Values and Reflective Processes, Empowering Exceptional People
If an entrepreneur starts or runs a for-profit company, there is a range of software and other infrastructure to help you run your business: explainer guides, AWS, Salesforce.com, etc. Similar infrastructure for not-for-profits and other NGOs exist, particularly cross-border. We are interested in finding a new generation of infrastructure that supports the creation and maintenance of the social sector. This could look like a next-generation low-cost fiscal sponsor or an accounting system focused on NFP accounting and filing 990s, anything that makes it easier to start and run institutions.
Studying Economics Growth Deterrents and Cost Disease
Economic growth
Economic growth has forces working against it. Cost disease is the most well-known and pernicious of these in developed economies. We are interested in funding work on understanding, preventing, and reversing cost disease and other mechanisms that are slowing economic growth.
(Inspired by Patrick Collison)
Accelerating Accelerators
Economic Growth
Y Combinator has had one of the largest impacts on GDP of any institution in history. We are interested in funding efforts to replicate that success across different geographies, sectors (e.g. healthcare, financial services), or corporate form (e.g. not-for-profit vs. for-profit).
Space Policy Lab
Space Governance, Epistemic Institutions
Human activity in space is intensifying with the growing challenge of space debris, the deployment of satellite mega-constellations, and the prospects of asteroid mining and long-term colonisation raising unique challenges to a vital yet neglected domain. Current space governance - the laws, rules, norms and institutions that structure interactions in space - is falls far short of meeting these challenges. A Space Policy Lab would research governance frameworks analyse policy issues shape expert discourse, and engage in advocacy for effective regulatory frameworks. We would like to see a Lab bringing together applied researchers, academia and societal stakeholders within a dynamic collaborative & transdisciplinary environment through undertaking policy experiments to identify levers for improving space governance.
AI alignment prize suggestion: Improve our ability to evaluate (and provide training signal for) fuzzy tasks
Artificial Intelligence
There are many tasks that we want AI systems to do, for which performance cannot be evaluated automatically (and thus training signal provision is hard). If we don't make progress on our ability to train systems for such tasks, we might end up in a world full of systems that optimise for that which is easy to measure, rather than what we actually want. One example of such a task is the evaluation of free-form text; there is currently no automated method to evaluate free-form text (with respect to criteria such as usefulness or correctness) that matches human evaluation. The Future Fund could offer prizes for work that takes a task for which the gold-standard of evaluation is humans, and demonstrates an automated evaluation method that matches human evaluation very closely (or work that demonstrates an automated evaluation method to be superior to human evaluation).
Note: This is crucially not the same as "training models to perform well on the task in question". There are a number of technical reasons why what I suggest is easier. Intuitively, evaluating performance is often considerably easier than generating good performance. For example, I can watch a movie and say if it's good, but I can't make a good movie.
International mass movement lobbying against x-risks
Biorisk and Recovery from Catastrophe, Great Power Relations, Values and Reflective Processes
In recent years, there has been a dramatic growth in grassroots movements concerned about climate change, such as Fridays for Future and Extinction Rebellion. Some evidence implies that these movements might be instrumental in shifting public opinion around a topic, changing dominant narratives, influencing voting behaviour and affecting policymaker beliefs. Yet, there are many more pressing existential risk... (read more)
AI alignment prize suggestion: Demonstrate a true sandwiching project
Artificial Intelligence
Sandwiching projects are a concrete way for how to make progress on aligning narrowly superhuman models. They “sandwich” the model in between one set of humans which is less capable than it and another set of humans which is more capable than it at the fuzzy task in question, and b) figure out how to help the less-capable set of humans reproduce the judgments of the more-capable set of humans. For example, first fine-tune a coding model to write short functions solving simple puzzles using demonstrations and feedback collected from expert software engineers. Then try to match this performance using some process that can be implemented by people who don’t know how to code and/or couldn’t solve the puzzles themselves.
Importantly, there are many ways to attack a sandwiching project that are slightly cheating. The most challenging version of a sandwiching project would need to make sure that no information whatsoever from the more-capable set of humans is used in the training process. The Future Fund could offer prizes for demonstrations of sandwiching projects on various levels of impressiveness and generality of the employed method.
Airdrop for EA Forum karma holders
Empowering Exceptional People, Effective Altruism
Take a snapshot from some time in the past (e.g. date of OP), and award $100 for each karma point to all EA Forum holders. This could be extended and scaled as appropriate to the AI Alignment Forum and perhaps r/EffectiveAltruism and other places as seen fit. As a one-off, this can't be gamed. It might encourage more participation going forward, but it should be made clear that there should be no expectation of a repeat. Ideally, the money would be no strings attached. It wo... (read more)
Slightly disappointed that this has ended up on negative karma. I think it's at least triggered some somewhat fruitful discussion. I do think a broad-based retroactive funding of public good in the EA community would be good; especially in terms of it's knock-on effects for the next generation of projects. Mediation of this via crypto and impact certificates seems promising, even if a direct airdrop based on an imprecise metric such as EA Forum karma isn't the way to go.
It wouldn't be a change. It's a one-off reward for past activity (a retro-active funding of public good as it were :))
There are some serious incentives issues here where the EAF users with the most karma (and thus most incentive to gain from this proposal) are also the ones with most strong upvote power. :O
I have 6611 karma and if y'all gave me $600k no strings attached, I'm not gonna lie I would buy a really nice house.
And now an extra $1.5k worth of house on top of that!
AI alignment prize suggestion: Introduce AI Safety concepts into the ML community
Artificial Intelligence
Recently, there have been several papers published at top ML conferences that introduced concepts from the AI safety community into the broader ML community. Such papers often define a problem, explain why it matters, sometimes formalise it, often include extensive experiments to showcase the problem, sometimes include some initial suggestions for remedies. Such papers are useful in several ways: they popularise AI alignment concepts, pave the way for further research, and demonstrate that researchers can do alignment research while also publishing in top venues. A great example would be Optimal Policies Tend To Seek Power, published in NeurIPS. Future Fund could advertise prizes for any paper that gets published in a top ML/NLP/Computer Vision conference (from ML, that would be NeurIPS, ICML, and ICLR) and introduces a key concept of AI alignment.
Facilitate U.S. voters' relocation to swing states
Values and Reflective Processes
A key difficulty of implementing alternative voting systems which can more effectively aggregate voters' preferences/information (and of implementing beneficial policies or constitutional amendments in general) is political gridlock. The political party that stands to lose power if a voting-system reform passes will vigorously attempt to obstruct it. The resolution of political gridlock could not only enable large-scale policy solutions to previously intractable societal... (read more)
Refinement of project idea #22, Prediction Markets
Add: "In particular, we'd like to see prediction platforms that do all of the following three: use real money, are very easy to use, allow very easy creation of markets.
Highly effective enhancement of productivity, health, and wellbeing for people in high-impact roles
Effective Altruism
When it comes to enhancement of productivity, health, and wellbeing, the EA community does not sufficiently utilise division of labour. Currently, community members need to obtain the relevant knowledge themselves and do related research, e.g. on health issues, themselves. We would like to see dedicated experts on these issues that offer optimal productivity, health, and wellbeing, as a service. As a vision, a person working in a high-impact role could book calls with highly trained nutrition specialists, exercise specialists, sleep specialists, personal coaches, mental trainers, GPs with sufficient time, and so on, increasing their work output by 50% while costing little time. This could involve innovative methods such as ML-enabled optimal experiment design to figure out which interventions work for each individual.
Note: Inspired by conversations with various people. I won't name them here because I don't want to ask for permission first, but will share the prize money with them if I win something.
Advocacy organization for unduly unpopular technologies
Public opinion on key technologies.
Some technologies have enormous benefits, but they are not deployed very much because they are unpopular. Nuclear energy could be a powerful tool for enhancing access to clean energy and combating climate change, but it faces public opposition in Western countries. Similarly, GMOs could help solve the puzzle of feeding the global population with fewer resources, but public opinion is largely against them. Cellular agriculture may soon face similar challenges. Public opinion on these technologies must urgently be shifted. We’d like to see NGOs that create the necessary support via institutions and the media, without falling into the trap of partisan warfare with traditional environmentalists.
Reducing gain-of-function research on potentially pandemic pathogens
Biorisk
Lab outbreaks and other lab accidents with infectious pathogens happen regularly. When such accidents happen in labs that work on gain-of-function research (on potentially pandemic pathogens), the outcome could be catastrophic. At the same time, the usefulness of gain-of-function research seems limited; for example, none of the major technological innovations that helped us fight COVID-19 (vaccines, testing, better treatment, infectious disease modelling) was enabled by gain-of-function research. We'd like to see projects that reduce the amount of gain-of-function research done in the world, for example by targeting coordination between journals or funding bodies, or developing safer alternatives to gain-of-function research.
Additional notes:
- There are many stakeholders In the research system (funders, journals, scientists, hosting institutions, hosting countries). I think the concentration of power is strongest in journals: there are only a few really high profile life-science journals(*). Currently, they do publish gain-of-function research. Getting high-profile journals to coordinate against publishi
... (read more)Cognitive enhancement research and development (nootropics, devices, ...)
Values and Reflective Processes, Economic Growth
Improving people's ability to think has many positive effects on innovation, reflection, and potentially individual happiness. We'd like to see more rigorous research on nootropics, devices that improve cognitive performance, and similar fields. This could target any aspect of thinking ability---such as long/short term memory, abstract reasoning, creativity---and any stage of the research and development pipeline---from wet lab research or engineering over testing in humans to product development.
Additional notes on cognitive enhancement research:
- Importance:
- Sign of impact: You already seem to think that AI-based cognitive aids would be good from a longtermist perspective, so you will probably think that non-AI-based cognitive enhancement is also at least positive. (I personally think that's somewhat likely but not obvious and would love to see more analysis on it).
- Size of impact: AI-based cognitive enhancement is probably more promising right now. But non-AI-based cognitive enhancement is still pretty promising, there is some precedent (e.g. massive benefit
... (read more)Pay prestigious universities to host free EA-related courses to very large numbers of government officials from around the world
Empowering Exceptional People
The direct benefit of the courses would be to give government officials better tools for thinking and talking with each other.
The indirect benefit could be to allow large numbers of pre-disposed officials to be seen by <some organisation> who could use the opportunity to identify those with particular potential and offer them extra support or opportunities so they can make an even bigger impact.
The need for it to be free is to overcome the blocker of otherwise needing to write a business case for attendance which may then require some sort of tortuous approval process.
The need for it to be hosted at a prestigious university is to overcome the blocker of justifying to bosses or colleagues why the course is worthwhile by allowing piggybacking off the University's brand.
EA Programming Bootcamp
Effective Altruism
Providing a programming bootcamp to members of the Effective Altruism community could be a way of assisting struggling community members whilst avoiding the issues inherent with directly providing cash assistence. It could also allow communities members to accelerate their career progression.
Notes: See the comments here for some of the issues with giving cash.
I suspect that the impact of this would be larger than it first appears as a) talented people generally want to be part of a community where people are successful b) if community members are struggling then that takes up the time of other community members who try to help them.
In case you drew inspiration from some of our suggestions in the megaprojects article, we would like to retroactively apply.
Sponsoring Debates on Future Fund Issues
Effective Altruism
The Fund Future could run debates on these issues with high-level debaters (ie. World Champions or finalists) receiving significant compensation to take part. One format which would be particularly exciting would involve prominent academics giving the opening speeches for both sides and debaters taking the debate from there (for example, imagine Bostrom and Peter Singer debating how much we should focus on x-risks from AI vs. the present day). The debates would be recorded and prominently advertised... (read more)
An EA Vegan Restaurant Chain:
Effective Altruism
Setting up a vegan restaurant chain associated with Effective Altruism could provide a cost-neutral or even profitable way of providing home bases for EA Societies in major cities. It would also provide opportunites to grow the community by prominently advertising any EA events running at the place.
Downside: This might be seen as cultish. It wouldn't surprise me if there was no-one who was value-aligned who had the relevant skills. That said, we might be able to sign a franchise agreement with an existing restaurant.
(Probably not a good idea, but when brainstorming it is better to share more rather than less)
High quality, EA Audio Library (HEAAL)
all/meta, though I think the main value add is in AI
(Nonlinear has made a great rough/low quality version of this, so at least some credit/prize should go to them.)
Audio has several advantages over text when it comes to consuming long-form content, with one significant example being that people can consume it while doing some other task (commuting, chores, exercising) meaning the time cost of consumption is almost 0. If we think that broad, sustained engagement with key ideas is important, making the cost of engagement much lower is a clear win. Quoting Holden's recent post:
What does high quality mean here, and what content might get covered?
-
... (read more)High quality means read by humans (I'm imagining paying maths/compsci students who'll be able to handle mathematical n
High-quality human performance is much more engaging than autogenerated audio, fwiw.
Starting EA community offices
Effective altruism
(Note: I believe someone actually is looking into starting such an office in Boston. I think (?) that might already be funded, but many other cities could plausibly benefit from offices of their own.)
Here is a more ambitious version:
EA Coworking Spaces at Scale
Effective Altruism
Here is an even more ambitious one:
Found an EA charter city
Effective Altruism
Foundational research on the value of the long-term future
Research That Can Help Us Improve
If we successfully avoid existential catastrophe in the next century, what are the best pathways to reaching existential security, and how likely is it? How optimistic should we be about the trajectory of the long-term future? What are the worst-case scenarios, and how do we avoid them? How can we make sure the future is robustly positive and build a world where as many people are flourishing?
To elaborate on what I have in mind with this proposal, it seems important to conduct research beyond reducing existential risk over the next century – we should make sure that the future we have afterwards is good as well. I'd be interested in research following up on subjects like those of the posts:
- "Disappointing Futures" Might Be As Important As Existential Risk - Michael Dickens
- Why I prioritize moral circle expansion over artificial intelligence alignment - Jacy Reese
- The expected value of extinction risk reduction is positive - Jan Brauner and Friederike Grosse-Holz and A longtermist critique of “The expected value of extinction risk reduction is positive”
- Should We Prioritize Long-Term Existential
... (read more)AI safety university groups
Artificial Intelligence
Leading computer science universities appear to be a promising place to increase interest in working to address existential risk from AI, especially among the undergraduate and graduate student body. In Q1 2022, EA student groups at Oxford, MIT, Georgia Tech, and other universities have had strong success with AI safety community-building through activities such as facilitating the semester-long AGI Safety Fundamentals program locally, hosting high-profile AI safety researchers for virtual guest speaker events, and running a research paper reading group. We'd also like to see student groups which engage students with opportunities to develop relevant skills and which connect them with mentors to work on AI safety projects, with the goal of empowering students to work full-time on AI safety. We'd be happy to fund students to run AI safety community-building activities alongside their studies or to take a gap semester, or to sponsor other people to support an EA group at leading university in building up the AI safety community.
Some additional comments on why I think AI safety clubs are promising:
- For those unfamiliar, the AGI Sa
... (read more)Incubator Incubator
Effective Altruism
Effective Altruism needs more incubators. Why not have an incubator to incubate them?
Risks: We end up with too many incubators.
(This is my least serious proposal)
EA Crisis Fund:
Effective Altruism/X-risk
The EA Crisis Fund would respond to crisises around the world such as the current crisis of Ukranian refugees. This would help develop the capabilities of EA to respond to novel situations on short timelines, provide great publicity and build connections and credibility with government. This would increase the chance that EA would have a seat at the table in important discussions.
Potential Downside: It may be hard to respond to these crisises in a way that builds credibility without burning a lot of money.
Better Reporting on Other Countries' Perspectives
Epistemic Institutions
(Refinement of better news)
It's very hard for a regular person to understand what the Russian or Chinese or Turkish perspectives on events are from reading Western media. It would be valuable to have a high-quality mainstream news media source that takes special effort to make sure that this is explored, including by having on-staff anthropologists. This would increase understanding between countries and reduce the chance of Great Power Conflict.
Moderators for EA/Longtermist FB/Groups or Discords
Effective Altruism
(Refinement of EA-relevant Substacks, Youtube, social media, etc. )
Given the huge amount of funding available to EA, we probably don't want to skimp on moderators for major Facebook or Slack or Discord groups even though these have traditionally been run by volunteers. It'd be worthwhile at least experimenting to see if paid part-time moderators would be able to add extra value by writing up summaries/content for the groups, running online calls, setting up networking spreadsheets and spending more time thinking through strategy.
Risks: We might end up paying money for work that we would have gotten for free. Attempts to set up networking spreadsheets or run calls might have minimal participation and hence minimal impact.
Agent Foundations and Philosophy Engagement Fund:
AI Safety
Agent Foundations research may potentially be important for AI Safety, but currently it has received very little engagement from the philosophical community. This fund would offer funding and or scholarships for people who want to engage with these ideas in an academic philosophical context. This project aims to improve clarity about whether this research actually worthwhile and, if so, to help make progress on these problems.
Nuclear/Great Power Conflict Movement Building
Effective Altruism
Given the current situation in Ukraine, movement-building related to nuclear x-risk or great power conflict would likely be much more tractable than it was up until recently. We don't know how long this period will last for and the memory of the public can be short, so we intend to advantage of this opportunity. This outreach should focus on people with an interest in policy or potential student group organisers as these people are most likely to have an influence here.
Salary Negotiation Service:
Effective Altruism
This service could negotiate salaries on behalf of EAs or others who would then commit a proportion of the extra to charity. This would increase the amount of money going to EA causes, promote Effective Altruism and draw people deeper into the community. Given the number of EAs who are working at high-paying tech companies this would likely be profitable.
(I remembered hearing this idea from someone else a few years back, but I can't remember who it was, unfortunately, so I can't give them credit unless they name themselves)
Risks: Might be expensive to find someone with the skills to do this and this might outweigh the money raised.
Leadership development:
Effective Altruism
People who are ambitious are often keen on developing their leadership skills. A program that supported ambitious and altruistic people could both increase people's individual impact and provide a form of EA outreach through sharing EA frames and perspectives. This program would also be useful for developing the leadership skills of people within EA.
Teaching secondary school students about the most pressing issues for humanity's long-term future
Values and Reflective Processes, Effective Altruism
EA Macrostrategy:
Effective Altruism
Many people write about the general strategy that EA should take, but almost no-one outside of CEA has this as their main focus. Macrostrategy involves understanding all of the different organisations and projects in EA, how they work together, what the gaps are and the ways in which EA could fail to achieve its goals. Some resources should be spent here as an exploratory grant to see what this turns up.
Risk modelling and preparedness for climate-induced risks
Research That Will Help Us Improve
Ambitious Altruistic Software Engineering Efforts
Values and Reflective Processes, Effective Altruism
There is a long list of altruistic software projects waiting to be built, with various worthy goals such as improving forecasting, improving groups' ability to intelligently coordinate, or improving the quality of research and social-media conversations.
(This is a refinement of Yonatan Cale's proposal)
Limited Scope Impact Purchase:
Various cause areas incl. AI Safety and Effective Altruism
The biggest challenge with impact purchases is that the market for selling is usually much larger than the market for buying. This project would limit the scope of the purchase to particular people to ensure a) that impact sellers were aware of the impact purchase's existence when they decided to pursue that project* and b) to address this market imbalance and therefore increase people's odds that they are paid and hence ... (read more)
Masters Degrees for Movement Building:
AI Safety
Many people want to contribute to AI safety and they may have strong technical abilities, but not yet be in a position to be able to contribute to research. Some of these people might also have experience in movement building. It might be worthwhile to pick Masters of AI programs that are highly ranked and pay for a pair of AI Safety movement builders to study there so that they can promote the idea among the school, whilst upskilling at the same time. (This could work for other cause areas like biosecurity)
Risks: Masters degrees are very expensive.
Advocacy for [metascience, land-use reform, clean energy technologies, or other individual planks of the progress studies platform]
Economic growth, Epistemic institutions
You already list high-skill immigration advocacy, pandemic-prevention breakthroughs, and a variety of institutional-innovation topics; why not the rest of the "abundance agenda"? (I already listed general/high-level philosophical research, but here I am suggesting specific sub-areas.)
Land use, construction costs, "yimby", etc. -- Has it gotten more difficult for civilization to build... (read more)
Mentors/tutors for AI safety:
AI Safety
Many people want to contribute to AI safety, but they may not be able to get up to the level where they would be able to conduct useful research. On the other hand, given time, many of these people could probably become knowledgeable enough about on a particular agenda in order to mentor potential researchers pursuing this agenda. These mentors could help people understand the reasons for and against pursuing a particular agenda, help people navigate the content that has been written on that topic, address common misconceptions and help people who are confused about a particular point.
Resilient ways to archive valuable technical / cultural / ecological information
Biorisk and recovery from catastrophe
In ancient Sumeria, clay tablets recording ordinary market transactions were considered disposable. But today's much larger and wealthier civilization considers them priceless for the historical insight they offer. By the same logic, if human civilization millennia from now becomes a flourishing utopia, they'll probably wish that modern-day civilization had done a better job at resiliently preserving valuable information. For example, over the past 120 years, around 1 vertebrate species has gone extinct each year, meaning we permanently lose the unique genetic info that arose in that species through millions of years of evolution.
There are many existing projects in this space -- like the internet archive, museums storing cultural artifacts, and efforts to protect endangered species. But almost none of these projects are designed robustly enough to last many centuries with the long-term future in mind. Museums can burn down, modern digital storage technologies like CDs and flash memory aren't designed to last for centuries, and many... (read more)
Research on the long-run determinants of civilizational progress
Economic growth
What factors were the root cause of the industrial revolution? Why did industrialization happen in the time and place and ways that it did? How have the key factors supporting economic growth changed over the last two centuries? Why do some developing countries manage to "catch up" to the first world, while others lag behind or get stuck in a "middle-income trap"? Is the pace of entrepreneurship or scientific innovation slowing down -- and if so, what can we do about it? Is increasing amounts of "vetocracy" an inevitable disease that afflicts all stable and prosperous societies (as Holden Karnofsky argues here), or can we hope to change our culture or institutions to restore dynamism? At FTX, we'd be interested to fund research into these "progress studies" questions. We're also interested in funding advocacy groups promoting potential policy reforms derived from the ideas of the progress studies movement.
A search engine for micro-level data
Macro-level data is easy to find these days. If you want to know the historical GDP of China or carbon emissions of the U.S., you can find the information on many non-profit and for-profit sites via Google.
But suppose you want to quickly look up "people's satisfaction with their daily lives" and "the amount they spend on food," you'd have to read dozens of papers, locate the names of the datasets used, find the places where such survey data is hosted (if it's available at all), create an account on the hosting site, down... (read more)
Physical AI Safety
Drawing from work done in the former Soviet Union to improve safety in their bioweapons and nuclear facilities (e.g. free consultations and install of engineering safety measures, at-cost upgrades of infrastructure such as ventilation and storage facilities, etc), developing a standard set of physical/infrastructure technologies to help monitor AI Development labs/hardware and provide physical failsafes in the event of unexpectedly rapid takeoff (e.g., a FOOM scenario). Although unlikely, some standard guidelines modifying current best-practices for data center safety (e.g., restrictions on devices, physical air gaps between critical systems and the broader world, extensive onsite power monitoring and backup generators) could be critical to prevent anxiety over both physical and digital security from encouraging risk-taking behaviors by AI Development programs (Such as rushing builds, hiding locations, inappropriate dual-use or shared facilities which decrease control over data flows). In particular, physical low-tech hardware such as low-voltage switches have already provided demonstrable benefit in safeguarding high-tech, high-risk activity (See the Goldsb... (read more)
Experiments to scale mentorship and upskill people
Empowering Exceptional People, Effective Altruism
For many very important and pressing problems, especially those focused on improving the far future, there are very few experts working full-time on these problems. What's more, these fields are nascent, and there are few well-defined paths for young or early-career people to follow, it can be hard to enter the field. Experts in the field are often ideal mentors - they can vet newcomers, help them navigate the field, provide career advice, collaborate on projects and gain access to new opportunities, but there are currently very few people qualified to be mentors. We'd love to see projects that experiment with ways to improve the mentorship pipeline so that more individuals can work on pressing problems. The kinds of possible solutions possible are very broad - from developing expertise in some subset of mentorship tasks (such as vetting) in a scalable way, increasing the pool of mentors, improving existing mentors' ability to provide advice by training them, experimenting with better mentor-mentee matchmaking, running structured mentorship programs, and more.
1. Longitudinal studies
Epistemic Institutions; Economic Growth
We are interested in funding long-term, large-scale data collection efforts. One of the most valuable research tools in social science is the collection of cross-sectional data over time, whether on educational outcomes, political attitudes and affiliations, health access, and outcomes. We are interested in funding research projects that intend to collect data over twenty years. The projects require significant funding to ensure follow-up data collection.
2. Replication funding and publicat... (read more)
Thanks so much for all of these ideas! Would you be up for submitting these as separate comments so that people can upvote them separately? We're interested in knowing what the forum thinks of the ideas people present.
Refining EA communications and messaging
Values and Reflective Processes, Research That Can Help Us Improve
If we want to motivate a broad spectrum of people about the importance of doing good and ensuring the long-term goes well, it's imperative we find out which messages are "sticky" and which ones are forgotten quickly. Testing various communication frames, particularly for key target audiences like highly talented students, will support EA outreach projects in better tailoring their messaging. Better communications could hugely increase the number of people that consume EA content, relate to the values of the EA movement, and ultimately commit their life to doing good. We'd be excited to see people testing various frames and messaging, across a range of target audiences, using methodologies such as surveys, focus groups, digital media, and more.
Building the grantmaker pipeline
Empowering Exceptional People, Effective Altruism
The amount of funding committed to Effective Altruism has grown dramatically in the past few years, with an estimated $46 billion dollars currently earmarked for EA. With this significant increase in available funding, there is now a greatly increased need for talented and thoughtful grantmakers, who can effectively deploy this money. It's plausible that yearly EA grantmaking could increase by a factor of 5-10x over the coming decade, and this requires finding and training new grantmakers on best practices, as well as developing sound judgement. We'd love to see projects that build the grantmaker pipeline, whether that's grantmaking fellowships, grantmaker mentoring, more frequent donor lotteries, more EA funds-style organisations with rotating fund managers, and more.
NB: This might be a refinement of fellowships, but I think it's particularly important.
A global observatory for institutional improvement opportunities
Research That Can Help Us Improve, Great Power Relations, Epistemic Institutions
Actions taken by powerful institutions—such as central governments, large corporations, influential media outlets, and R&D labs—can dramatically shape people's lives today and cast a shadow long into the future. It can be hard to know what philanthropic strategies would be most likely to drive better would outcomes, however, because each individual institution is itself a complex ecosystem of incentives, external pressures, norms, policies, and bureaucratic structures. An ongoing project to document how important institutions operate in practice and spot relevant windows of opportunity (e.g., legislation under consideration, upcoming leadership transitions, etc.) as they emerge would be very helpful for mapping the strategic landscape across virtually all of our other interest areas.
TL;DR: EA Retroactive Public Good's Funding
In your format:
Deciding which projects to fund is hard, and one of the reasons for that is that it's hard to guess which projects will succeed and which will fail. But wait, startups have solved this problem perfectly: Anybody is allowed to vet a startup and decide to invest (bet) their money on this startup succeeding, and if the startup does succeed, then the early investors get a big financial return.
The EA community could do the same, only it is missing the part where we give big financial returns to projects that turned out good.
This would make the fund's job much easier: They would have to vet which project helped IN RETROSPECT, which is much easier, and they'll leave the hard prediction work to the market.
Context for proposing this
I heard of a promising EA project that is for some reason having trouble raising funds. I'm considering funding it myself, though I am not rich and that would be somewhat broken to do. But I AM rich enough to fund this project and bet on it working well enough to get a Retroactive Public Good grant in the future, if such a thing existed. I also might have some advantage over the EA Fund in vetting this project.
In Vitalik's words:
https://medium.com/ethereum-optimism/retroactive-public-goods-funding-33c9b7d00f0c
Making Public Information More Public
Access to public information is hampered by arcane systems and government roadblocks that prevent people from getting direct access to data. Federal court records are behind a government paywall. Filing and keeping up with a freedom of information act requests requires herculean dedication. Government data are sometimes timed to be released when people are least likely to focus on it. This is but just a few examples of the barriers placed between what should be public information and the actual public. As a result, unad... (read more)
Retrospective grant evaluations
Research That Can Help Us Improve