A part of the system's apology.
Or: the deadly combination of new research ideas and new ways to fund them
When it comes to research, the EA movement engages in two kinds of efforts: (i) Encouraging new research directions, such as AI alignment and AI ethics (ii) Supporting new ways of doing scientific research. I claim that while both these efforts may make sense on their own, their combination is far from optimal. One might assume a combination of two good ideas would be better than either one on its own. However, the combination of two highly innovative ideas is likely to yield unpredictable results, which are also hard to understand in retrospect. Therefore, the situation where the vast majority of the research funding offered by the EA movement is distributed in non-conventional ways on non-conventional topics is likely to be far from effective.
The paragraph above describes the main, and almost only, claim I wish to convey here. If you have a long reading list and already see the point, this is a good time to move on.
Why do most researchers not engage in current EA opportunities?
Don’t academics look for grants anyway? Well, yes, but let’s compare two cases. In each case, an academic comes to their colleague and says:
- Hi, did you read the last mail from the faculty secretary about the call for funding proposals on detecting unexpected behaviors in ML systems? It sounds not too far from what we are doing, do we want to apply?
- Hi, I just read a blog post about a crypto-billionaire trying to support EA causes. They are willing to fund AI safety / Biorisk mitigation / Space Governance (?) / Better values (??) / Better Epistemic Institutions (???). Do we want to apply?
I do not wish to undermine the great people and efforts behind the FTX foundation or any other similar organization. I am just trying to portray how weird this funding opportunity sounds to most academics (and this is assuming they even get to hear about it).
One may still think that any young researcher interested can just apply without the collaboration of (say) a more senior PI. Yet, a young academic pursuing an EA opportunity, instead of, for example, sticking to their post-doc position is facing a huge risk to their academic future. Not only do they engage with an off-mainstream research topic; but they are also not doing so within the standard academic track. While this may look like a minor issue from an outside perspective, this can be quite detrimental to the average young researcher's career, especially if they do not live in the US.
So how is state-of-the-art funding for academic research done? A good question. I guess that sending info through the faculty secretary could be a good start. More importantly, you probably want to consult people occupying senior relevant positions and not some pseudo-random guy on the internet (aka, yours truly).
Why may the funding organization want to be part of the system?
I claim that when it comes to supporting scientific research, sponsoring academic research traditionally is likely to be the most effective choice. Despite the liabilities, academically oriented researchers may still be the most promising target audience for EA opportunities. This can be seen through two lenses: the size of the research community within academia, and its omnipresence throughout research fields, times, and political conditions.
Can a better system for funding and conducting research be implemented? Well… maybe. The university system has survived for centuries as the way most human societies support open-ended research. The academic structure, with all its downsides, has been replicated in a large variety of political and economic conditions: be it the original city-states where it was formed, liberal democracies, dictatorships, communist states, or even Islamic republics. This is not to say attempts at other kinds of systems should not be tried, but that they should be measured with respect to this very strong baseline. Until new ways of funding research outperform conventional ways of driving scientific progress, it might be best to keep things conventional; figuring out a single component at a time.
Why may the researcher want to be part of the system?
People tend to think that scientific researchers are motivated by goals such as the desire to make exciting discoveries, financial incentives, or the desire to make a positive impact on the world. However, in practice, scientific researchers aim to optimize other goals as well such as (i) Social Motivations: being a part of a community (e.g. an at-least-somewhat-prestigious institute or research group) with which they engage on a regular basis (ii) Future perspectives: having the opportunity to keep engaging in high-quality scientific research in the future. Although the last motivation might apply more to young researchers, these are often the majority of researchers in emerging fields, especially in the kind of fields potentially funded by EA. While projects funded by EA suit the first kind of motivations (excitement, short-term financial motivation, impact), they are less promising in terms of social motivations or future perspectives.