Most grantmaking systems are built for scarcity.
They assume money is the main constraint, so the core task is deciding which worthy projects to leave unfunded. Under scarcity, many inefficiencies are tolerable. A funder can move slowly, rely on a small set of trusted contacts, or default toward polished applicants, and the damage is limited by the fact that not much money is moving anyway.
But if AI safety and adjacent causes experience a genuine funding windfall over the next few years, that assumption breaks. Between potential OpenAI and Anthropic-related philanthropy, large new donors entering the space, and widespread concern that timelines may be short, we should at least take seriously the possibility that funding could scale much faster than the institutions around it.
If that happens, money will stop being the binding constraint. Judgment will.
This matters because abundance creates its own pathologies. When capital floods a field faster than its institutions can absorb it, the result is not simply faster progress. Often it is status cascades, shallow evaluation, donor congestion, performative urgency, and a lot of money chasing whatever already looks legible. In that world, the problem is not “how do we spend more?” It is “how do we keep thinking while spending more?”
The right response is not a single super-fund, nor a norm that every donor should just decide for themselves. We need a funding stack: multiple layers of institutions, incentives, and norms suited to different kinds of opportunities. The goal is not perfect allocation. It is to preserve discrimination under abundance.
A lot of today’s effective grantmaking depends on things that do not scale well.
It depends on a handful of unusually thoughtful people. It depends on informal trust networks. It depends on funders having time to read deeply, talk to applicants repeatedly, compare notes with peers, and build an intuitive model of who is serious, who is impressive, and who is merely fluent in the local language of ambition and concern.
That can work reasonably well in a small field. It becomes much less reliable when both the number of applicants and the amount of deployable capital grow rapidly.
Several failure modes seem especially likely.
First, status cascades. In fast-growing ecosystems, funders often converge on the same people, orgs, and projects, not because they are definitely best, but because they are already being funded by others. This is individually rational. If many smart people appear excited about something, backing it feels safer than making an independent bet. But collectively this creates herding. A few projects become overcapitalized while weird, low-status, or difficult-to-explain opportunities remain starved.
Second, legibility bias. When there is too much to evaluate, polished proposals and familiar organizational shapes beat stranger but potentially higher-upside efforts. It is easier to fund “another promising research nonprofit with strong advisors” than “an unusual individual with deep taste who wants six months to build a safety evaluation pipeline no one has tried before.” Under time pressure, legibility wins.
Third, donor congestion. In a windfall environment, promising founders and researchers may spend enormous amounts of time talking to funders, coordinating with funders, signaling to funders, and reshaping their work to be interpretable to funders. Money that is meant to accelerate progress can, paradoxically, create a tax on the people doing the most valuable work.
Fourth, panic deployment. Once large pools of capital exist, there is pressure to move them. People worry about missing the moment. They worry that slow deployment is equivalent to negligence. The result is that “we should probably do something” becomes the standard of proof for grants that would not survive calmer scrutiny.
Fifth, institutional amnesia. Many grants are made with limited public reasoning, little structured follow-up, and almost no systematic postmortems. This is understandable in normal times. But if funding scales dramatically, lack of memory becomes a major liability. The field will repeat avoidable mistakes because it never built the habit of writing down why decisions were made and how they turned out.
The first failure mode of a windfall is not corruption. It is loss of discrimination.
What will actually be scarce
If money becomes abundant, what stays scarce?
At least seven things:
1. Skilled evaluators.
People who can distinguish a merely plausible project from one that is genuinely high-potential are rare. People who can do this across multiple subfields are even rarer.
2. Founder and researcher attention.
The best people are bottlenecks. You cannot infinitely multiply their judgment just by offering more grants.
3. Management capacity.
Even good orgs can absorb only so much capital before quality falls. Hiring, onboarding, and coordination are real constraints.
4. Credible benchmarks.
Many important projects are hard to evaluate ex ante. Without better benchmarks, funding scales faster than feedback loops.
5. Institutional memory.
A field that does not remember what it tried, what failed, and what surprised it will spend like an amateur no matter how intelligent its participants are.
6. Taste.
This sounds soft, but it matters. Some projects are underfunded not because no one has seen them, but because few evaluators have the taste to recognize their importance before everyone else does.
7. Trustworthy coordination.
In abundance, there is more value in donors sharing models, division of labor, and lessons learned. But coordination is hard, especially when people want to preserve independence and avoid groupthink.
This suggests a broader point: when funding grows 10x or 100x, the main task is no longer “find more money.” It is “design institutions that convert money into careful bets.”
We need a funding stack, not a single allocator
I do not think the answer is one giant fund run by the best people we can find. That would concentrate too much power, narrow the field’s epistemics, and make failure too correlated.
I also do not think the answer is pure donor atomization, where everyone independently tries to become a mini-program officer. That would waste enormous amounts of attention and reward the applicants best at repeatedly reframing themselves for different audiences.
A better model is a layered funding ecosystem.
1. Exploratory microgrants
At the bottom of the stack, there should be fast, lightweight funding for individuals, pilot projects, early investigations, and unusual bets.
These grants should optimize for speed, variance, and option value. They are not meant to fund mature institutions forever. They are meant to surface talent and ideas that would otherwise never become legible.
This layer matters because many of the highest-upside opportunities initially look unserious. They are too small, too weird, too interdisciplinary, or too under-specified to pass a normal grant process. If abundance leads only to bigger checks for already-legible orgs, the field will systematically miss some of its best opportunities.
2. Specialist regrantors
The middle of the stack should include many small domain-specific regranting nodes: people or teams with real taste in narrow areas, each managing modest pools of capital.
This helps in several ways. It decentralizes judgment. It reduces congestion on major donors. It allows funders to learn by doing. And it creates a more diverse portfolio of epistemic styles.
One of the biggest dangers in a funding surge is false consensus: everyone starts acting as if the same few people obviously know best. Specialist regrantors create healthier disagreement. A field that can sustain multiple evaluative centers is more robust than one that routes all serious decisions through a tiny elite.
3. Milestone-based growth grants
Larger and more durable grants should often be milestone-based.
This is not because funders should become overbearing or force every project into simplistic KPIs. Some of the most important work is difficult to measure in the short term. But milestone-based funding can create a useful middle ground between naive trust and bureaucratic micromanagement.
The basic idea is simple: fund initial growth, then fund further expansion conditional on concrete signs that the project can productively absorb more capital. This could include research output, talent recruitment, ecosystem effects, technical progress, or other indicators appropriate to the domain.
The point is not to punish uncertainty. The point is to avoid the common pattern where a project gets “fully funded” based mostly on promise and social proof, then drifts because nobody built a natural checkpoint into the relationship.
4. Prizes, bounties, and advance market commitments
Some problems are better funded by paying for outputs than by funding organizations.
If the field needs evaluations, benchmarks, incident writeups, educational infrastructure, policy memos, interpretability tools, or specific public goods, prizes and AMCs can sometimes outperform traditional grants. They widen participation, reward actual delivery, and reduce the burden on evaluators to predict everything in advance.
This mechanism will not fit every domain. But it is underused, and it becomes more attractive in abundance because the cost of experimenting with multiple incentive structures falls.
5. Public reasoning and postmortems
Every serious funding ecosystem should produce shared memory.
That means more public grant writeups, more explanations of reasoning, more retrospective reviews, and more honest discussion of what did not work. Not every detail can be public, and not every grant deserves a long memo. But the current norm of sparse institutional memory will not survive abundance well.
Public reasoning does at least three things. It improves accountability. It helps new donors learn faster. And it creates a body of examples that makes future judgment less dependent on personal access to insiders.
If a field is about to experience unprecedented inflows of capital, “write down why you did things” should become a core norm.
The cultural side matters too
Institutional design is only half the story. Funding systems also depend on attitudes.
A healthy funding culture under abundance would include several norms that are easy to praise and hard to preserve.
First, it should be acceptable to say no slowly. There will be pressure to move fast. Sometimes that will be right. But “money exists and timelines are short” is not a sufficient reason to lower standards everywhere at once.
Second, grantees should not have to perform certainty. One bad dynamic in grantmaking is that applicants feel compelled to sound more confident, more linear, and more polished than reality justifies. This gets worse when more money is available, because the incentive to be legible gets stronger. Funders should reward honesty about uncertainty rather than forcing everyone into artificial crispness.
Third, boring infrastructure should be fundable. In a windfall environment, glamorous projects attract attention automatically. The less glamorous things that make a field function well, like operations capacity, evaluation infrastructure, documentation, and ecosystem maintenance, need active protection.
Fourth, the ecosystem should tolerate redundancy. It is fine, even healthy, for multiple groups to explore related ideas. Trying to centrally eliminate all overlap is a good way to suppress experimentation.
The goal is not elegance. It is resilience.
Conclusion
If funding for AI safety and adjacent work grows dramatically, the danger is not merely waste. It is epistemic collapse: a world where too much money moves too quickly through too little judgment.
Scarcity hides weaknesses. Abundance exposes them.
The right question is therefore not “who should control the money?” It is “what kind of funding ecosystem can remain thoughtful when dollars stop being scarce?” My answer is: one with multiple layers, decentralized evaluators, milestone-based scaling, output-based incentives where appropriate, and much stronger norms of public reasoning and institutional memory.
We do not need a perfect allocator before the money arrives. We probably cannot have one.
But we can do better than hoping a small number of smart people will somehow scale their judgment at the speed of capital.
If the coming years bring a true funding windfall, the bottleneck will not be generosity. It will be allocation legibility. The task is not just to spend more. It is to build systems that can still think.
