Reconsidering the Celebration of Project Cancellations: Have We Updated Too Far?
Epistemic status: Low certainty. These are tentative thoughts, and I’m open to alternative perspectives.
Posting from an alt account.
The Effective Altruism community has made significant strides in recognizing the importance of quitting projects that don’t deliver short-term results, which helps counteract the sunk cost fallacy and promotes the efficient use of resources. This mindset, in many cases, is a positive development. However, I wonder if we’ve over-updated in this direction. Recent posts about project cancellations, like the one regarding the Center for Effective Aid Policy (CEAP), have received considerable attention—CEAP’s closure post garnered 538 karma, for instance. While I don’t have a strong opinion on whether it was prudent to shutter CEAP, I am concerned that its closure, and the community's reaction to it, vibes in a direction where initial setbacks are seen as definitive reasons to quit, even when there might still be significant long-term potential.
From an outside perspective, it seemed that CEAP was building valuable relationships and developing expertise in a complex field—global aid policy—where results may take years to materialize. Yet, the organization was closed, seemingly because it wasn’t achieving short-term success. This raises a broader concern: are we in danger of quitting too early when projects encounter early challenges, rather than giving them the time they need to realize high expected value (EV) outcomes? There’s a tension here between sticking with projects that have a low probability of short-term success but could yield immense value in the long run, and the temptation to cut losses when things aren’t immediately working out.
High-EV projects often have low-impact modal outcomes, especially in the early stages. It’s entirely possible that a project with a 20% chance of success could still be worth pursuing if the potential upside is transformative. However, these projects can look like failures early on, and if we’re too quick to celebrate quitting, we may miss out on rare but important successes. This is particularly relevant in fields like AI safety, global aid policy, or other high-risk, high-reward areas, where expertise and relationships are slow to develop but crucial for long-term impact.
At the same time, it’s essential not to continue investing in clearly failing projects just because they might turn around. The ability to pivot is important, and I don’t want to downplay that. But I wonder if, as a community, we are at risk of overupdating based on short-term signals. Novel and complex projects often need more time to bear fruit, and shutting them down prematurely could mean forfeiting potentially transformative outcomes.
I don’t have an easy answer here, but it might be valuable to explore frameworks that help us better balance the tension between short-term setbacks and long-term EV. How can we better distinguish between projects that genuinely need to be ended and those that just need more time? Are there ways we can improve our evaluations to avoid missing out on projects with high potential because of an overemphasis on early performance metrics?
I’d love to hear thoughts from others working on long-term, high-risk projects—how do you manage this tension between the need to pivot and the potential upside of sticking with a challenging project?
Personally, I do not think we have swung too far on this. Even in the AIM/CE cohorts (from which both of the recent public shutdown posts came), I still think people err too much on the side of keeping projects running when they are not performing.
I think how to think about shutting down projects comes down a lot to counterfactuals. It's not that a project could not work or definitely will fail; it's about how good a marginal bet it is. Every dollar and high-talent hour going into the project could, in theory, be going somewhere else—often somewhere with pretty high impact. When I think about a cohort of 6 charities founded out of AIM, after 2 years, I would expect shutting down the bottom 2 and having the staff/funding join the top 2 would result in far more impact for the world. That would result in a ~33% shutdown rate, which is way above what happens in the charity world and about double what happens in AIM right now.
Thanks for the input - you are certainly in a great position to judge this and I love this framework. I also agree shutting down earlier can be far easier, because if you are 3 years in with increasing funding, while realizing you aren't up to much it could be difficult socially and status wise to pull out at that point.
A slight caveat I would have is that it might depend on where follow-on funding was coming from. In a closed system with limited EA resources, then yes the only important Factor might be how good a marginal bet the org is.
If an organisation with a lower marginal chance of of success managed to get decent funding from NON- EA sources then it might be worth continuing with a worse bet. For example if maternal health initiative got a big foundation grant or if the recently closed policy org for funded through a thinktank or government grant. One issue here is that often orgs need 3-4 years of operation before people will grant them much money (outside of EA ecosystem)
But I would imagine getting big outside funding isn't super common for marginal bet CE orgs, if it has happened at all. I'm also not sure to what extent the source of funding should change our bet (this has been discussed on the forum but would love to see some math done - mine isn't good enough)
Another even less important caveat might be if talented people were so passionate about what they their initial org did, that they might not be able to easily maximize on their talent in another role.
I would be interested to get a vague idea of what CE founders usually end up doing after closing their orgs. Have many joined other higher performing CE orgs? This might be sensitive info though.
From the little I know of CE peeps I would imagine they are mostly doing pretty great stuff.
Joining another CE charity is pretty common, as is working at EA meta-orgs (AIM, GiveWell, etc.). I would guess that around 75% do something most people would regard as very high impact.
Agree regarding external marginal funding, but I would say, at least in AIM's case, this correlates with early-stage success.
Thanks for your thoughtful response, Joey. I had originally approached this issue more from the perspective of founders and leaders being less "soldiery" for their projects, but I see that the funder’s viewpoint, especially regarding the counterfactual uses of money, is quite different and valid.
One key difference is that the counterfactual reallocation of talent from failed AIM charities may not be as impactful as the reallocation of funds by AIM-adjacent funders. As you mentioned, many people who worked at a failed AIM charity are likely to join another CE charity or work for an EA meta org, but these roles are in high demand and often attract top-tier talent regardless. It’s not clear that the movement of talent between these organizations would have as large an impact as reallocating funds to more successful initiatives.
This is where the dynamic between founders and funders diverges. From the leader’s perspective, it might make more sense to continue to pivot, seek out other funding sources, and keep the project alive, particularly if they still believe in the long-term potential. On the other hand, from the funder’s perspective, cutting their losses and focusing on capitalizing on wins may provide a much clearer path to maximizing impact. It seems that the optimal decisions for founders and funders could diverge, depending on their roles in the ecosystem.
I appreciate your insight into how marginal bets play into these decisions and how AIM's cohort-based structure could actually benefit from higher shutdown rates. It seems like there’s a balance between empowering founders to pursue potential breakthroughs while ensuring funders can make optimal reallocation decisions for broader impact.