TL;DR: I describe some key considerations for prioritization when the potential size of a funding opportunity matters (i.e. its 'room for more funding'). I show how this relates to the ITN framework and I propose new prioritization components, including 'Scalability', as clarification of Tractability, and several different components of 'Neglectedness'.

It would be nice if all we needed to know when prioritizing funding opportunities was their marginal good per dollar. In reality, funders always need to make two decisions: what to fund and how much.

Research, evaluation and execution takes time and effort. When we find a great opportunity we should be even more thrilled if we can put many dollars into it rather than just one. Scalability matters.

However, if we start throwing a lot of dollars at an opportunity then other funders are going to react. Coordination is hard and our dollars are likely to crowd out or 'funge' others. The extent to which this is an issue depends, among other things, on the Neglectedness of the opportunity.

These considerations, and others, arise when we think about the interaction of our funding decisions with others (i.e. as part of an 'equilibrium'). They are even more relevant when we have the ability to put so much funding into an opportunity that it changes the opportunity itself (i.e. is 'non-marginal').

In this post I share a framework that we have developed at the Total Portfolio Project (TPP) to account for these considerations. I'll refer to this as the 'framework'. But, note that it's just one tool among many that we think are useful for assessing opportunities.

We developed this framework out of necessity because of our goal to develop tools for prioritization that work across a wide range of funding opportunities. And when size matters. Now, with the increasing wealth and ambitions of EA, it seems like these considerations are more relevant than ever to the community at large.

Comments, questions and feedback are very welcome.

See this post's twin for an explanation of both ITN and TPP's framework using a mathematical model.

Review of ITN

The Importance, Tractability and Neglectedness (ITN) style frameworks are important tools for estimating the 'value of allocating marginal resources to a problem'. 80,000 hours describes the components of their ITN-style framework as follows:

• Importance - If we solved the problem, how good would it be?
• Tractability - If we doubled the resources dedicated to solving this problem, what fraction of the problem would we expect to solve?
• Neglectedness - How many resources are already going towards solving this problem?

I think these are the right questions, when interpreted carefully,  for assessing the value of adding marginal resources to a problem. 

However, there is a good deal of confusion and disagreement about how the components should be assessed in practice (e.g. see most of the posts here). Diversity of thought is a good thing. But it seems like many of these disagreements are unnecessary.

Differences of opinion, confusion and debate applies to all three ITN components. But, in my experience, it applies the most to Neglectedness. As noted here: "Ideally, we would like to not simply select causes that are neglected, but select causes that are neglected for reasons other than their impact." 

I think these issues are a symptom of ITN not addressing the question of why an opportunity exists in the first place. Thinking about impact in 'equilibrium' requires tackling this question. So, it is a valuable exercise just on this count alone. It also naturally leads to multiple considerations that capture different aspects of what we might mean by 'neglectedness'.

Equilibrium effects

TPP's framework can be applied at the problem level and to individual enterprises (nonprofits and for profits). To be neutral, I refer to what is being assessed as the 'opportunity'.

The figure below illustrates the value produced by a funding opportunity.

Our assessment starts when investment into the opportunity is at IO. What we want to know is what the new level of investment IA will be if we fund the opportunity, and, how much value this will create (the yellow area).

If it was as easy as adding a dollar of investment, so that IA=IO+1, then all we'd want to know is the height of the yellow area. This is what ITN is intended to assess: the marginal value per dollar of investment. We could also assess this with a direct cost-effectiveness analysis.

However, for each dollar we commit, only a fraction of this dollar will go towards increasing investment. 

Furthermore, while the total value per dollar exceeds our opportunity cost we're not going to want to leave 'value on the table'. We don't just want to create a tiny sliver of yellow (area). If possible we want to extend the yellow area out to the point where the neglected value meets the opportunity costs.

This means that an equilibrium funding model is going to tell us both how much we should plan to invest and how much value this will generate. But, it won't tell us if we should fund the opportunity. That is up to us to decide after comparing the net values of all available opportunities - after comparing the total increase in value (the yellow area) minus the costs of our funding.

As detailed here, TPP's framework breaks an opportunity's net value into the following components:

• Baseline Effectiveness - how effective are the other funders?
• Scalability - if we doubled the resources dedicated to solving this opportunity, by what factor do we expect this will increase its activity?
• Conviction - how effective at scaling has the opportunity been with its existing access to wealthy funding?
• Neglectedness - the extent to which, compared to other funders, we think the opportunity can produce more net value per dollar. Broken down into the parts:
  • Relative importance - how much more value per dollar do we expect from the opportunity can produce compared to the other funders?
  • Risk - to what extent is existing funding limited by perceived risks and uncertainties that our funding would help alleviate?
  • Excess opportunity costs - to what extent are our opportunity costs higher or lower than other funders?
• Relative Risk Appetite (RRA) - how much more prepared are we, compared to other funders, to take on the risks and uncertainties associated with funding the opportunity?
• Alignment - how much do we expect to value the alternative investments of the other funders that we might crowd out?

Some components are about the opportunity itself, while others are more about the other funders in the market. Each component is discussed in more detail below.

Baseline Effectiveness

In principle we could use the ITN framework to assess Effectiveness. However, as John Halstead notes here, in cases where we can actually do a direct cost-effectiveness assessment, ITN doesn't need to be used.

Equilibrium reasoning suggests an alternative angle. One can ask:

• What are the opportunity cost of the other funders? How effective do the things they fund tend to be?
• How much do we think funding for this opportunity is constrained by risks or uncertainties that we can help alleviate with our funding?
• Do we think additional funding for this work is more or less valuable than the other funders?

The answer to the first question gives a baseline assessment of the opportunity's effectiveness. Other funder's wouldn't fund it if it was less effective than their benchmark alternative or 'bar'. The second and third questions are covered by Neglectedness.

Scalability

Conceptually this is exactly the same as Tractability. I think 'Scalability' is a better word for this consideration. Scalability better captures what the associated question is asking: how much can work on this problem be scaled with additional resources? 

Also, in my experience, 'Tractability' is often believed by users of ITN to mean something more like Effectiveness. This post makes a similar point and proposes applying Tractability to a new component that is more like Effectiveness.

Having Scalability as a key consideration seems aligned with Joey Savoie and Charity Entrepreneurship's focus on the 'limiting factor'.

To illustrate the difference between Scalability, Effectiveness, and ITN's Importance, consider that earlier stage opportunities are likely to have low scalability but high expected effectiveness and high expected importance (or scale). Because promising early-stage opportunities will initially involve non-scalable work that sets the foundation for future scale. 

The expected future scale and Effectiveness go together because each bit of work now is enabling that future. But Scalability is about how much additional funding now  will enhance that future. If work today depends mostly on a few key people, then current funding is unlikely to be the limiting factor.

Conviction

Conviction can be thought of as the ratio of existing investment to the existing funders wealth. It's about asking: how much has this opportunity been able to convert its access to funding into useful investment? 

This is 'Conviction' in the sense that it is the opposite of 'funging'. 

Between two opportunities with the same level of existing investment, the one that has achieved this from a smaller, less wealthy pool of funders will have more enthusiastic funders. More enthusiastic funders means that you are less likely to funge with them. Equivalently, it means your dollar will do more, than in the other opportunity.

Equivalently, but perhaps surprisingly, between two opportunities with the same existing access to wealth, the one that has converted this to more investment means that it is more exciting to other funders. So Conviction would suggest to prioritize the one that has received the higher level of existing investment. This aligns with this post.

Having the wealth of existing funders as the denominator of this ratio makes it similar to 'Neglectedness' or 'Uncrowdedness' in 80'000 Hours and Owen Cotton-Barratt's ITN style frameworks. But, having Investment as the numerator makes it 'Conviction'.

Neglectedness

Neglectedness naturally arises in the equilibrium model in the three ways described in the following subsections. Value and risk neglectedness contribute positively to overall neglectedness, whereas excess opportunity costs subtract from it. 

Note that this is a different conception of "Neglectedness" from ITN. But arguably a more appropriate use of the term. The closest equivalent of ITN's Neglectedness is the Conviction component discussed above.

Relative importance

'Relative importance' captures the possibility that, compared to other funders, we think the opportunity is going to produce more value per unit of activity. Other funders will only fund the opportunity up to the point where they think the opportunity's marginal value matches their opportunity cost. Since we see more value than others, we should be willing to allocate more funding to such opportunities.  

Relative importance can and will be negative for areas or enterprises that we think are less effective or less urgent than other funders. Note also that the value of information, coordination and other considerations can be included in this component.

As an example of related thinking, 80,000 hours seems to be making a statement about value neglectedness here:

"One particularly important way that a problem can end up neglected for bad reasons is if other people simply don’t value it. This article argues that if you care about something X times more than the average person, you should expect to be able to have X times as much impact by working on that area (by your lights). For instance, we think that the interests of future generations are dramatically undervalued by society, so by working on issues that aid future generations, we can have far more impact."

Risk

Risk is about the extent to which the opportunity is associated with risks and uncertainties that our funding can help alleviate. With for-profit companies this is going to be the financial risk. With nonprofits this obviously doesn't apply. 

However, 'risk-like' costs can arise for non-profits. For example, other funders can be uncomfortable being too large a percentage of an opportunity's funding. Or, if they aren't super wealthy, then devoting too much funding now could start to increase their opportunity costs (by reducing funding for their alternatives). Such considerations may cause them to underfund some opportunities. 

Our participation in funding the opportunity can alleviate these 'risk-like' costs. Thus, opportunities that seem to be (partly) limited by Risk give us the possibility to add value to the market.

It is crucial that our funding can 'help alleviate' the 'risk-like' costs. Some kinds of uncertainty of other funders, e.g. fundamental moral uncertainty, may not be something that changes with our participation. I would account for such uncertainties in Effectiveness and Excess Value. 

Excess opportunity costs

Theoretically, all funders would have the same opportunity costs. Funders with lower opportunity costs would lend money to those with higher ones until there are no more trades like this that are worth doing. In reality, there are limits to such trading and a funder's opportunity costs will depend on their unique opportunity set.

It is not great for a funder with high opportunity costs to go into an opportunity where they crowd out funders with lower costs. It is likely to destroy value overall. So it is important to consider that there may be significant differences in funder opportunity costs.

Funders with special access or special knowledge are more likely to have higher quality opportunities and higher opportunity costs. Relative to the average person, EA community members would seem to have higher opportunity costs. But then EA's in special situations (e.g. active entrepreneurs, professional funders) are in turn more likely to have higher opportunity costs than the community average. 

Relative Risk Appetite

In equilibrium, if we have greater appetite for risk, then we can have impact even if all other funders have pretty much the same beliefs. Because we will be willing to take on more risk and disburse more funding where others won't. In terms of the diagram, we will contribute more to reducing the slope of the cost curve.

However, if we have an unusual tolerance for risk this will most likely apply across all opportunities. So it is probably only an important consideration when we observe that the risk appetites of the other funders in a particular opportunity are unusually high or low.

RRA seems like it might be the funder version of 'personal fit'. That is, if someone is more talented in an area or more passionate about an area, then they can take on more career risk by being more ambitious. Then, ultimately, they will probably have a greater impact.

Alignment

Unlike the other considerations, which factor into both the value and costs of an opportunity, alignment is only about the costs. It is about the value we place on what other funders will do with their money if we crowd them out.

For example, if we are considering funding AMF and we expect the other funders alternative is to fund Malaria Consortium, then we would likely judge this to be a situation of high alignment. Whereas, if we believed that AMF was the only highly effective charity that the other funders knew about (maybe they like the logo or something), then we might expect them to give to a much less effective charity as an alternative (or to not give at all). This would be a case of low alignment. 

This is an important consideration as there are many opportunities where we can expect most additional funding to crowd out existing funding. In these cases, the main effect of funding the specific opportunity is actually to increase funding for the market's alternative. If the market is unaligned then this could make funding a highly-effective, but crowded, opportunity a net loss.

Conclusion

In this post, I've described TPP's framework for assessing opportunities for impact in equilibrium. 

Compared to ITN, TPP's framework suggests that funders:

• Start by assessing the Baseline Effectiveness and Alignment of other relevant funders.
• Use Tractability according to its 80'000 Hours definition, but relabeled as Scalability.
• Analyze Neglectedness in terms of several granular parts, plus Conviction.
• Include an assessment of Relative Risk Appetite that has parallels to 'personal fit'.

We believe that these ideas could be useful for other researchers and funders. That said, I'm conscious that TPP's goal with this framework is for it to apply to a wide range of funding opportunities. It wouldn't be surprising if it misses crucial considerations in particular areas. I'd be interested to hear views about that specifically, as well as about this framework in general. I would greatly appreciate further discussion either in the comments below or via email.