I think this would be good. one thing is that In many situations If you can write p(sucess) in a meaningful way then you should consider running a competition instead of grantmaking. Not going to work in every situation but I find this the most fair and transparent when possible.
This article asks whether EA grantmakers should publicly disclose the probability of success (p(success)) for their funded projects, and discusses the potential benefits such as improved community norms and accountability, as well as potential drawbacks and implementation considerations.
Recently I’ve been thinking about the possibility of EA grantmakers publicly sharing the p(success) of their funded projects. This article is intended to start a discussion by exploring the potential benefits and drawbacks of this approach, and is by no means exhaustive or super detailed.
This is an idea I had this week and initial conversations with people in Trajan House about the idea were interesting and positive enough that I thought it’d be worth opening up the conversation. I have no experience as a grantmaker, only as a grant applicant, and so I’m sure that I have a very poor understanding of how grantmaking actually works. Therefore I’m sure there are reasons I haven’t heard or thought about as to why this suggestion might have already been rejected, or wouldn’t be a good idea - which I’d like to read. I tried to get some grantmakers to take a look/comment on a pre-post draft of this, but didn’t have much luck. In any case, thanks to the two people that did give brief feedback.
I’d suggest that grantmakers publish a grantee-independant p(success) alongside the public grant disclosures that some of them already publish,[1]. [2]That is to say that I assume grantmakers are able to look at a project proposal independent of the grant applicant, and assign a p(success). Once they’ve done that, they then might factor in their subjective belief in the aptitude/competence/track record of the applicant and adjust the p(success|grantee) up or down, and keep that private. I think to avoid worries about how this disclosure might affect grantee’s mentality regarding their proposed project pre-execution, it could be that p(success) only be shared publicly after the grant period is over.
This used to be on FTXFF’s website but it no longer exists.
Here are Givewell’s - although I don’t know if I think my p(success) suggestion would be as important for them.
e.i. People in the community who seem to have a good reputation/high status are people who seem to have a track record of success, and people with a track record of failure don’t get much fame or recognition, even though we don’t know anything about how ambitious/low-p either person's projects were.
i.e. it shows that we put our money where our mouth is when it comes to how we make individual grant decisions, not just which causes we choose or how we choose causes.
I think most grants are not binary "sucess" or "failure" but outcomes have a lot more granularity than that -- so it would probably need to be a distribution of outcomes.
I think there are also often benefits like value of information (it's useful for someone to try, even if the project doesn't seem like it will be "successful" as written), upskilling, and the like to projects, which I expect many grantmakers (especially if you're thinking about things like small grants or funding a one-off project) try to evaluate and plan on.