Personal Strategist working with EAs. More @ teebarnett.com
More on my coaching trials with a dozen EA leaders: https://bit.ly/3DwyVgg
I'm a co-founder and current board director of Rethink Charity (RC), a project collective that launched and/or incubated several EA community building projects, including Rethink Priorities, RC Forward, the EA Hub, the EA Survey, EA Giving Tuesday, and fiscal sponsorship for numerous startup EA-aligned projects.
RC Projects that have since closed down include Students for High-Impact Charity, the Local Effective Altruism Network and Rethink Grants.
I've got a similar feeling to Khorton. Happy to have been pre-empted there.
It could be helpful to consider what it is that legibility in the grant application process (for which post-application feedback is only one sort) is meant to achieve. Depending on the grant maker's aims, this can non-exhaustively include developing and nurturing talent, helping future applicants self-select, orienting projects on whether they are doing a good job, being a beacon and marketing instrument, clarifying and staking out an epistemic position, serving an orientation function for the community etc.
And depending on the basket of things the grant maker is trying to achieve, different pieces of legibility affect 'efficiency' in the process. For example, case studies and transparent reasoning about accepted and rejected projects, published evaluations, criteria for projects to consider before applying, hazard disclaimers, risk profile declarations, published work on the grant makers theory of change, etc. can give grant makers 'published' content to invoke during the post-application process that allows for the scaling of feedback. (e.g. our website states that we don't invest in projects that rapidly accelerate 'x'). There are other forms of pro-active communication and stratifying applicant journeys that would make things even more efficient.
FTX did what they did, and there is definitely a strong case for why they did it that way. In moving forward , I'd be curious to see if they acknowledge and make adjustments in light of the fact that different forms and degrees of legibility can affect the community.
why it’s at least a non-obvious decision
Will we provide feedback to rejected applicants in the future? Possibly, but I think this involves complex tradeoffs and isn't a no-brainer
So I don’t think we should be doing this now, but I’m not saying that we won’t try to find ways to give more feedback in the future (see below).
Very much appreciate the considerate engagement with this. Wanted to flag that my primary response to your initial comment can be found here.
All this makes a lot of sense to me. I suspect some people got value out of the presentation of this reasoning. My goal here was to bring this set of consideration to yours and Sam's attention and upvote its importance, hopefully it's factored into what is definitely non-obvious and complex to decide moving forward. Great to see how thoughtful you all have been and thanks again!
Okay, upon review, that was a little bit too much of a rhetorical flourish at the end. Basically, I think there's something seriously important to consider here about how process can negatively affect community health and alignment, which I believe to be important for this community in achieving the plurality of ambitious goals we're shooting for. I believe FTX could definitely affect in a very positive way if they wanted to
an opportunity cost to providing feedback
huge mistake for Future Fund to provide substantial feedback except in rare cases.
Yep, I'd imagine what makes sense is between 'highly involved and coordinated attempt to provide feedback at scale' and 'zero'. I think it's tempting to look away from how harmful 'zero' can be at scale
> That could change in future if their other streams of successful applicants dry up and improving the projects of people who were previously rejected becomes the best way to find new things they want to fund.
Agreed – this seems like a way to pick up easy wins and should be a good go-to for grant makers to circle back. However, banking on this as handling the concerns that were raised doesn't account for all the things that come with unqualified rejection and people deciding to do other things, leave EA, incur critical stakeholder instability etc. as a result.
In other words, for the consequentialist-driven among us, I don't think that community health is a nice-to-have if we're serious about having a community of highly effective people working urgently on hard/complex things
Related thread that Sam and Nick are speaking on: https://forum.effectivealtruism.org/posts/hDK9CZJwH2Cqc9n9J/some-clarifications-on-the-future-fund-s-approach-to?commentId=JLDaxyk8uxdwcYNye#comments
Thanks to Sam and Nick for getting to this. I think it's very cool that you two are taking the time to engage. In light of the high esteem that I regard both of you and the value of your time, I'll try to close the loop of this interaction by leaving you with one main idea.
I was pointing at something different than what I think was addressed. To distill what I was saying: >> Were FTX to encounter a strong case for non-negligible harms/externalities to community health that could result from the grant making process, what would your response to that evidence be? <<
The response would likely depend on a hard-to-answer question about how FTX conceives of its responsibilities within the community given that it is now the largest funder by far.
Personally, I was hoping for a response more along the lines of "Oh, we hadn't thought about it that way. Can you tell us more? How do you think we get more information about how this could be important?"
I was grateful for Nick's thoughtful answer about what's happening over there. I think we all hear what you're saying about chosen priorities, complexity of project, and bandwidth issues. Also the future is hard to predict. I get all that and can feel how authentically you feel proud about how hard the team has been working and the great work that's been done already. I'm sure that's an amazing place to be.
My question marks are around how you conceive of responsibility and choose to take responsibility moving forward in light of new information about the reality on the ground. Given the resources at your disposal, I'd be inclined to view your answer within the lens of prioritization of options, rather than simply making the best of constraints.
As the largest funder in the space by far, it's a choice to be open to discovering and uncovering risk and harms that they didn't account for previously. It's a choice to devote time and resources to investigate them. It's a choice to think through how context shifts and your relationship to responsibility evolves. It's a choice to not do all those things.
A few things that seem hard waive away:
1) 1600 -1650 (?) rejected applications from the largest and most exciting new funder with no feedback could be disruptive to community health
Live example: Established organization(s) got rejected and/or far less than asked for with no feedback. Stakeholders asked the project leaders "What does it mean that you got rejected/less than you asked for from FTX? What does that say about the impact potential of your project, quality of your project, fitness to lead it, etc." This can cause great instability. Did FTX foresee this? Probably not, for understandable reasons. Is this the effect that FTX wants to have? Probably not. Is it FTX's responsibility to address this? Uncertain.
2) Opaque reasoning for where large amounts of money goes and why could be disruptive to community health
3) (less certain regarding your M&E plans) Little visibility on M&E given to applicants puts them in a place of not only not knowing what is good, but also how they know they're doing well. Also potentially disruptive
In regards to the approach moving forward for FTX, I wouldn't be surprised if more reflection among the staff yielded more than 'we're trying hard + it's complex + bandwidth issues so what do you want us to do?' My hope with this comment is to nudge internal discussions to be more expansive and reflective. Maybe you can let me know if that happened or not. Insofar as I delivered this in a way that hopefully didn't feel like an attack, if you feel including me in a discussion would be helpful, I'd love to be a part of it.
And finally, I'm not sure where the 'we couldn't possibly give feedback on 1700 applications' response came from. I mentioned feedback, but there's innumerable ways to construct a feedback apparatus that isn't (what seemed to be assumed) the same level of care and complexity for each application. A quick example – 'stratified feedback' – FTX considers who the applicant is and gives varying levels of feedback depth depending on who they are. This could be important for established EA entities (as I mentioned above), where for various reasons, you think leaving them completely in the dark would be actively harmful for a subnetwork of stakeholders. My ideal version of this would also include promising individuals who you don't want to discourage, but for whatever reason their application wasn't successful.
Thanks for taking the time. I hope this is received well.
Also not trying to lay this all at FTX's doorstep. Hoping that raising this will fold into some of the discussions about community effects behind closed doors over there
Thanks for writing this up, Nick. It seems like a pretty good first step in communicating about what I imagine is a hugely complex project to deploy that much funding in a responsible manner. Something for FTX to consider within the context of community health and the responsibilities that you can choose to acknowledge as a major funding player:
– How could a grant making process have significant effects on community health? What responsibilities would be virtuous for a major funding player to acknowledge and address? –
I've picked up on lots of (concerning) widespread psychological fallout from people, especially project leaders, struggling to make sense of decision-making surrounding all this money pouring into EA (primarily from FTX). I wouldn't want to dichotomize this discussion by weighing it against the good that can be done with the increased funding, but there's value in offering constructive thoughts on how things could be done better.
What seems to have happened at FTX is some mixture of deputizing several individuals as funders + an application process (from what I've been hearing) that offers zero feedback. For those involved over there, is this roughly correct?
If indeed there are no other plans to handle the fundamentals of grantmaking beyond deployment of funds, fundamentals that I believe dramatically affect community health, unless I someone can persuade me otherwise, I'd predict a lot more disoriented and short-circuited (key) EAs, especially because many people on this community orient themselves in the world of legible and explicit.
In particular, people are having trouble getting a sense of how merit is supposed to work in this space. One of the core things I try to get them to consider, which is more pronounced perhaps now more than ever, is that merit is only one of many currencies upon which your social standing and evaluation of your project rests. This is hard for people to look at.
I hope FTX plans to take more responsibility for community health by following up with investment in legible M&E and application feedback. Echoing what I said a month ago about funders in general: https://bit.ly/3N1q3To
"Funders could do more to prioritize fostering relationships – greater familiarity with project leaders reduces inefficiencies of all sorts, including performative and preparation overhead, miscommunication, missed opportunities, etc.
In my opinion, this should also apply to unsuccessful projects. A common theme that I’ve seen from funders, partly due to bandwidth issues though not entirely, is aversion to giving constructive feedback to unsuccessful projects that nonetheless endure within the community. Given my firsthand experience with many clients who are fairly averse to interpersonal conflict, it wouldn’t surprise me if aversion to conflict + public relations considerations + legal issues (and other things) precluded funders from giving constructive feedback to failed applications. Funders would likely need to hold the belief that this feedback would meaningfully improve these projects prospects, and therefore the community overall, in order to put in the requisite effort to get through these blocks to this type of action. They’d also likely need to feel reassured that the feedback wouldn’t be excessively damaging reputationally (for both themselves and others), destabilize the community, or the integrity of community norms.
EA leaders are often at least partially in the dark regarding expectations from funders. This could be the case for many reasons, but a common reasons among leaders included the following:
• Reputational fears – Reticence to reach out due to some (un)justifiable fear of reputational harm
• Value system clash/lack of familiarity – not wanting to waste the time of funders, usually due to lack of familiarity and fears of how they would be received, but also sometimes a principled decision about not wanting to bother important decision-makers
• Not having considered reaching out to funders regarding expectations at a meaningful enough grain of detail
• (Likely not always misplaced) concerns about arbitrariness of the evaluation process
• Preparation overhead – not being ‘ready’ in various ways. In some cases, My outside view of the situation led me to believe that quite a bit of preparational overhead and perfunctory correspondence could be avoided if funders made it clearer that they care less about certain aspects of performative presentation "
Yeah. On the face of it, I could see how this feels like an easy ask, but I intentionally constructed this post in such a way as to have my work stand up and be evaluated on its own, without being associated with (or positioned against) other programs, coaches, or theoretical paradigms for now. I'll have to spend a bit more time thinking through the differences between displaying in terms of 'highlighting', 'promoting', 'recommending' and 'publicly outing'. What to look out for in both positive and negative senses sounds like something that actually could be a great post on its own. Maybe we could co-author that.
With that said, I'm curious to hear what high quality you'd like to promote. I'm guessing Paradigm?
This answer might make the above make more sense. My understanding is that Paradigm isn't currently active, but were it still an option, I would restrict the scope of my recommendation to attending their workshops and working closely with specific coaches. For someone looking for a well-structured coaching program and hoping for a widely-recognized credential to earn, it wouldn't be a very good choice. For me personally, my style of learning is boosted tremendously by fruitful individual relationships (great mentors, coaches, etc.) I like to think it worked out quite well in that sense
Can confirm that Luke was a huge proponent of this from our interactions from ~2016 – ~2019. It's one of the primary reasons Rethink Charity created and maintained our governance structure, which I thought was only moderately good but likely above average relative to what I've seen and heard about in the community