We think WAI’s grantmaking criteria—such as Neglectedness, Scope, and Impact—are explicitly designed to prioritize cost-effectiveness and maximize counterfactual impact for large numbers of animals. Beyond that, their distribution may be limited by the types of projects they receive suitable applications from.

Thanks for the positive feedback! 

Shrimp Welfare Project’s ranges are narrower for a few reasons. Because SWP works directly with farmers, they can track and estimate the number of shrimp on partner farms, reducing uncertainty about the animals affected. We also either used point estimates or narrow ranges for other parameters, such as the duration of impact (based on the lifespan of electrical stunners) and the duration of improved water quality. This means the main source of uncertainty in SWP’s CEA lies in the SADs estimates, whereas other charitie... (read more)

We don’t intentionally aim to represent a broad range of approaches among our Recommended Charities. While we take steps to invite a pluralistic pool of applicants—especially from underfunded areas—those considerations don’t factor into our selection for evaluation, our assessments, or our decision making. If we thought that funding a marginal charity would have less impact than supporting the others, we wouldn’t recommend them, even if their inclusion could add more diversity of approaches to our list of Recommended Charities.

The animal advocacy movement ... (read more)

Ideally, we would be fully reliant on SADs, which take into account the species' capacity to suffer as well as the intensity and duration of their suffering. However, SADs are still a new method with some speculative inputs and ongoing updates. To account for this methodological uncertainty, our CEAs show results in both SADs averted per dollar and animals helped per dollar. In our decision-making, we look at both of these metrics and more, and interpret them alongside the broader context of the intervention. 

While we have high confidence in the quality of their fellowship program (with fellows reporting high rates of improved leadership skills, increased confidence, and motivation to pursue roles to help animals), as well as in the thorough monitoring, evaluation, and learning (MEL) that New Roots Institute conducts for their programs, we’re not sure about the extent to which fellows are significantly stronger advocates because of the fellowship, and whether they fill key talent bottlenecks. Overall, this leads us to not being convinced that their cost effecti... (read more)

Thanks for your comments and your interest in WAI’s work! 

While we agree that an established field should focus on helping the most abundant animals, we also agree with WAI’s reasoning that while building the field, having a singular focus on optimizing for the number of animals would come at the expense of other strategic field-building goals. 

We address this in WAI’s review, e.g., here: “Though not all grants funded have a very high scope, this aligns with WAI’s long-term strategy that balances maximizing immediate impact with building a diverse and engaged scientific field. This dual strategy is based on sound reasoning and endorsed by several experts we spoke to.”

ACE’s Evaluations program has a higher bar for uncertainty than Movement Grants, given that (i) the financial and non-financial benefits we direct toward recommended charities are greater, (ii) our target audiences have different expectations, and (iii) the downside risks are higher. These aversions include, but are not limited to, a lack of track record/wins/achievements, variance in possible outcomes, low probability of very high impact, and unknown probabilities.

We thought that LIC’s cost-effectiveness analysis was very promising, but it was ultimately ... (read more)

You can find more information about our selection process here. In 2024, GFI decided to postpone re-evaluation to a future year to allow their teams more time to focus on opportunities and challenges in the alternative proteins sector. They decided not to apply to be evaluated in 2025.

Thanks, Vasco! We appreciate the feedback. For a complete view of each charity’s spending and cost-effectiveness, we encourage looking into their respective reviews, particularly the Cost-Effectiveness and Financials and Future Plans spreadsheets.

It’s correct that we maintained a 45% weight for the six academic estimates of disutility across different pain levels, and that these estimates are orders of magnitude lower than those reported in the EA and animal advocacy community surveys (which we weighted slightly higher at 55%). As you know, we’re... (read more)

Thank you for raising questions about ACE’s values and priorities. While we understand that the original claim made in the post has since been retracted, we still wanted to take the opportunity to respond—both for transparency and trust within the community, and to engage our team in clarifying our approach. These conversations help us reflect, improve, and ultimately strengthen our work to reduce animal suffering as effectively as possible.

ACE’s mission remains squarely focused on identifying, promoting and fundraising for impactful strategies and organiz... (read more)

Hi Vasco, we intend to publish a blog post on the consequences of farmed animal welfare interventions for wild animals, after the busy work of charity evaluations is wrapped up for the season. Thank you.

Hi Vasco, thank you for encouraging us to think about the downstream effects of farmed animal interventions on wild animals whose experiences are so neglected. As you noticed by the selection of charities we've made, we are not confident enough yet of the potential impact on the wellbeing of free ranging individuals like nematodes or even insects and larger wild animals. It is possible that in our theory of change analyses of charities this question will come up. But we expect that the uncertainty will mean we won't give the answer much weight in this evaluation round. Thanks.

Note: this comment has been posted in response to both Vetted Causes’ first and second posts about Sinergia, as it addresses points raised in each.

ACE welcomes rigorous external evaluations of our work, as such feedback strengthens our ability to provide high-quality recommendations to donors and, ultimately, helps us maximize our impact for animals. To this end, we engage with external experts on a) our methodology for evaluations and grantmaking and b) on evaluations and grant reviews themselves. For instance, we interact with the EA and FAST forums and ... (read more)

Note: this comment has been posted in response to both Vetted Causes’ first and second posts about Sinergia, as it addresses points raised in each.

ACE welcomes rigorous external evaluations of our work, as such feedback strengthens our ability to provide high-quality recommendations to donors and, ultimately, helps us maximize our impact for animals. To this end, we engage with external experts on a) our methodology for evaluations and grantmaking and b) on evaluations and grant reviews themselves. For instance, we interact with the EA and FAST forums and ... (read more)

Thanks for this interesting perspective on how to balance different values within the work of evaluations, Devin. Considering you drafted this in 2022, we do want to note that a lot has changed at ACE in the last three years, not least of which has been a shift to new leadership. Since early 2022, ACE has transitioned to a new Executive Director, Programs Director, Charity Evaluations Manager, Movement Grants Manager, Operations Director, and Communications Director. 

That said, ACE continues to assess organizational health as part of our charity evalu... (read more)

Hi Vasco,

We agree that the majority of our analysis should focus on the future work that would be enabled by ACE’s recommendation. However, forward-looking CEAs are inherently more subjective because they rely on projected metrics rather than actual past results. For this reason, we tend to create backward-looking CEAs and then assess whether there are any reasons to expect diminishing returns in the next two years (the duration of an ACE recommendation). When GWWC shared with us anonymized comments from the experts they consulted on this topic, the commen... (read more)

Thanks for the questions!

As noted in GWWC's report, our reasoning for recommending ÇHKD is that we think they're very plausibly competitive with our other recommended charities, such as Sinergia. Sinergia's CEA rested on more high uncertainty assumptions than ÇHKD’s did, and their CEA covered a smaller percentage of their work. We think it's reasonable to support both a charity that we are more certain is highly cost-effective (such as ÇHKD) as well as one that we are more uncertain is extremely cost-effective (such as Sinergia). We also think ÇHKD may hav... (read more)

Thank you, Vasco! Yes, that's what I will be working toward next and assessing over time. I will be sure to share my findings :) 

Holly

Please find more information on animal suffering below: 

Farmed animal suffering

Trillions of farmed animals (including fishes) experience cruel treatment and brutal deaths on factory farms. Farmed animal advocacy presents one of the most significant opportunities to reduce animal suffering on a large scale.

Chickens

Chickens are among the most abundant farmed animals, outranked only by farmed fishes and certain invertebrates, with an estimated 75 billion killed annually. When given enough space and an appropriate environment, chickens will forage fo... (read more)

We would like to extend our gratitude to Giving What We Can (GWWC) for conducting the "Evaluating the Evaluators" exercise for a second consecutive year. We value the constructive dialogue with GWWC and their insights into our work. While we are disappointed that GWWC has decided not to defer to our charity recommendations this year, we are thrilled that they have recognized our Movement Grants program as an effective giving opportunity alongside the EA Animal Welfare Fund.

Movement Grants

After reflecting on GWWC’s 2023 evaluation of our Movement Grants (MG... (read more)

Hi Isaac! Now that we’ve announced our 2024 Recommended Charities, we’ve had more time to process your feedback. Thanks again for engaging with our work.

As mentioned before, we’ve substantively updated our evaluation methods this year. This was informed in part by detailed feedback we received as part of Giving What We Can’s 2023 ‘Evaluating the Evaluators’ project, some of which aligns with your feedback. 

One of these changes is that we now seek to conduct more direct cost-effectiveness analyses, rather than the 1-7 scoring method that we u... (read more)

Hi Pablo, important question! GFI decided to postpone re-evaluation to a future year to allow their teams more time to focus on new opportunities and challenges in the alternative proteins sector. Our researchers will be available to answer more questions about our 2024 charity recommendations in our AMA next week (Nov 19, 8-10am PT) on the FAST Forum. We hope to catch you there! Thanks, Holly

Hi Michael, I'm glad you're happy to see the cost-effectiveness models! And thank you for letting us know; there's been much more traffic than we're used which has made our website slow. We're actively trying to resolve this. Please hold on a bit while we get our site back up. Clearing your cache may also help :) Thanks for your patience everyone!

Thank you for spending time analyzing our methods. We appreciate those who are willing to engage with our work and help us improve the accuracy of our recommendations and reduce animal suffering as much as possible.

Based on previously received feedback and internal reflection, we have significantly updated our evaluation methods in the past year and will be publishing the details next Tuesday when we release our charity recommendations for 2024. From what we can tell from a quick skim, we think that our changes largely address Vetted Causes’ concerns here,... (read more)

Rest assured, our new charity application process and commitment to serious impact-driven advocacy is no joke. We're here to keep the conversation on animal advocacy going strong, even on April Fool's Day!

Thank you, Toby! We appreciate the positive feedback and definitely share your thoughts about the value of this exercise.

- Max

Hi Michael, thanks a lot for the helpful comments, and for taking the time to be so thorough in your feedback. We've been thinking a lot about how to produce proxies for impact that can be meaningfully compared with one another, with BOTECs being one possible way to help achieve that, so it's really useful to get your views. We'll talk these through as a team as we consider improvements to our process for the coming years.

- Max

Thanks for the kind words! Really glad to hear you're likely to support the great work being done by our Recommended Charities.

Like you say, involvement with EA is a hard thing to judge: I wouldn't feel qualified to name directors that I believe are involved in EA, for example. Also, while all the charities we recommend use evidence-driven strategies to achieve the maximum benefits for animals, many of them might not consider themselves EA, or might not want to be labelled as such for strategic reasons. In answer to your more specific question, two of our ... (read more)

Thank you for your thoughtful question and interest in our evaluation approach. At ACE, we recognize the unique challenges present in our domain, where there is often less data and consensus on effective interventions compared to GiveWell's focus on global health and poverty. We also evaluate charities using a diverse range of 26 types of interventions, some with complex, long-term Theories of Change that are challenging to quantify.

For these reasons, we currently don't apply a specific cost-effectiveness bar across all charities, but we are consistently r... (read more)

First, we want to sincerely thank Giving What We Can for running this “Evaluating the Evaluators” exercise. We recognize that ACE has set ourselves a difficult task, compounded by the fact that we’re the only organization doing what we do. Therefore, receiving this kind of feedback is both very rare and very welcome. There’s a great deal in GWWC’s report that will help us improve our processes for 2024, which ultimately means more animals will be helped and spared. While we were disappointed that GWWC has decided not to defer to our recommendations this ye... (read more)

Thank you for your comment! While there is limited hard evidence on the effectiveness of ballot measures, there have been some successful cases of them improving animal welfare standards over the last few decades in the U.S. (Schukraft, 2020). We agree that citizen initiatives such as ballot measures and other types of policy work are promising interventions, and hopefully, we will see more studies backing this up soon.

- Alina

Hi Nuño, we've now published our blog post on our approach to assessing Cost Effectiveness, including a charity’s approaches to implementing interventions, their recent achievements, and the costs associated with those achievements. Thanks, Holly

Hi Ben! Thanks for question, and I'm glad you're excited about our recommendations.

The situation was as you noted: a significant portion of New Harvest's assets were in the stock of companies that had not gone public. In retrospect, when we last evaluated New Harvest in mid-late 2021, it would have been more accurate for ACE not to count assets that may be difficult to liquidate quickly, because they are not truly available to maintain operations. As you'll see in the upcoming "Our Room for More Funding Approach in 2022" blog post, we have since updated ou... (read more)

Thank you for this initial feedback, Nuño - we appreciate you taking the time. In the coming weeks, we'll be publishing one blog post for each of the evaluation criteria that we use when reviewing charities (for a total of 4 posts). The blog posts will give more detail on how we made our assessments, how this year's approach differs from previous years, and any limitations we see. The Cost Effectiveness blog post will be published on Dec 15th, so we look forward to sharing more details then.

- Elisabeth

We continuously work to improve and strengthen our evaluations process and appreciate your thoughtful and constructive critique of it. As our research team expands under new leadership, we welcome the opportunity for you to contact our team to discuss the points above as we consider improvements to our processes. We would also be more than happy to clear up any misunderstandings you’ve outlined regarding the current evaluations process we have in place. Best of luck with your submission!