Hello! One point that seems important to make: "People in the space" being skeptical of a startup idea, or even being confident it's a bad idea, is not good evidence that it's a bad idea.

Whilst we can expect subject matter experts to be skeptical of ideas that turn out to be bad, we can also expect them to be skeptical of a lot of ideas that turn out to be good! 

This is true of many extremely successful for-profit start-ups (it's mentioned in Y-Combinator lectures a lot) and of non-profits as well, including many of CE's most successful incubated charities. If i'm not mistaken, it's even true of CE itself! When it's co-founders asked experts for their advice when considering launching the Incubation Program, the majority view was that it was a bad idea. I, for one, am glad they didn't listen! If you look around the room you're in now, you're almost certainly surrounded by a number of inventions that "people in the space" said were impossible or a bad idea. The job of an entrepreneur (or a researcher identifying start-up ideas for entrepreneurs) is to figure out when "people in the space" are wrong, or at least likely enough to be wrong for it to be worth trying.

So in conclusion: Track record of CE incubated charities -- good indicator. Whether or not people in the space were skeptical -- not a good indicator. 

Another animal advocacy research organization supposedly found CE plagiarizing their work extensively including in published reports, and CE failed to address this.

Hi, I am Charity Entrepreneurship (CE, now AIM) Director of Research. I wanted to quickly respond to this point.

I believe this refers to an incident that happened in 2021. CE had an ongoing relationship with an animal advocacy policy organisation occasionally providing research to support their policy work. We received a request for some input and over the next 24 hours we helped that policy organisation draft a note on the topic at hand. In doing so a CE staff member copy and pasted text from a private document shared by another animal advocacy research organisation. This was plagiarism and should not have happened. I would like to note: firstly that this did not happen in the course of our business as usual research process but in a rushed piece of work that bypassed our normal review process, and secondly that this report was not directly published by us and it was not made clear to the CE staff member involved that the content was going to be made into a public report (most other work for that policy organisation was just used privately) although we should of course have considered this possibility. These facts of course do not excuse our mistake here but are relevant for assessing the risk that this was any more than a one-off mistake.

I was involved in responding when this issue came to light. On the day the mistake was realised we: acknowledged the mistake, apologised to the injured party, pulled all publicity for the report, drafted an email to the policy org asking to have the person who's text was copied added as a co-author (the email was not sent until the following day after as we waited for approval from the injured party). The published report was updated. Over the next three weeks we carried out a thorough internal risk assessment including reviewing all past reports by the same author. The other animal rights research organisation acknowledged they were satisfied with the steps taken. We found no cases of plagiarism in any other reports (the other research org agreed with this assessment), although one other tweak was made to a report to make acknowledgment more clear. 

FWIW I find mildlyanonymous' description of this event to be somewhat misleading referring to multiple "reports" and claiming "CE failed to address this".

CE's reports on animal welfare consistently contain serious factual errors ... noticing immediately that it had multiple major errors, sharing that feedback, and having it ignored due to their internal time capping practices.

I don’t know what this is about. I know of no case where we have ignored feedback. We are always open to receiving feedback on any of our reports from anyone at any time. I am very sorry if I or any CE staff ignored you and I open to hearing more about this, and/or hearing about any errors you have spotted in our research. If you can share any more information on this I can look into it, please contact me (I will PM you my email address, note I am about to go on a week's leave). It is often the case if we receive minor critical feedback after a report is published we do not go back and edit our report but note the feedback in our Implementation Note for future Charity Entrepreneurship founders, maybe that is what happened.

First a meta note less directly connected to the response:

Our funding circles fund a lot of different groups, and there is no joint pot, so it's closer to a moderated discussion about a given cause area than CE/AIM making granting calls. We are not looking for people to donate to us or our charities, and as far as I understand, OpenPhil and AWF do not have a participatory way to get involved other than just donating to their joint pot directly. This request is more aimed at people who want to put in significant personal time to making decisions independent from existing funding actors.
 

More connected response:

Thanks for the thoughts, and the support you have given our past charities. I can give a few quick comments on this. Our research team might also respond a bit more deeply.

1) Research quality: I think in general, our research is pretty unusual in that we are quite willing to publish research that has a fairly limited number of hours put into it. Partly, this is due to our research not being aimed at external actors (e.g., convincing funders, the broader animal movement, other orgs) as much as aimed at people already fairly convinced on founding a charity and aimed at a quite specific question of what would be the best org to found. We do take an approach that is more accepting of errors, particularly ones that do not affect endline decisions connected directly to founding a charity. E.g., For starting a charity on fish in a given country, we are not really concerned about the number of fish farmed unless that number is significant determining factor in terms of founding a charity in that space. We have gone back and forth as to how much transparency to have on research and how much time to spend per report and have not come to a fixed answer. We are more likely to get criticism/pushback on higher transparency + lower hours per report but typically think it will still lead to more charities that are promising in the end.
 

2) CE’s animal charity quality: I think both our ordering and assessment of charity quality would be different from what is described here. I also think animal welfare funds and Open Phil's (both of who have funded the majority of these projects) assessments would also not match your description. However, in some ways, these are small differences as our general estimate is that 2/5 charities in a given area are highly promising. It is quite a hits-based game, and that is the number we would expect (and would rank internally) about how many charities are performing really well.
 

2.5) Feedback on animal charities: I did a quick review of charities that got the most positive vs negative feedback at the time of idea recommendation from the broader animal community relative to your rank order and relative to our internal one and did not find a correlation. Generally, I think the space is pretty uncertain and thus the charity that got the most positive expectations were typically the charities that deviated the smallest amount from other actions already taken in the space. I think that putting more time into the research reports (including getting more feedback) is one way to improve charity quality (at the cost of quantity) but I'm pretty skeptical it's the best way. So far, the biggest predictive factor has not been idea strength but the founder team, so when thinking about where to spend marginal resources to improve charities, I would still lean that way (although it's far from clear if that will always be the case).
 

3) I would be interested in doing a survey on this to get better data. I get the impression that we are seen as pretty disconnected from the animal space (and I think that is fairly true). I think we are far more involved in e.g., the EA space both when it comes to more formal research and when it comes to softer social engagement. I think our charities tend to go deeper into whatever area they are focusing on than our team does, and I am pretty comfortable with that. I would not be surprised if we both were invited less and attended less coordination events and meetings connected to the animal space; we like to stay focused quite directly on the problems we are working on.
 

Thanks again for writing this up. I put some chance that these are issues that are correct and important enough to prioritize, and it's valuable to get pushback and flags even if we end up disagreeing about the actions to take.

Thanks for sharing your thoughts! I feel like your comment would be more valuable/credible if you elaborated further on your claims. You say the ideas for the animal charities were bad, but you provide no justification, and most of them not succeeding should not update one much given charities being successful is unlikely on priors.

2. CE's charities working on animal welfare have mostly not been very good, and listening to external feedback prior to launching them would have told them this would happen.

[...] doesn't do CE's original proposed idea anymore

On the point of the charities not doing CE's originally proposed idea anymore, I want to clarify that we don't see charities tweaking an idea as a failure but rather as the expected course of action we encourage. We are aware of the limitations of desktop research (however in-depth), and we encourage organizations to quickly update based on country visits, interactions with stakeholders, and pilot programs they run. There are just some informations that a researcher wouldn't be able to get, and they need input from someone working on the ground. For example, when Rethink Priorities was writing their report on shrimp welfare, they consulted SWP extensively to gain that perspective. Because CE charities operate in extremely neglected cause areas, there is often no other "implementer" our research team can rely on. Therefore, organizations are usually expected to change the idea as they learn in their first months of operations. I see this as a success in ingraining the values of changing one's mind in the face of new evidence, seeking this evidence, and making good decisions on the side of co-founders with the support of their CE mentors, and we are happy when we see it happen. 
There is a complex trade-off to be made when balancing the learning value from more in-depth desktop research vs. more time spent on learning as one implements, and I don't think CE always gets it right, but the latter perspective is often misunderstood and underappreciated in the EA space. 

Regarding charities specifically, in general, we expect about a 2/5 "hit rate" (rarely because of the broad idea being bad, more often because the implementation is challenging for one reason or another), and many people, including external charity evaluators and funders, have a different assessment of some of the charities you list. That being said, if you have any specific feedback about the incubated organization's strategies or ideas, please reach out to them. As you mentioned, they are open to hearing input and feedback. Similarly, if you have specific suggestions about how CE can improve its recommendations, please get in touch with our Director of Research at sam@charityentrepreneurship.com; we appreciate specific feedback and conversation about how we can improve. Thank you for your support of multiple CE charities so far! 

Hey, just chiming in here on behalf of the organization I co-founded (Fish Welfare Initiative). We went through AIM’s charity incubation program in 2019—their first formal cohort.

The following are a couple points I had:

1 - Echoing requests for evidence

As some people have already commented above, insofar as you have serious criticisms about various charities (CE or otherwise) it’d be helpful for you to provide some evidence for them.

In particular, it’d be interesting to learn more why you think AAC is “okay”, why Animal Ask “hasn’t had much impact”, and/or why FWI “hasn’t worked very well.”

I really think I would be happy to consider these arguments, but I first want to understand them.

I would personally wager that CE leading giving in the animal space would be net-negative for the space compared to the status quo (which, to be fair, is very bad already)

It’d also be helpful to know why you think the animal space, or maybe just giving in the animal space, is “very bad already”. (I know that in particular might be a lot for you to respond to though.) This brings me to my second point.

2 - Just because animal/CE charities are flawed doesn’t mean they’re not worth supporting.

One thread of your comments is one I really resonate with: The animal movement is not good enough. Our evidence is often subpar, decisions are made hastily, we don’t have the right people, etc. Unfortunately, I think this is all true.

But what should we really do differently? If, as you suggest, CE produces not super great animal charities, but it’s still (as you say) “the best bet in the animal space for future new high impact orgs”, then should we just resign ourselves to not launching and running any new animal-focused charities?

My point here is that just because something isn’t as good as we would like (e.g. IMO the best animal charities don’t have even 10% the evidence base of GiveWell’s top charities), that doesn’t mean they’re not worth doing or supporting. Sometimes I think we do ourselves a disservice by always comparing ourselves to human health/poverty alleviation charities: These human-focused orgs literally have decades or even a century more of an evidence base built up than we do. They don’t have an entrenched opposition. And they aren’t trying to change something people derive pleasure from 3 times a day.

We need to build a large and effective movement for reducing animal suffering and ending factory farming. That is going to require starting somewhere, no doubt with lots of early mistakes in the early days.

Of course, I don’t mean to say that anything goes—some ideas are still certainly too dumb to start and some charities too poorly-run to continue. However, I think we need to appreciate that we’re in the very early days of animal advocacy and we need to think about our approaches as such.

3 - On taking the advice of the EA Funds and OpenPhil over CE

This seems to be an important actionable takeaway you’d like people to have:

>>I don't think donors should take much guidance from them, compared to OpenPhil or the EA Animal Welfare Fund

Just wanted to point this out in case you’re not already aware, but these two granting bodies already heavily grant to CE-incubated animal orgs.

For instance:

• FWI has received about 5 grants from the EA AW fund over the years, and 1 grant from Open Philanthropy.
• Animal Ask has received at least 1 grant from the EA AW fund and 2 grants from OpenPhil.

And I believe SWP and AAC have also received money from one or both of these funders.

So it seems like either you should think that a) CE animal orgs are actually more promising than you claimed, b) the EA AW Fund and OpenPhil are actually less promising than you implied, or c) these funds are just scraping the bottom of the barrel and grant to CE orgs for lack of better options.

Fwiw, and after talking a reasonable amount with these funders, I’m fairly of the opinion that correct answer is mostly A here.

4 - About Fish Welfare Initiative (FWI) specifically

It’s worth noting that FWI has varied a fair bit from the original idea (see the short published report here) that CE had made when we first launched. Broadly though, CE didn’t give us that certain of a direction—rather, we understood that there are serious problems with how humans raise farmed fish, dissolved oxygen is one of them, and we should do further research to design a specific intervention to help them. Of course it would have been better if there was better research or a more concrete direction for us to go in, but again: We are in the early days of the animal movement and there’s still not enough of an evidence base for most things.

I also agree with Karolina above that it’s not necessarily bad that charities pivot from the original idea (provided that they pivot to something useful).

As for how promising FWI is today, I’d be interested to hear (as I stated in Point 1 above) why you think FWI “hasn’t worked very well”. As I state in Point 2, I think we have certainly made loads of mistakes, but that we’re also having a moderate impact right now and investing in tackling a very important and very neglected problem. You can learn more specifically about all this in our last year in review, or also by seeing our current projects.

Also as mentioned in Point 3, we have received grants from OpenPhil and the EA AW Funds, and are a recommended charity by ACE. Perhaps you think that these organizations have made some mistake in recommending FWI, but then I think you’re in a position of doubt on the entire animal movement (which, to be fair, seems like that might be the position you are in). To that, I would say see my Point 2—these are the early days, and even though no org is perfect we need to start somewhere.

5 - Feel free to dm me

I think it’d be interesting to hear your response to some or all of these points publicly as other people seem to have similar questions, but if you feel uncomfortable doing that feel free to dm or email me. I think there’s a good chance we already know each other, in which case I’d be especially interested to chat more to come to some shared truth here.

Sorry again all for the novel of a comment!

There is a widely held view in the animal research community that CE's reports on animal welfare consistently contain serious factual errors

To the extent this view is both valid and widely-held, and the reports are public, it should be possible to identify at least some specific examples without compromising your anonymity. While I understand various valid reasons why you might not want to do that, I don't think it is appropriate for us to update on a claim like this from a non-established anonymous account without some sort of support.

I don't think donors should take much guidance from them, compared to OpenPhil or the EA Animal Welfare Fund, and I would personally wager that CE leading giving in the animal space would be net-negative for the space compared to the status quo (which, to be fair, is very bad already).

If you had total control over all donations in the EA animal space, how would you change things compared to the status quo?

For the main point of your argument, I echo Vasco Grilo's point that your critiques of specific would be more compelling with justification or sources backing up your views. For any given charity idea, I have no reason to think that the fact that somebody on the internet thinks it's a bad idea prior to launch correlates with that idea actually being bad. Every new idea has people who are sceptical of it - that doesn't provide much information one way or the other. I'd be more interested to see a detailed evaluation of each charity in terms of actual impact they may have (or have not) delivered. I can only speak for my experience at Animal Ask, but a couple of recent, detailed evaluations do exist, and we invest a great deal of energy into critically evaluating our own work (and having it evaluated by others).

(As always, my views are my own, not those of my employer.)

CE has a fairly strong reputation of being hostile / non-collaborative

Could you elaborate on "being hostile"? Do they have a reputation for causing harm, or is it just about not listening to feedback?

Most of these are just "people in space knew this wouldn't work". Could you share more specific criticisms? As Aidan said, the biggest successes come from projects no one else would do, so without more information that seems like a very weak criticism. 

Just to note: I have a COI in commenting on this subject. 

I strong downvoted your comment, as it reads to me as making bold claims whilst providing little supporting evidence. References to "lots of people in this area" could be considered to be a use case of the bandwagon fallacy. 

It also concerns me that I’ve seen 5 instances of this post being disagree voted and strong downvoted, then a CE staff member commenting right after. I think those are obviously things people have a right to do if it is CE staff downvoting and disagreeing, but it means this post, outside of CE staff, might have fairly strong agreement from many people, which seems like an important note, given that it is still very positive karma, and without those votes might be positive agreement on balance.