Are you planning on doing any additional work towards "Explore preventative approaches" front? Where there any another avenues you saw for reducing/ preventing fragmentaiton over time similar to the enviormental risk assesment? Are/ could these being deployed in the regions driving the trend?
Intersting previous post! I think your caculations work similary assuming 100% of the demand for meat in pet food translates to farmed animals. The effect of a decrease in demand for ABPs on the number of farmed animals will be less than 1-1. Based on the Peter Alexander paper that would suggest a 45% to 63% discount on the contribution. (Which brings your number down to more like 1-2 billion, still alot in absolute terms). Although even then its more of a simplifying assumption as the real decrease would depend on counterfactual uses and how shifts in demand and prices eventual wash out. You also don't include fish or developments for insects that could boost the numbers back up the other way.
Although I don't think total numbers is main crux. Importance or neglectedness are only really good proxies for wether to look into an area which I think the case for vegan pet food has done fairly well.
Tractablity is very important. If we survey humans similary for are they open to plant-based diets we get a similar high or higher number, "38% of German and UK adults intend to change their diets by eating more plant-based foods". Broad intentions are not a great indicaiton of how cost-effective interventions to talk with pet owners, pet food companies or goverments to transition would be. For 1-1 outreach I'd expect targeting pet owners to be less effective than vegan outreach just for the fact that 2/3 of people won't be relevant.
I think more informative direction for future research in this area would be to do target research and some inital conversations and testing. I'd want to see that:
1. companies are shifting a large enough volume of product (when discounted for the ABP effect on overall demand for animal farming) that targeting this could plausibly be worthwhile 2. work with Pet food companies, retailers or other institutions to shift proportions of ingredients in food to be more plant-based for better animal welfare and climate outcomes seems tractable as a campaign. Thinking why they would do this? Would people support action against them if they don't? Are there other incentives we could use? 3. It does not backfire. We would have to watch out the small animal replacement problem if they shift to chicken or fish for climate reasons.
Maybe there are potential cost-effective campaign targets but high meat proportion is usually a selling point for pet feeds. At the moment i'd expect running a campaign like that would be more difficult relative to how important the campaign target is compared to the marginal cage-free or broiler campaign.
I do think some companies are acting based on being more aligned such as high-end brands like Waitrose in the UK. Even in these cases, it can be a kind of getting things over the line scenario, where talking to them is the small nudge that results in counterfactual changes.
But as FAI mentions the cost of "bad cop" actions to companies seems significant. If you're looking for RCT-level evidence of this unfortunately we don't have it. This mostly looks at case studies, broadly how companies value their reputations and how comparable corporate scandals affect market evaluation and performance. I'd be interested to see this replicated specifically for cage-free. Taking historic or upcoming campaigns by working with groups for intel and tracking their effect on companies.
Largely agree with these only one I would add to/ expand on is 2. There is both the vision/disentanglement type aspect of this but also having sufficient evidence that a particular type of research is worth doing. Maybe I'm just bad at 2 but I suspect the reason 1 is never reported as a constraint is it's much easier to think of plausible exciting research projects and hard to confidently prove/disprove a project's theory of change. Researchers in lots of disciplines come up with seemly good research questions (to them) all the time but very little research is actually high impact. So I suspect in practise most research organizations should be constrained by this more than they are.
I'm interested what you think is both the constraight most organizations would report and if you think this lines up with what their actual constraints are
Awesome, sure that would be great experience for using similar collective action in other causes! Had similar thoughts when writing this, although we focus on animal advocacy in principle all our approach reports could be used for other asks in other cause areas. I'd be interested in takes from those working in these domains
Great idea, for the more risky actions that could be a good approach if there are aligned unions. Do you think this is something you could have got the unions you helped on board with? Either animal advocacy or other potentially impactful causes?
At Animal Ask we did later hear some of that feedback ourselves and one of our early projects failed for similar reasons. Our programs are very group-led, as in we select our research priorities based on groups looking to pursue new campaigns. This means the majority of our projects tend to focus on policy rather than corporate work, given more groups consider new country-specific campaigns and want research to inform this decision.
In the original report from CE, they do account for the consolidation of corporate work behind a few asks. They expected the research on corporate work to be 'ongoing' deeper' and 'more focused research'. So strategically would look more like research throughout the previous corporate campaign to inform the next with a low probability of updating any specific ask. The expectation is that it could be many years between the formation of corporate asks.
So in fact this consolidation was highlighted in the incubation program as a reason success could have so much impact. As with the large amount of resources the movement devotes to these consolidated corporate asks ensuring these are optimised is essential.
As Ren outlined we have a couple of recent, more detailed evaluations and we have found that the main limitations on our impact are factors only a minority of advisors in the animal space highlighted. These are constraints from other organisation stakeholders either upper management (when the campaigns team had updated on our findings but there was momentum behind another campaign) or funders (particular individual or smaller donors who are typicaly less research motivated than OPP, EAAWF, ACE etc.)
You can see this was the main concern for CE researchers in the original report. "Organizations in the animal space are increasingly aware of the importance of research, but often there are many factors to consider, including logistical ease, momentum, and donor interest. It is possible that this research would not be the determining factor in many cases".
From the main body of the text: "Plant-based products represented 0·011 % of product unit sales in the pre-intervention period. This increased to 0·016 % during the intervention period and 0·012 % in the post-intervention period. Meat products represented 26·52 % of sales in the pre-intervention period, 26·51 % during the intervention period and 26·32 % in the post-intervention period. The remainder of sales were represented by non-meat products (73·47 % in pre-intervention and intervention periods, 73·67 % in the post-intervention period)."
One thing to flag on reading into this as evidence against plant-based sales leading to lower meat consumption is how much harder it will be to detect a significant effect on meat consumption. The background variation on those is much higher so to detect a significant effect from a campaign like Veganuary we would need a much larger total effect size. Even if the 0.05% of pre/post change in plant-based came 100% from meat I'd expect it still would not be significant.
That seems like the 80/20 of this would be appropriate for a lot of candidates. I guess I assume that a lot of EA candidates have a higher bar for claims made in typical fundraising material so would benefit from delving deeper into the numbers. This depends on how much trust you already have in organisaions. Where if you think groups are already assessed with enough rigor by funders, e.g they have a GiveWell recommendation, then the time cost of going through the numbers makes less sense. I think this would work best for meta-groups like the organisaion I work for Animal Ask or others like Animal Advocacy Careers, Charity Entrepreneurship, 80,000 hours, Rethink Priorities, Global Priorities Insitute etc.
Hey Sofia, Great idea. Groups have usually indicated they would spend <10% of the time we spend researching without our involvement, so this seems like a more viable idea than one may expect. There are some reasons this may not entirely cross-apply to the rest of our work. Such as concerns with groups anchoring too much to their more shallow research, which usually results in more optimistic assessments (Optimizer's Curse). Or possibly a selection effect with the groups that are willing to do this being more likely to make better decisions. We are tracking the asks other similar organisations are using in the regions or areas we have worked in to. This gives us some sense of this, but a more direct experiment of this kind could be valuable. Particularly if we ran it with a few groups using different advocacy methods. We will look into the idea more as well as some of the other ways we could amend our pre/post surveys before we partner with the next group!
I agree that in hindsight a summary of each indicator would probably have been useful to provide the reader with an overall assessment given the information I reviewed in the report.
That model is roughly the way I was thinking of this assessment, with validity and interpersonal comparisons being how much I would update on a perfectly accurate measure, and reliability giving some sense how wide the confidence interval would be from a real world measurement. The trade off of these, between groups of indicators and individual indicators, adds some nuance so that a single physiological measure is reliable but can vary due to numerous other factors but a combination of them allows us to measure the welfare benefit of things that health can’t capture easily. For example, it may be better to minimise disease rates instead of blood glucose levels if given no context but disease rates would be unable to assess the importance of different types of environmental enrichment.
If more people comment to express interest in an overview of each section, I am happy to invest the time to go back through the report to add in these sections.
I think the ideal system would have a single measure that perfectly tracks what matters, no?
I definitely agree, which is partially why I put an example of self-reports in humans (which are in my opinion as close to ideal as we can get) alongside the measures we have available in other animals. This is what I currently view as the best available ('ideal') system given the weaker methods available.
My last question is: what are y'all's thoughts on making across species comparisons? This is the question that really interests me, and most of these indicators presented seem to be much, much more suitable to within species assessments of welfare.
In this context, many of these indicators struggle on cross species comparisons. Take cortisol for example, where different species have different cortisol levels, making it difficult to compare levels or even percentage changes across species. We can gain some sense of the relative importance of different improvements or events for an individual from the degree of change of an indicator. An example of this within an operant test could be a human showing a mild preference for social contact vs food compared to a fox who shows the opposite relationship. Yet, this still only gives us information within the range of their utility functions and doesn't tell us how their ranges compare. It's a challenging question and due to this we have mostly been deferring to Rethink Priorities’s work on moral weights and Open Philanthropy Project’s report on consciousness. At the moment, we approach this by using an assessment of an animal's quality of life, to gauge how important an improvement is within an individual's utility function, and then adjust this based on these considerations. However, I would be cautious about concluding that an ask is more promising if the deciding factors are based across species comparisons given the range of plausible views on the topic.
Are you planning on doing any additional work towards "Explore preventative approaches" front? Where there any another avenues you saw for reducing/ preventing fragmentaiton over time similar to the enviormental risk assesment? Are/ could these being deployed in the regions driving the trend?
Intersting previous post! I think your caculations work similary assuming 100% of the demand for meat in pet food translates to farmed animals. The effect of a decrease in demand for ABPs on the number of farmed animals will be less than 1-1. Based on the Peter Alexander paper that would suggest a 45% to 63% discount on the contribution. (Which brings your number down to more like 1-2 billion, still alot in absolute terms). Although even then its more of a simplifying assumption as the real decrease would depend on counterfactual uses and how shifts in demand and prices eventual wash out. You also don't include fish or developments for insects that could boost the numbers back up the other way.
Although I don't think total numbers is main crux. Importance or neglectedness are only really good proxies for wether to look into an area which I think the case for vegan pet food has done fairly well.
Tractablity is very important. If we survey humans similary for are they open to plant-based diets we get a similar high or higher number, "38% of German and UK adults intend to change their diets by eating more plant-based foods". Broad intentions are not a great indicaiton of how cost-effective interventions to talk with pet owners, pet food companies or goverments to transition would be. For 1-1 outreach I'd expect targeting pet owners to be less effective than vegan outreach just for the fact that 2/3 of people won't be relevant.
I think more informative direction for future research in this area would be to do target research and some inital conversations and testing. I'd want to see that:
1. companies are shifting a large enough volume of product (when discounted for the ABP effect on overall demand for animal farming) that targeting this could plausibly be worthwhile
2. work with Pet food companies, retailers or other institutions to shift proportions of ingredients in food to be more plant-based for better animal welfare and climate outcomes seems tractable as a campaign. Thinking why they would do this? Would people support action against them if they don't? Are there other incentives we could use?
3. It does not backfire. We would have to watch out the small animal replacement problem if they shift to chicken or fish for climate reasons.
Maybe there are potential cost-effective campaign targets but high meat proportion is usually a selling point for pet feeds. At the moment i'd expect running a campaign like that would be more difficult relative to how important the campaign target is compared to the marginal cage-free or broiler campaign.
Re-the impact of groups on corporations. I wrote a piece on this here.
I do think some companies are acting based on being more aligned such as high-end brands like Waitrose in the UK. Even in these cases, it can be a kind of getting things over the line scenario, where talking to them is the small nudge that results in counterfactual changes.
But as FAI mentions the cost of "bad cop" actions to companies seems significant. If you're looking for RCT-level evidence of this unfortunately we don't have it. This mostly looks at case studies, broadly how companies value their reputations and how comparable corporate scandals affect market evaluation and performance. I'd be interested to see this replicated specifically for cage-free. Taking historic or upcoming campaigns by working with groups for intel and tracking their effect on companies.
I think attribution is broadly a fair concern though and could affect many interventions outside of anything you are directly paying for e.g. any lobbying-based interventions would have the same concern regardless of cause area.
Largely agree with these only one I would add to/ expand on is 2. There is both the vision/disentanglement type aspect of this but also having sufficient evidence that a particular type of research is worth doing. Maybe I'm just bad at 2 but I suspect the reason 1 is never reported as a constraint is it's much easier to think of plausible exciting research projects and hard to confidently prove/disprove a project's theory of change. Researchers in lots of disciplines come up with seemly good research questions (to them) all the time but very little research is actually high impact. So I suspect in practise most research organizations should be constrained by this more than they are.
I'm interested what you think is both the constraight most organizations would report and if you think this lines up with what their actual constraints are
Awesome, sure that would be great experience for using similar collective action in other causes! Had similar thoughts when writing this, although we focus on animal advocacy in principle all our approach reports could be used for other asks in other cause areas. I'd be interested in takes from those working in these domains
Great idea, for the more risky actions that could be a good approach if there are aligned unions. Do you think this is something you could have got the unions you helped on board with? Either animal advocacy or other potentially impactful causes?
At Animal Ask we did later hear some of that feedback ourselves and one of our early projects failed for similar reasons. Our programs are very group-led, as in we select our research priorities based on groups looking to pursue new campaigns. This means the majority of our projects tend to focus on policy rather than corporate work, given more groups consider new country-specific campaigns and want research to inform this decision.
In the original report from CE, they do account for the consolidation of corporate work behind a few asks. They expected the research on corporate work to be 'ongoing' deeper' and 'more focused research'. So strategically would look more like research throughout the previous corporate campaign to inform the next with a low probability of updating any specific ask. The expectation is that it could be many years between the formation of corporate asks.
So in fact this consolidation was highlighted in the incubation program as a reason success could have so much impact. As with the large amount of resources the movement devotes to these consolidated corporate asks ensuring these are optimised is essential.
As Ren outlined we have a couple of recent, more detailed evaluations and we have found that the main limitations on our impact are factors only a minority of advisors in the animal space highlighted. These are constraints from other organisation stakeholders either upper management (when the campaigns team had updated on our findings but there was momentum behind another campaign) or funders (particular individual or smaller donors who are typicaly less research motivated than OPP, EAAWF, ACE etc.)
You can see this was the main concern for CE researchers in the original report. "Organizations in the animal space are increasingly aware of the importance of research, but often there are many factors to consider, including logistical ease, momentum, and donor interest. It is possible that this research would not be the determining factor in many cases".
From the main body of the text: "Plant-based products represented 0·011 % of product unit sales in the pre-intervention period. This increased to 0·016 % during the intervention period and 0·012 % in the post-intervention period. Meat products represented 26·52 % of sales in the pre-intervention period, 26·51 % during the intervention period and 26·32 % in the post-intervention period. The remainder of sales were represented by non-meat products (73·47 % in pre-intervention and intervention periods, 73·67 % in the post-intervention period)."
One thing to flag on reading into this as evidence against plant-based sales leading to lower meat consumption is how much harder it will be to detect a significant effect on meat consumption. The background variation on those is much higher so to detect a significant effect from a campaign like Veganuary we would need a much larger total effect size. Even if the 0.05% of pre/post change in plant-based came 100% from meat I'd expect it still would not be significant.
That seems like the 80/20 of this would be appropriate for a lot of candidates. I guess I assume that a lot of EA candidates have a higher bar for claims made in typical fundraising material so would benefit from delving deeper into the numbers. This depends on how much trust you already have in organisaions. Where if you think groups are already assessed with enough rigor by funders, e.g they have a GiveWell recommendation, then the time cost of going through the numbers makes less sense. I think this would work best for meta-groups like the organisaion I work for Animal Ask or others like Animal Advocacy Careers, Charity Entrepreneurship, 80,000 hours, Rethink Priorities, Global Priorities Insitute etc.
Hey Sofia, Great idea. Groups have usually indicated they would spend <10% of the time we spend researching without our involvement, so this seems like a more viable idea than one may expect. There are some reasons this may not entirely cross-apply to the rest of our work. Such as concerns with groups anchoring too much to their more shallow research, which usually results in more optimistic assessments (Optimizer's Curse). Or possibly a selection effect with the groups that are willing to do this being more likely to make better decisions. We are tracking the asks other similar organisations are using in the regions or areas we have worked in to. This gives us some sense of this, but a more direct experiment of this kind could be valuable. Particularly if we ran it with a few groups using different advocacy methods. We will look into the idea more as well as some of the other ways we could amend our pre/post surveys before we partner with the next group!
Hello Joel,
I agree that in hindsight a summary of each indicator would probably have been useful to provide the reader with an overall assessment given the information I reviewed in the report.
That model is roughly the way I was thinking of this assessment, with validity and interpersonal comparisons being how much I would update on a perfectly accurate measure, and reliability giving some sense how wide the confidence interval would be from a real world measurement. The trade off of these, between groups of indicators and individual indicators, adds some nuance so that a single physiological measure is reliable but can vary due to numerous other factors but a combination of them allows us to measure the welfare benefit of things that health can’t capture easily. For example, it may be better to minimise disease rates instead of blood glucose levels if given no context but disease rates would be unable to assess the importance of different types of environmental enrichment.
If more people comment to express interest in an overview of each section, I am happy to invest the time to go back through the report to add in these sections.
I definitely agree, which is partially why I put an example of self-reports in humans (which are in my opinion as close to ideal as we can get) alongside the measures we have available in other animals. This is what I currently view as the best available ('ideal') system given the weaker methods available.
In this context, many of these indicators struggle on cross species comparisons. Take cortisol for example, where different species have different cortisol levels, making it difficult to compare levels or even percentage changes across species. We can gain some sense of the relative importance of different improvements or events for an individual from the degree of change of an indicator. An example of this within an operant test could be a human showing a mild preference for social contact vs food compared to a fox who shows the opposite relationship. Yet, this still only gives us information within the range of their utility functions and doesn't tell us how their ranges compare. It's a challenging question and due to this we have mostly been deferring to Rethink Priorities’s work on moral weights and Open Philanthropy Project’s report on consciousness. At the moment, we approach this by using an assessment of an animal's quality of life, to gauge how important an improvement is within an individual's utility function, and then adjust this based on these considerations. However, I would be cautious about concluding that an ask is more promising if the deciding factors are based across species comparisons given the range of plausible views on the topic.
Thanks for the feedback