JP

Jacob_Peacock

708 karmaJoined Nov 2018

Comments
64

I've made some updates and corrections to this paper—(2) and (3) are most important in my opinion and make the Malan 2022 field experiment a somewhat weaker test of the PTC hypothesis. Thanks to all who commented!

Changes are noted below (which I've also added to the post):

  1. Correction to Figure 1 to indicate Ikea 2019 data represent all stores globally, rather than just the United States.

Corrections and updates in the "Malan 2022 field experiment" section:

  1. Replace "On Thursdays, students had the option of receiving prepared burritos with either Impossible ground beef, animal-based steak, or veggies, while the build-your-own entree line offered Impossible ground beef every day alongside animal-based ground beef." with "On Thursdays, students had the option of receiving prepared burritos with either Impossible ground beef, animal-based cubed steak, or veggies, while the build-your-own entree line offered Impossible ground beef every day alongside animal-based cubed steak, shredded beef and other animal products."

  2. Replace the second paragraph in this section. Briefly, this updates and corrects the text to indicate that a negligible portion of participants did pay for meals; an animal-based ground beef equivalent to the Impossible ground beef was not served; and dining hall staff served all food. Furthermore, it adds information on a survey of participants' perception of the taste of Impossible ground beef. The paragraph previously read:

    In this study, price is entirely equivalent since students pay for dining hall access for the entire semester, not individual meals. With regards to taste, Impossible ground beef specifically has not been subjected to any public taste tests. However, as reviewed above, the Impossible Burger, which is made of similar ingredients, has been found to taste equivalent in some studies. The study does not describe exactly the form of the beef in the steak burrito, making its taste equivalence less certain but probably still a reasonable inference. For the ground beef served on the build-your-own entree line, taste equivalence seems very likely. We can further surmise that the Impossible ground beef meals in the study were at least desirable: a follow-up survey found that 71% of purchasers were repeat purchasers (Malan, 2020, p. 189). Convenience is likely equivalent as well since the burritos are prepared for students by dining hall staff, and the build-your-own entree line is self-serve for both animal- and plant-based ground beef.

    It now reads:

    In this study, price is almost always equivalent since students pay for dining hall access for the entire semester, not individual meals; a negligible proportion of meals were purchased with other means of payment. With regards to taste, Impossible ground beef is not directly comparable to either the cubed steak or shredded beef served in the study, nor has it been subjected to any public taste tests comparing it with ground beef. However, as reviewed above, the Impossible Burger, which is made of similar ingredients, has been found to taste equivalent to ground beef burgers in some studies. Furthermore, Malan (2020) includes a survey on taste perceptions of 215 participants at the intervention site, although some of the question phrasing might evoke slight agreement or social desirability bias, and the survey may be affected by selection bias. Of the 96 surveyed participants who self-selected to try the Impossible ground beef, 86% agreed or somewhat agreed it was delicious; 85% that Impossible ground beef "is a satisfying alternative to animal meat" (Malan, 2020, Table 20); and 71% choose to eat it more than once (Malan, 2020, p. 189). Of the 89 open-ended responses describing what was liked about the Impossible ground beef, 52% mentioned the flavor, feel or texture and 30% its similarity to animal-based meat (Malan, 2020, Table 27). Conversely, of the 49 open-ended responses on what was disliked, 29% mentioned the flavor or feel and 24% the texture (Malan, 2020, Table 28). Given these results it may be reasonable to conclude the Impossible ground beef was fairly well-liked among those participants who consumed it, although it is not equivalent to the other beef products offered. Finally, convenience is likely equivalent since all meals are prepared for students by dining hall staff.

  3. Replace "Dining halls were not randomized to intervention and control status, and participants were free to cross over between dining halls during the study, both factors that could bias effects in either direction. Control dining hall A was adjacent to the intervention dining hall, so intervention materials were potentially visible, while control dining hall B was isolated from the intervention." with "Dining halls were not randomized to intervention and control status, and participants were known to cross over between dining halls (Malan et al., 2022, p. 226), both factors that could bias effects in either direction. Control dining hall A was adjacent to the intervention dining hall, and some intervention materials were thus visible (Malan, 2020, p. 119), while control dining hall B was isolated from the intervention."

Clarifications and a correction in the "Conclusions & recommendations" section:

  1. Clarify "The underlying premise of PTC as the key determinants of food choice is not supported by evidence from cross-sectional surveys on consumers' self-reported determinants."
  2. Correct "Finally, a controlled experiment introducing high-quality plant-based meat to a dining hall—at almost always equal price and convenience to animal-based meat—shows that most participants did not choose plant-based meat."
  3. Correction to replace "six" with "five" in "Across five lines of evidence, it is clear that the empirical evidence opposes the PTC hypothesis."

This source suggests the rate of self-identified veganism in Germany is about 3% in 2022. (We did not do any data collection ourselves; this report is a re-analysis of existing data collected by Brachem et al.)

they don’t discuss (let alone defend) “strong form PTC” theory.

I suppose we simply disagree here. The first quote I cite states "the products need to taste the same or better and cost the same or less." The next sentence strongly implies that "the market can kick in and take it from there, just shoot us up the S-curve," with "necessary but not sufficient" relegated to a "quibble." In conjunction with the Q&A, I think reasonable audience member would infer that your statements mean roughly "if price and taste parity were met, a majority of consumers would soon switch." Conversely, it's hard to imagine audience members construing "up the S-curve", "huge, huge dent" and "change the massive trajectory" to mean, for example, 20% of people switch over two decades.

And in the second case, since we’re at 1% plant-based meat right now and 0% cultivated meat, my statements that “we can have many times the penetration that we have right now if we can get to price and taste parity” and “if you can get to price and taste parity, you can make a huge, huge dent”: 1) don’t mean that nothing else is required; and also 2) don’t mean that we magically reach 50%+.

Can you clarify roughly what number you did intend "many times the penetration" and "huge, huge dent" to refer to here?

It feels curious to me that you continue to claim I believe something that I am telling you explicitly that I don’t believe; you are essentially saying “you believe this and you’re wrong,” and I’m saying “I agree that’s wrong, and I don’t believe it.”

I don't think you believe this given you're clearly saying you do not. Instead, as I wrote, "I’d contend that you (and GFI) have prominently promoted and supported the strong PTC hypothesis. Or, at the very least, made statements that reasonable people interpret to support the strong PTC hypothesis." The situation to me begins to resemble a motte-and-bailey fallacy, with the strong PTC hypothesis as the bailey and the weak as the motte.

With regard to your four specific critiques: I think the overwhelming evidence of the importance of taste and price (including in the three sections from your paper) are a strong response to specific critiques about specific studies. i.e., the overwhelming preponderance of the evidence indicates the importance of taste and price to food choice.

You're simply reasserting your disagreement and declining to engage the critiques, despite being asked multiple times now (1, 2). In fact, none of the studies you cited address all four of the issues, and studies simply repeating the issues do not make for overwhelming evidence. I don't follow your argument against "specific critiques about specific studies?" Presumably vague critiques of unspecified studies would be unhelpful. A third time I'd ask, are you able to address these critiques, especially in those studies that predate 2015, when you started claiming price and taste as the most important factors in food choice?

In the end, I think we need a both/and approach, but I think that alt proteins are the only approach that has a shot at slashing the global consumption of industrial animal meat.

This seems self-contradictory: why would you support another solution, if you think alternative proteins are "the only approach that has a shot"? By assumption, that other solution would not have a shot.

I look forward to your comments on my forthcoming work on other strategies to reduce meat usage. I'll let you have the last word here.

Thanks! My subsequent reply to Bruce might be helpful here—while Bruce doesn't defend the claim here, I do think he says things that strongly resemble it elsewhere.

Are you referring to the blind taste test?

Yes. The Sogari blind taste test is indeed affected by saltiness; it also includes an informed taste test similarly effected (but again finding Impossible and animal-based meat tied for first). There is a second blind taste test cited immediately thereafter (Chicken and Burger Alternatives, 2018), although salt levels were not reported.

Have you compared these foods yourself?

No, I haven't.

It seems really hard to draw conclusions about taste competitiveness of a meat substitute from this kind of n=1 study, beyond saying that you are in the same vague zone.

I agree, food is varied and such comparisons are hard—that's part of why I argue we should do more taste tests! Can you clarify what you mean by an N of 1 study, as usually this refers to a study with a single participants, but Sogari indeed had many participants. If you're suggesting comparison against multiple burgers, this gets a bit tricky since one has to decide which burger you actually want to be equivalent to, if that's your goal.

I disagree. Right now I think that plant-based meat substitutes have a reputation as tasting worse than meat largely because they actually taste worse.

Can you clarify what specifically you disagree with here? I don't think I especially disagree with anything you wrote that follows from here. Instead, I think it's indeed perception of taste that matters for the impact of PBM and we can likely best measure that perception with informed, rather than blind, taste tests. Overall, as I write, I think actually operationalizing a taste test to identify whether "taste competitiveness" is obtained is non-trivial. The literature so far neglects such operationalizations. What do you have in mind as an ideal experiment to conduct to measure taste competitiveness?

Hi Paul, thanks for checking the analysis so closely! (And apologies for the slow reply; I've been gathering some more information.)

But wouldn't Impossible be a comparison for ground beef, not for steak? Am I misunderstanding something here?

This is a good point and I've now confirmed with the authors that the steak was cubed, rather than minced or ground, so indeed not likely directly comparable to Impossible ground beef. I'll be making some updates to the paper accordingly. Thank you!

The build-your-own-entree bar offers shredded beef, which while also not the same, might be a more similar comparison. Unfortunately, I wasn't able to get more granular data at this time to test whether that was more readily displaced. Overall, despite these caveats on taste, lots of plant-based meat was still sold, so it was "good enough" in some sense, but there was still potentially little resulting displacement of beef (although maybe somewhat more of chicken).

I don't think it's really equivalent on taste

Yes, I'm not entirely certain Impossible meat is equivalent in taste to animal-based ground beef. However, I do find the evidence I cite in the second paragraph of this section somewhat compelling.

If I'm understanding you correctly then 22% of the people previously eating steak burritos switched to Impossible burritos

I'm not sure where you're getting this exact figure, but I don't put much credence in it. Instead, I'd refer to estimates in Fig 3, which range from 0.3 to 4.0 percentage points of beef displacement, after accounting for behavior at the control sites and/or spillover effects. That is compared to an 5.0 or 11.4 pp increase in Impossible meal sales, respectively.

Furthermore, it's important to keep in mind "the study employed several co-interventions designed to reduce meat consumption (Malan, 2020). These included environmental education, low carbon footprint labels on menus, and an advertising campaign to promote the new product, all of which have some evidence demonstrating their effectiveness." So the effect is likely not entirely attributable to the Impossible meat.

even if you did have taste equivalence here I wouldn't expect people's decisions to be perfectly informed by that fact

I agree and discuss this issue some in the Taste section. In short, this is part of why I think informed taste tests would be more relevant than blind: in naturalistic settings, it is possible that people would report not liking the taste of PBM even though it passes a blind taste test. So I think this accurately reflects what we should expect in practice.

in your first deployment

In this case, ~32% of (surveyed) participants answered yes to "Have you tried the Impossible™ burger, Beyond Meat™, or similar products anywhere other than [the intervention site]?" (Table 19) Note the study was conducted in the Fall of 2019, right after the summer Impossible launched in Burger King. Furthermore, the study covers 10 weeks and 71% of participants who ever selected the Impossible product at the intervention site went on to select it again. So I wouldn't think of this as a first deployment: many students were familiar with these sorts of products already and took multiple opportunities to try the product over two and a half months. (These figures are based on a (small) survey of 200 participants, 96 of whom self-reported ever selecting Impossible products at the intervention site.)

Hi Bruce, thank you for your reply. I'll focus on a few key disagreements here, although I'm happy to elaborate further if it's helpful.

Finally fifth: I'm not sure about your current thesis (the “strong-form” version of PTC). [...] In my opinion, these are very weak citations, and your inference based on them is not (I don’t think) tenable.

I'll address this first as I think it's trenchant to determine whether the hypothesis I work to refute is in fact held. I’d contend that you (and GFI) have, at times, prominently promoted and supported the strong PTC hypothesis. Or, at the very least, made statements that reasonable people interpret to support the strong PTC hypothesis. I don't agree with some of your objections to the sources I've already cited (for example, your statements in a short interview seem perfectly relevant), but I'm glad to cite more. Here's a quote from your talk in June 2023 at EAG London:

That is GFI's entire theory of change - the products need to taste the same or better and cost the same or less. Then you can quibble about whether that is necessary but not sufficient or whether the market can kick in and take it from there, just shoot us up the S-curve. But even if you think that is not sufficient, I would contend that that is absolutely necessary if we're going to change the massive trajectory through 2050. (9:07)

You specifically describe price and taste equivalence as GFI's "entire" theory of change. What you now are suggesting has been your view—the weak PTC hypothesis—you describe as a 'quibble' with the strong PTC hypothesis. Perhaps you would argue that by referencing the "massive trajectory through 2050," you mean future rather than current consumers; however, in the Q&A you say:

Q: Looking at plant-based meat sales, they’re flagging in recent years, so what gives you confidence that they can in fact replace animal meat? A: [...] There have been probably 15 studies of why people have not tried plant-based meat or why they tried it and stopped eating it and literally 100% of those studies come up with the first two are “we didn’t like them,” “they didn’t taste good enough,” or “they cost too much,” which validates our theory of change. If you can get to price and taste parity, you can make a huge, huge dent. [emphasis added] I do think some people are just going to want to eat meat and that’s where cultivated meat comes in, but I think we can have many times the penetration that we have right now if we can get to price and taste parity and what’s happening with plant-based meat is a validation of that, not a challenge to it. (41:10)

This directly refers to the preferences of current consumers and argues, if we satisfied those current preferences, there would be a "huge, huge dent" in meat consumption. I think a reasonable audience member would perceive your supporting a view that closely resembles the strong PTC hypothesis.

Of course, you can argue that in the most literal sense this doesn’t amount to an endorsement of the strong PTC hypothesis, and I'd acknowledge that you sometimes provide more careful caveats. However, audiences are (understandably) led to believe you're arguing the strong PTC hypothesis is true. For example, I think a lay audience member of your talks is likely to get the take home message of “factory farming is a vast problem and price & taste-competitive PBM is the solution.” Conversely, I think it’s unlikely they get something like the weak PT hypothesis: “If we had price & taste-competitive PBM and met some other important unspecified conditions, a meaningful minority of consumers would switch in the next 50 years.”

This is where perception becomes especially relevant: having talked to dozens of people over the years about this report, people immediately identify the strong PTC hypothesis with you and GFI. I maintain that the perceptions of 5 experienced advocates (myself, Jacy Anthis & Aidan Kankyoku, who I cite, and Abraham Rowe & Lizka Vaintrob, who have commented) are credible evidence that the strong PTC hypothesis is commonplace at least in the communities they inhabit, if not in the experience of other commenters. Furthermore, if people didn’t find the strong PTC hypothesis relevant to their experiences, the post wouldn’t likely have received 85 votes, ~two dozen comments and ~20 emails; presumably, people would get to the third sentence and stop reading if I was obviously arguing against a strawman.

I’ve indeed noticed the lack of clear position papers that actually stipulate precisely what you or GFI believe would happen if the stated goal of taste and price parity were reached. Furthermore, I’ve argued in this paper that even the condition of taste, price and convenience parity is insufficiently defined. I think we agree the conversation would benefit from such clarity.

First: I think our main disagreement is over how important taste and price are to food choices.

In this case, it would be helpful if you could address the four specific critiques I've made of the studies you adduced to support this point, especially critiques 1, 3 & 4 and with a focus on sources that pre-date when you first started making this claim in 2015 (at the latest).

Second: I don’t think you offer convincing evidence to the contrary. In attempting to cast doubt on the importance of taste and price to food choice, you discuss [...]

Unfortunately, this doesn't correctly capture my argument. I focus on refuting the PTC premise (that price, taste and convenience are the primary determinants of food choice) here and the following section. The studies you are referring to are intended as "Empirical tests of the PTC hypothesis." (Where “PTC hypothesis” refers to, “if plant-based meat is competitive with animal-based meat on PTC, the large majority of current consumers would replace animal-based meat with plant-based meat.”)

(I do think you make some important points—as well as some errors—in your response here. Happy to elaborate if helpful, but wanted to focus on the core disagreements for brevity.)

The reporter indicates that your solution to skyrocketing meat demand is to “integrate natural plant-based foods — based on whole proteins like lentils, nuts and soy — into the larger food landscape.” Is that right? If so, I’m curious about your support for this strategy: Is this different in some way from what has been tried over the past fifty years (and then some), even as meat demand has skyrocketed? Is there some new angle of this strategy that you’re excited about? And how do you see that scaling?

Not exactly right, no. I hope to provide a paper length description of my (still developing) views here at some point. In lieu of that, I'll say I think some of the interventions discussed in the various systematic reviews of meat reduction I cite, like defaults, labeling, classroom education, shifting social norms, and non-analog plant-based options may also have promise. Furthermore, I think more forceful, negative but targeted meat reduction campaigns may also have promise. Of course, it bears repeating that I also think plant-based meats and analog products have some promise as well.

Load more