Or: how the Adventist Health Study-2 had a pretty good study design but was oversold in popular description, and then misrepresented its own results.

When I laid out my existing beliefs on veganism and nutrition I asked people for evidence to the contrary. By far the most promising thing people shared was the 7th Day Adventist Health Studies. I got very excited because the project promised something of a miracle in nutrition science: an approximate RCT. I read the paper that included vegan results, and while it’s still very good as far as nutrition studies go it’s well below what I was promised, and the summaries I read were misleading. It’s not a pseudo-RCT, and even if you take the data at face value (which you shouldn’t) it doesn’t show a vegan diet is superior to all others (as measured by lifespan). Vegan is at best tied with pescetarianism, and in certain niche cases (e.g. being a woman) it’s the worst choice besides unconstrained meat consumption. 

I’m going to try not to be too sarcastic about this, the study really is very good data by nutrition science standards, but I have a sour spot for medical papers that say “people” when they mean “men”, so probably something will leak out.  Also, please consider what the state of nutrition science must be to make me call something that made this mistake “very good”.

Background

The 7th Day Adventists are a fairly large Christian sect. For decades scientists have been recruiting huge cohorts to study their diet, and publishing a lot of papers.

The Adventists are a promising group to use to study nutrition for lots of reasons, but primarily because the Church discourages meat, smoking, and drinking. So you lose the worst confounders for health, and get a population of lifelong, culturally competent vegetarians, which is a pretty good deal. Total abstinence from meat isn’t technically required – you’re allowed to eat kosher meat – but it’s heavily discouraged. 7DA colleges only serve vegetarian meals, and church meals will typically be vegetarian.

Some popular descriptions say that rules vary between individual churches, which could give you an almost RCT effect. AFAICT this isn’t true, and the paper I read never claimed it was. Both the internet and my ex-Adventist friend say that individual churches within the US (where the study took place)  vary a little in recommendations, but most of the variety in diet is based on individual choice, not local church rules. 

I was really hoping this project could shed light on what happened to people who had medical difficulties with plant-exclusive diets in plant-exclusive food cultures, but they didn’t try, and from the abundance of meat eaters within 7DA I’d guess the answer is that they eat animal products. Nor is the study very informative about naively switching to a plant-exclusive diet, since most members grow up in the culture and will have been taught a reasonable plan-based diet without necessarily needing to consciously think about it. 

The Adventist Health Studies program has produced a lot of papers over the years. I focused on Vegetarian Dietary Patterns and Mortality in Adventist Health Study 2, because someone commented on my last post and pointed to that paper as addressing veganism in particular (other papers look like they only consider vegetarianism, although they might be using it as an umbrella term). I also read the AHS-2 cohort profile, but not any of the other papers, due to time constraints. It’s possible I will raise questions those papers would have answered, but there are at least 15 papers and, as I’ll talk about later, I wasn’t feeling hopeful. 

So let’s talk about that one paper. It breaks people down into 2 main categories, one of which has 4 subtypes. The big category is “nonvegetarian”, which they defined as eating meat more than once per week (48% of the sample). If you eat meat less than or equal to once a week you qualify for some category of vegetarian. More specifically:

• Vegan (8%): consumes any animal product <1/month.
• Lacto-ovo-vegetarian (29%): unlimited consumption of eggs and milk, meat < 1/month.
• Pesco-vegetarian (10%): unlimited fish, eggs, and milk, other meat < 1/month
• Semi-vegetarian (6%): unlimited eggs and milk, meat+fish combined > 1/month but no more than 1/week.

I’m already a little at lumping all four of those into one category, much less calling it vegetarian, but it will get worse. 

I wish we knew more about the specific diets of the nonvegetarians (although not enough to trawl through 20 papers hoping to find that data). Do they eat pork or shellfish, which violates Church requirements and would thus be a great marker for low conscientiousness? Are they eating Standard Shitty American Diet, or similar to the lacto-ovo-vegetarians but with meat 5x/month? 

Statistics

In this sample, eating lots of meat is clearly correlated with multiple activities known to impair health, like smoking and drinking. The paper attempts to back out those effects, but that’s impossible to do fully. You can back out 100 things and still miss the effect of traits that makes doing all those things more or less likely. That’s how you get results like “theater attendance reduces the risk of death even after controlling for a laundry list of confounders”.

The mortality results in this paper are controlled for: age, smoking, exercise, income, education level, marital status, alcohol consumption, and sleep.  In women they also adjusted for menopausal status and hormone therapy (vegans were ½ to ⅓ less likely to be on hormone therapy, compared to other groups).

What kinds of things does this leave out? Conscientiousness, religiosity (which has social implications), and overall concern for health (anything with enough of a cultural “health halo” will eventually show a correlation with long life, because it will be done more by people who care the most about health, and they’re likely also doing other things that help). It will also be confounded if poor health is the thing that causes people to consume more animal products. 

My statistician described their methods of backing out the confounding effects as “This isn’t what I would have done but it doesn’t seem unreasonable”, which easily puts this in the top 10% of medical papers I’ve asked him to evaluate. He didn’t scream in pain even a little bit. I can’t check their actual math without the raw data, but for purposes of this post I feel comfortable assuming they correctly adjusted for everything they listed.

Claims of vegetarianism’s benefits are greatly exaggerated

Note: most of these results don’t reach the level of statistical significance, but let’s ignore that for now. I’m also going to ignore concerns they failed to fully back out health effects with non-dietary causes, because even if they did, the data doesn’t support their own conclusions, much less the idea veganism is superior. 

The abstract says “Significant associations with vegetarian diets were detected for cardiovascular mortality, noncardiovascular noncancer mortality, renal mortality, and endocrine mortality. Associations in men were larger and more often significant than were those in women.”

Sure, if you count “unlimited fish” as vegetarian. 

But it’s much worse than that. If you look at men, you see veganism’s hazard ratio is tied with pescetarianism, followed by other forms of vegetarianism, and then nonvegetarian. Seems unfair to quote this as total vindication for veganism, although it is definitely a blow against unrestricted meat eating.

But then we get to that niche group, women. This data is a little noisy because women only made up ⅔ of the sample, but you can nonetheless get a faint hint that veganism is barely distinguishable from unrestricted omnivorism, and the diet correlated with the lowest death rate is pescetarianism, with lacto-ovo-vegetarianism and semi-vegetarianism somewhere in the middle.

And again, that’s treating the confidence intervals as minimal, when in reality they heavily overlap with each other and with the nonvegetarian death rate. 

The extra weird thing here is that in men veganism was most helpful against cardiac issues, whereas in women it appears to be actively harmful to cardiac health. Any benefit veganism has in women comes from the “other death” category, whereas in men the “other death” category is where it loses ground against pescetarianism. 

The paper describes this as “Effects were generally stronger and more significant in men than women”, which is a weird way to say “women and men had very different results”. 

Why the gender gap?

Could be any number of things. Maybe nonvegetarian women had healthier diets than men in the same category (they cite another paper that checked and said there were no “striking differences” between the sexes within a given category, but I’m not feeling very trusting right now).  Maybe nutritional intake has a bigger impact on women due to menstruation and pregnancy. Maybe women were more likely to use veganism as cover for an eating disorder. Maybe they did the math wrong.  

Fish seem pretty good though

If you’re going to conclude anything from these papers, it’s that fish are great. At least as good as veganism in men, and better in women. I’m more inclined to trust that result than I otherwise would be, because pescetarianism has less of a health halo around it, as witnessed by seafood eaters having roughly the same prevalence of bad habits like smoking and drinking, relative to lacto-ovo-vegetarians and semi-vegetarians. My ex-Adventist friend confirms that veganism is viewed more favorably than pescetarian or semi-vegetarian in the community, although not compared to vegetarianism, which surprised me. 

So the benefits of pescetarianism are less likely to be downstream of being the choice of people who care a lot about health, or are highly consceintious. I briefly got motivated to eat more fish until I remembered I count as semi-vegetarian by their standards (and I’m female), so the gains are small even if you take the results at face value.

Nutrition is still really complicated, the study still has a bunch of flaws, I wouldn’t update too much on this even if it didn’t agree with my existing beliefs. But this clearly undercuts veganism as the healthiest choice for women, and doesn’t really support it for men either.