Understanding Nutrition Research

Coming up on this mini episode of the Doctor's Pharmacy.

For the majority of people, 67% of people in these studies reported a calorie intake that was so low that not even an elderly, frail, bedridden woman could survive on it.

When it comes to looking at studies on nutrition, we need to ask the right questions. What type of study is it? Who funded it? What kinds of variables were and weren't accounted for? In this mini episode, Dr. Hyman sits down with functional medicine clinician and educator Chris Kresser, to help unpack what we need to consider when it comes to nutritional research.

When you hear a headline like this, and I'm sure people are gonna have heard this headline in the news that meat kills, it doesn't actually represent what the science says. So the consumer's confused, the media's confused, doctors get confused, and everybody's confused, but it's not so confusing. And you shed light on why it's not so confusing. So let's start with, at first, why we have problems with these studies and why we're so confused. And then we'll get into the real data on meat.

Okay.

Yeah.

So, I mean, as you pointed out, this study, along with many others like it, I mean, I could set my watch by how often these studies come out claiming to show that meat increases the risk of death. We see it all the time. My email box blows up, social media blows up. But you said, yeah, you said. And, and, you know, now at this point, I'm just linking to of articles I've written that has everything there. Not even bothering to write anymore, 'cause it's the same response every single time. And the response is, as you pointed out, when you do a large observational study, you're just showing two factors that are occurring together. You're not demonstrating a causal relationship that one factor is causing another factor. And there are many different examples of this. There's one blog I think the guy's name is Tyler. I can't remember his last name, but it's called Spurious Correlations. Yes. And he has basically collected a bunch of correlations that have nothing, clearly nothing to do with one another. Like, the margarine consumption is like 99.7% correlated with the divorce rate in Maine. Maybe it's connected because, you know, trans.

Fats may have- I may have gotten.

A divorce probably if my wife was feeding me margarine. Honestly, my mother used to give me.

Fleischmann's margarine when I was a kid, because that was the 70s and and Tang and Fleischmann's margarine were the future foods, right?

Right. So it's really tempting to assume that variables are causally related when they're not. And it's not a safe assumption, especially with the case of red meat, because of something called the healthy user bias or the unhealthy user bias, depending on how you look at it. So for decades, we've been told that red meat is not a healthy food. And so, People who on average in studies, if you're looking at the general population, people who tend to eat more red meat also tend to engage in more behaviors that are perceived as unhealthy. So they might smoke more, they might drink more, they eat fewer fruits and vegetables, they're not as physically active, they're not as well educated, they're lower income, which has nothing to do with their value as a person, but these are correlated with lower, you know, higher risk of death more strongly than any nutritional factor. And, you know, they probably haven't, their microbiome is probably not as healthy because they eat lots of processed and refined food. And researchers try to control for some of these confounding factors, but there's no way that they can ever control for all of them. Another huge problem with nutritional epidemiology is the way that data are collected. So most people are shocked when they learn how just how ridiculous this is. But they use these assessments called food frequency questionnaires or other what are known as memory based assessments or memory based measures.

Like, what did you have for lunch last Thursday?

What did you have for lunch last Thursday? How many servings of red meat did you eat four weeks ago, Mark? I mean, you and I think about.

Food probably more than six, five servings.

And you and I think about food more than probably just about anybody, you know?

I think about it yesterday, but that's about it.

Right, right. So, and, and, and to illustrate this, some researchers did an analysis of the nurses health data, which, which is what these new studies were based on. The Nurses Health Study, the Nurses Health.

Study, the meat kills.

It's not like it was a 10% increase in risk. We're gonna get into that. Yeah, we're gonna get into that. So they did an analysis of these memory based assessments and they found that the average, that for the majority of people, 67% of people in these studies reported a calorie intake that was so low that not even an elderly, frail, bedridden woman could survive on it. That's right. So these are obese, overweight people that were reporting a starvation level calorie intake. So that alone just throws out the validity of all of the rest of the data because it would skew protein intake, fat intake, carbohydrate intake, and intake of every other food and nutrient.

In other words, how do they have bad memories of their lying? I remember when I was in training, we were taught, okay, whatever people tell you they eat, double it. And whatever people tell you they exercise, cut in half.

That's right. People always report- They want to please you. They want to please you. They want to please you, or they want to do like, if they want to report what they think you want to hear, essentially.

That's what it is. And unless you're like weighing and measuring every single meal, every day and writing it down, then you're not gonna be able to report what you ate. I mean, I don't know how much I ate yesterday, whatever.

I'm like, I ate a little this, I had some of that, I took a few bites of this. if you had two plates and the only difference, you know, one had 100 or 200 more calories, visually you couldn't even tell the difference. So like you said, unless you're weighing and actually specifically measuring them. And even then there are challenges on how to do that best, you know, in a ward.

And Mayo Clinic wrote a big review showing how these types of assessments, these memory-based assessments or food frequency questionnaires really weren't valid and they were so undermining the quality of all the science they're based on. So, almost everything we hear about nutrition, almost everything is based on these type of studies which are fundamentally flawed. And there's a guy named John Ioannidis, you know, is a Stanford professor who loves to study studies. And he's like, 80% of these type of studies, we call them observational or population studies or comparison studies, they 80% get proven wrong. ultimately when they're subjected to randomized controlled trials, which is a true experiment, right?

Yeah. I mean, he's extremely critical of nutritional epidemiology. He's basically said that it's worthless in the way that it's constructed now. And it could be improved by better measurement techniques and using some new technologies.

It may be like people take pictures of their food and then it goes.

Into a computer AI system and it measures them. That would be great. And then they're testing that and that would help a lot. The other problem you also mentioned is in any field outside of nutritional epidemiology, an increase of risk of just 10% would be seen as completely significant, meaningless, that you have to noise. At least a doubling of risk, like a 100% increase or a twofold increase or more in order to be able to know that you're not just dealing with chance, you know, indistinguishable from chance.

And the guy who published the study, Walter Willett and others at this comes out of Harvard. These are standup guys, but they've spent their whole life committed to epidemiology. And they defend it tooth and nail. And I remember speaking to Ron Krauss, who's an experimental scientist who studies cholesterol and heart disease. And he's like, listen, those are helpful for generating hypothesis, but they don't prove anything. And these guys who run around saying that they're proving something are misleading people. And he said, you know, you need to see a change of at least, you know, two. And Walter Willett said to me once, you know, well, we found this with smoking, that smoking causes cancer, but the increased risk was 20 or 2000% 2 to 3000%. Not 10%.

Yeah.

So like, if it's 10% and not 1000 or 2000%, at least 100% It's just worth ignoring. Like the thing with eggs that came out recently, which were, oh, eggs are.

Terrible, but it's like- 13%, 8%. I mean, rarely in nutritional epidemiology do you see any effects, even over 50%, much less 100%. And usually it's more in like the 10 to 15% range, which is meaningless.

It's crazy. And you can also see on both sides of the aisle, you see studies that are epidemiology that show that meat's completely safe. in the same way and they're both kind of meaningless.

Yeah, I mean, so, you know, to be fair, it's really hard to do large randomized controlled trials in nutrition because you gotta, like you said, lock people in a metabolic ward and keep them there for 20 years because the effects, especially if we're looking at real outcomes, which rather than just endpoints like cholesterol, there is something, there's something called the Bradford Hill criteria, which are criteria that actually, I think, were created around the time that these smoking studies were done because they're like, look, we can't just do a randomized control trial with smoking. We can't wait around for, yeah, that's true.

Okay, you guys smoke for 20 years. You guys don't smoke, we'll see what happens.

So we need to figure out ways to better determine whether these correlations are actually meaningful and that there might be a causal relationship. And one of those ways is what is the effect size? You know, how much of an increase do you see? Another is, is it monotonic? Like, does it continually go up with an increased dose?

Is there a linear relationship?

Yeah, is there a dose-effect response? And there are many other criteria that you can use to kind of get closer to the idea that there's a causal relationship, but those are rarely applied in these kinds of studies.

We are bombarded with confusing information that can make deciding what to eat totally overwhelming. But with the right knowledge and resources, we can learn to decipher dietary studies to get to the real truth. Thanks for tuning in to this mini episode of the Doctor's Pharmacy. If you enjoyed this episode, please consider sharing it with a friend. Until next time.

Don't forget, there's a way to listen completely ad-free. With Hyman Plus on Apple Podcasts, you can enjoy every episode without any breaks. Just open Apple Podcasts and tap Try Free to start your seven-day free trial.