Vitamin D Part 2: Good for more than just your bones?

[Kristin] (0:00 - 0:04)
They called vitamin D supplementation robustly negative. I kind of love that.

[Regina] (0:05 - 0:10)
Robustly negative. I've been on some dates that have been robustly negative.

[Kristin] (0:15 - 0:38)
Welcome to Normal Curves. This is a podcast for anyone who wants to learn about scientific studies and the statistics behind them. It's like a journal club, except we pick topics that are fun, relevant, and sometimes a little spicy.

We evaluate the evidence, and we also give you the tools that you need to evaluate scientific studies on your own. I'm Kristin Sainani. I'm a professor at Stanford University.

[Regina] (0:39 - 0:44)
And I'm Regina Nuzzo. I'm a professor at Gallaudet University and part-time lecturer at Stanford.

[Kristin] (0:45 - 0:50)
We are not medical doctors, we are PhDs, so nothing in this podcast should be construed as medical advice.

[Regina] (0:50 - 0:55)
Also, this podcast is separate from our day jobs at Stanford and Gallaudet University.

[Kristin] (0:55 - 1:10)
This is our second part of our two-parter on vitamin D. Part one scrutinized the claim that a large swath of the population is vitamin D deficient, and we focus largely on the effects of vitamin D on bones.

[Regina] (1:10 - 1:12)
Bones! Viagra for bones!

[Kristin]
Yeah, exactly.

[Regina]
I remember, boners.

[Kristin] (1:13 - 1:37)
For part two, today, we're going to look at the claim that upping your vitamin D level will make you healthier and happier because, Regina, there is a huge body of literature that claims to show that raising your vitamin D level confers multiple health benefits outside of the skeleton, such as preventing depression and cancer and boosting immunity. We're going to take a critical look at this body of evidence.

[Regina] (1:38 - 1:47)
I am really excited about this episode because I think it's going to get at the sorts of things I care about, depression, immune system, that's what I struggle with, personally.

[Kristin] (1:47 - 1:51)
Right, so you're taking vitamin D supplements because you think it might help with immunity and depression.

[Regina] (1:51 - 1:55)
That's what I hope. I admit I have a bias there.

[Kristin] (1:55 - 2:28)
That's okay. It's good that you acknowledge that bias. We are all human.

Regina, in the course of this episode, we're also going to get a bit of an epidemiology lesson because vitamin D provides a great example of how epidemiologic research typically evolves. Scientists start with clues from lab studies, from patterns of diseases. They then move to association studies and, finally, to randomized trials.

But let's go back for a minute now to the basic biology of vitamin D. Interestingly, vitamin D is misnamed. It's actually a hormone, not a vitamin.

[Regina] (2:28 - 2:33)
Not a vitamin, a hormone? You're kidding. It sounds so different when you say hormone.

[Kristin] (2:34 - 2:49)
I know, right? It sounds way more powerful when you call it a hormone. But this is why people think that vitamin D does things outside of bones.

It turns out that most cells in the body have vitamin D receptors. Even cells that have nothing to do with calcium absorption or both.

[Regina] (2:50 - 2:56)
And that leads to the idea, maybe it does more than just keep your bones hard.

[Kristin] (2:56 - 3:06)
Exactly. Vitamin D binds to the vitamin D receptor, and that's a protein that controls the expression of a bunch of genes, including genes involved in things like cell growth, immune function, inflammation.

[Regina] (3:07 - 3:12)
Cell growth might be related to cancer, immune function, fighting infections.

[Kristin] (3:12 - 3:17)
Exactly. And it turns out vitamin D does a bunch of exciting things in a petri dish, like killing cancer cells.

[Regina] (3:18 - 3:19)
I am not in a petri dish.

[Kristin] (3:20 - 3:36)
No, exactly. I mean, yeah, plenty of things kill stuff in a petri dish. Of course, that doesn't necessarily translate to humans.

The other mistake people make when looking at these kind of clues from laboratory studies is to say, well, just because it does something in the body, therefore, more must be better.

[Regina] (3:37 - 3:44)
Mmm, a little Viagra is good, but too much will send you to the emergency room. Ouch, yes. It hurts.

[Kristin] (3:45 - 3:58)
Exactly. I mean, another example, every cell in your body needs salt, sodium, right, to survive. But that does not mean you should eat as much salt as possible.

[Regina]
Are you sure? Because I love French fries. I know.

[Kristin]
Too much salt actually will kill you.

[Regina] (3:59 - 3:59)
Yeah.

[Kristin] (3:59 - 4:06)
Same with hormones, right? Women need estrogen, but too much estrogen is not better. Too much estrogen can lead to breast cancer.

[Regina] (4:07 - 4:19)
I remember learning about this in biology many, many years ago. Homeostasis, right? The Goldilocks effect, not too little, not too much, just right, is what you need.

[Kristin] (4:20 - 4:27)
Exactly. So, we can't extrapolate too much from these preclinical studies. They're good clues. They're a starting point, but ultimately, we have to study humans.

[Regina] (4:27 - 4:30)
And, Kristin, what are those studies in humans?

[Kristin] (4:30 - 4:40)
Well, interestingly, the earliest studies on vitamin D in humans that didn't have to do with bone, they actually didn't involve vitamin D at all. They didn't measure vitamin D.

[Regina] (4:40 - 4:42)
What did they measure then?

[Kristin] (4:42 - 4:59)
All right. So, these are what we call ecological studies. In an ecological study, we look at patterns of diseases in groups of people.

We don't look at individuals. And so, for vitamin D, scientists would compare disease rates in a sunny place like Florida to an unsunny place like New England.

[Regina] (4:59 - 5:02)
Where I grew up versus where you grew up.

[Kristin] (5:03 - 5:11)
Yes, because if a disease is less common in sunnier areas where people get more vitamin D, maybe that's a clue that vitamin D protects against that disease.

[Regina] (5:12 - 5:28)
Ecological study. I really want to point out the name of this because it's horrible. When I first learned about ecological studies in epidemiology years ago at Stanford, I was so confused.

Why did they call it an ecological study? Where are the jungles and the plants? I thought ecology.

[Kristin] (5:29 - 5:40)
This is not well named. We are not ecologists studying the environment.

I don't know why it's called that. An ecological study means we are studying groups of people rather than individuals.

[Regina] (5:40 - 5:44)
We have got to rename this. It's just poorly named. It's bad.

[Kristin] (5:44 - 5:56)
Right. Lots of things in epidemiology, I have to admit, are poorly named. But these studies are nice because they're easy, right? Governments collect data on disease rates in different regions and you can download them and look for these kinds of patterns.

They're easy to do.

[Regina] (5:57 - 6:01)
And these patterns might give us clues about the causes of disease. That's the idea.

[Kristin] (6:01 - 6:29)
So, let me just give a famous paper from 1980, Garland and Garland. This is actually the paper that kicked off the idea that vitamin D might be linked to cancer and it's still frequently cited as evidence of that link. All they did was look at rates of colon cancer deaths in different states and they showed that states that are less sunny, like those in the Northeast, Massachusetts, had more colon cancer deaths than sunnier southern states like Florida.

And they said, well, maybe that's due to the lower levels of vitamin D in the Northeast.

[Regina] (6:30 - 6:40)
It does make sense that people would look at that pattern and say, okay, cancer is related to latitude, latitude is related to sun, sun is related to vitamin D, so cancer is related to vitamin D. Boom.

[Kristin] (6:41 - 7:07)
Exactly. Here's the problem with that one, though.

If you look at recent data, the pattern is actually reversed. There are actually fewer colon cancer deaths in the Northeast than in the South currently. Hmm.

So, you have to be a little skeptical of these patterns because, you know, sometimes they're ethereal. And even if that pattern is real, that doesn't necessarily mean that it has anything to do with vitamin D. It's kind of a lot of steps to get to vitamin D.

[Regina] (7:07 - 7:39)
A lot of logical steps you had to do. I love to ask this question when I see surprising results. It's like a bit of a game.

What else could explain this pattern? And here, knowing nothing about this really, North and South really do differ on a lot of things besides just sunshine. There's a lot of cultural differences.

And if we are talking about colon cancer, I mean, I don't know the North very well. I grew up in the South. I picture you guys eating a lot of, like, Polish sausages, and that could be related to colon cancer.

[Kristin] (7:39 - 7:42)
Yeah. Something like, there's other explanations.

[Regina] (7:42 - 7:43)
Other explanations. Yes.

[Kristin] (7:43 - 7:44)
Besides the sun and vitamin D.

[Regina] (7:44 - 7:44)
Mm-hmm.

[Kristin] (7:45 - 7:49)
So that's why these ecological studies, you have to take them a bit with a grain of salt.

[Regina] (7:49 - 7:51)
A lot of salt. A big chunk of salt.

[Kristin] (7:51 - 8:01)
Yes. And Regina, there's some fun examples of the ridiculousness that can occur when you take the logic too far with these ecological studies.

[Regina] (8:01 - 8:01)
Right, right.

[Kristin] (8:01 - 8:17)
One of my favorites, teenage pregnancy rates. If you look at the United States, teenage pregnancy rates are higher in the South than in the North. You could look at it and say, oh, well, teenage pregnancy rates track with the sun and therefore with vitamin D, and therefore vitamin D causes teenage pregnancy.

[Regina] (8:19 - 8:21)
Well, of course it does. Just like having sex on the beach.

[Kristin] (8:22 - 8:25)
Right. Maybe it's confounded with sex on the beach.

[Regina] (8:25 - 8:33)
Right, right. Or maybe the fact in Florida that teenage premarital sex is just a hobby. Right?

It's like a state pastime.

[Kristin] (8:34 - 8:44)
Mm-hmm. The problem here, of course, with these ecological studies, correlation is not causation. We sometimes call that the ecological fallacy.

You see a pattern, and then you can make up pretty much any story you want to explain that pattern.

[Regina] (8:44 - 8:44)
Mm-hmm.

[Kristin] (8:44 - 8:47)
But presumably vitamin D does not cause teenage pregnancy.

[Regina] (8:47 - 8:49)
Right, right. I'm going to guess not.

[Kristin] (8:49 - 9:00)
Yeah. One of my favorite studies, I don't know if you remember this study, Regina, the one on chocolate consumption and Nobel Prize winners, it was Tongue-in-Cheek, satirical paper, New England Journal of Medicine, 2012.

[Regina] (9:00 - 9:03)
I love this one.

[Kristin]
Isn't it a great study?

[Regina]
Go ahead. Give us the details.

[Kristin] (9:03 - 9:18)
Yeah. It was not meant to be serious. It was totally tongue-in-cheek. But what they showed is that there is actually, this is real, a strong correlation.

Between a country's chocolate consumption and the number of Nobel Prize winners that it produces. And this is per capita, adjusted for population size.

[Regina] (9:19 - 9:36)
So, of course, we're going to take that to its logical conclusion and say that if we give out more chocolate in the country, then we're going to have more Nobel Prizes. Or maybe I need to eat more chocolate, and then I will have my own Nobel Prize. Right. Sure.

[Kristin] (9:37 - 10:13)
So, again, if correlation gets translated to causation, that's where you would go with this. And I love this. It's a beautiful satirical paper, because not only did they draw causal conclusions again tongue-in-cheek, they also quantified the effect.

So they did what a lot of ecological studies do. They draw a line to the data, and then they extrapolate, and they extrapolated that if you could up a country's chocolate consumption by almost a pound per person per year, that would generate one additional Nobel Prize. And then we can go further.

If you up everybody's chocolate consumption by two pounds per year, two more Nobel Prizes for your country.

[Regina] (10:14 - 10:17)
My stomach is feeling a little ill right now, I have to say.

[Kristin] (10:17 - 10:24)
Well, you're going to have to pick up my slack. I've mostly given up chocolate, Regina, so you're going to have to eat my two pounds as well. So that's four pounds for you, but it's your patriotic duty.

[Regina] (10:25 - 10:35)
My patriotic duty, instead of maybe the real explanation is that if your population is eating more chocolate, it's more developed, and it also has more university degrees.

[Kristin] (10:36 - 11:06)
Exactly. Yes. So there's an easy explanation for why these two things would go together.

And it's not that the flavanols in chocolate make you smarter, sadly. Funny thing, though, when this study came out, some scientists and some journalists didn't get the joke. There were actually some articles written up in the media where they took it as serious.

And I think if you actually read the paper, you can tell that he's being tongue in cheek. But they actually wrote it up and said, oh, this might be evidence that the flavanols in chocolate make you smarter.

[Regina] (11:08 - 11:14)
This is why journalists need to listen to this podcast, so they make sure that they're not taken in by these crazy things.

[Kristin] (11:15 - 12:09)
But it's a really good illustration of how easily people can get sucked into a story. So the problem with a lot of the studies on latitude gradients and vitamin D is they have been overinterpreted, a little bit like this chocolate study, if taken seriously, has been overinterpreted. You see people citing these as evidence that vitamin D reduces cancer risk or reduces asthma or whatever it might be.

To give an example, very first chapter of Michael Holick's book The Vitamin D Solution, he introduces us to a hypothetical 10-year-old girl growing up by the equator and a hypothetical 10-year-old girl growing up in the north, either in the U.S. or Europe. The girl growing up by the equator is illiterate and living in poverty, but he paints this picture where she is going to grow up living this blissful, disease-free life versus the girl in the north who is going to have cancer and heart disease and all these terrible things. It is a ridiculous picture based on over-extrapolation from ecological studies.

[Regina] (12:10 - 12:21)
Yeah, this does not sound right. I mean, maybe the girl by the equator would seem to have less disease just because she has less access to doctors who keep a lot of good medical records.

[Kristin] (12:21 - 12:30)
I mean, come on. If you die at 45, you are less likely to get cancer, true, or if you never see a doctor, it cannot be diagnosed.

[Regina] (12:31 - 13:05)
No, no, no, no. Okay, so to try to sum up this little section, it sounds like these ecological studies can give hints or clues, but the problem is that the studies are not very rigorous and yet people are making some pretty big fantastical stories out of them anyway. Yep.

So, it feels a little like me at the bar and the guy across the room, and he looks at me, and so right away, bam, I'm like, okay, he thinks I am hot, obviously, and it is happening. We are going home tonight. We're going to get married, right?

[Kristin] (13:06 - 13:12)
I do think, Regina, that all the boys looking at you across the bar do think that you're hot and do want to go home with you.

[Regina] (13:12 - 13:18)
Or it's just they're looking at me because behind me is the bathroom, and they've had too much beer to drink.

[Kristin] (13:18 - 13:20)
You might be over-interpreting.

[Regina] (13:20 - 13:28)
I might be over-interpreting. And so, that's a hypothesis-generating observation right there, and then I need to go do a follow-up study.

[Kristin] (13:28 - 13:32)
I will walk over to you and ask him his intent for you, and that will be our follow-up study.

[Regina] (13:32 - 13:36)
That is a follow-up study where we are doing individual studies, right?

[Kristin] (13:37 - 13:46)
Yes, exactly. We need to follow up with these studies. This is starting point, but we need to actually go and do studies in humans on individuals where we actually measure people's vitamin D, right?

[Regina] (13:46 - 13:46)
Rigorous, yes.

[Kristin] (13:46 - 13:58)
Regina, continuing with our epidemiology lesson, we move from ecological studies to observational studies where we measure the vitamin D levels in people's blood and then correlate that with diseases and conditions.

[Regina] (13:59 - 14:07)
So, studies on individual people, but we are only observing them. We're not giving them vitamin D pills to see what happens.

[Kristin] (14:07 - 15:11)
Right. For example, we might compare the vitamin D levels in people who have cancer with people who don't have cancer. Or we might measure people's vitamin D levels and then follow them over time to see if people who have lower levels of vitamin D go on to develop more cancer or more depression or to die faster.

And in thousands of studies, guess what? Scientists have found higher rates of all sorts of diseases and problems in people with lower vitamin D levels. Everything from cancer to depression to dementia and even things like menstrual dysfunction.

[Regina]
And that is how we get the hype and headlines around vitamin D.

[Kristin]
Tons of hype and headlines, and it makes it sound like vitamin D is a miracle hormone. But vitamin D is a particularly thorny variable to study.

It links to lots of health conditions, but that doesn't necessarily mean that it causes those health conditions. And I think it might help to illustrate this point by diving into an example. Regina, I want to talk about one of my favorite studies that I use in teaching that illustrates the fundamental issue with studying vitamin D.

[Regina] (15:12 - 15:37)
Oh, I cannot wait. But first, how about we take a short break?

Welcome back. We're looking today at the claim that upping your vitamin D makes you healthier in all kinds of ways. And Kristin, you were about to tell us about this vitamin D study that you like to use for teaching.

[Kristin] (15:37 - 16:25)
Yes, this is a really fun study for teaching. I'm going to start with some of the media coverage that the article got. This was from 2019.

The study linked higher vitamin D levels with better aerobic fitness. And I'm going to read a quote from an article by the health writer Amanda McMillan. She didn't actually interview the authors.

She is just summarizing from their paper, but their discussion section was quite long and I think she boiled it down nicely. She writes, quote, The study notes that vitamin D could potentially affect cardiorespiratory fitness in several ways. For starters, the nutrient has been shown to boost the production of muscle protein and aid in calcium and phosphorus transport on a cellular level.

It may also affect the body's makeup of fast-twitch muscle fibers, suggesting that vitamin D may improve aerobic fitness, the authors wrote.

[Regina] (16:25 - 16:55)
You know, it sounds very causal, Kristin, like vitamin D does something to your body to make you become more fit, right? But they're suggesting that you could just sit on your couch, pop some vitamin D pills and it's going to affect your, what, muscle fibers or fast-twitching cell thingies, right? And all of a sudden you're going to be able to run faster.

[Kristin]
Yeah, exactly.

[Regina]
And personally, I hope this is true. Tell us more about the actual study.

[Kristin] (16:55 - 17:09)
So let's talk about the study. This was a study in about 2,000 people and these people were all participants in this health survey that's done periodically in the U.S. So it's a good representative sample of people in the U.S. But they only looked at younger people, so ages 20 to 49.

[Regina] (17:10 - 17:11)
Too old. We don't qualify.

[Kristin] (17:11 - 17:23)
Well, we just missed that age cutoff, thank you, Regina. They measured their vitamin D with a blood test and they measured their VO2 max on a treadmill. Hopefully not at the same time.

[Regina] (17:24 - 17:26)
So what's involved in this VO2 max test?

[Kristin] (17:26 - 17:37)
You get on a treadmill, you run as hard as you can while wearing a mask and it's measuring how much oxygen your body can use during intense exercise. And it tells us how fit you are, how aerobically fit.

[Regina] (17:38 - 17:48)
That sounds like torture. First of all, with a mask and everything. VO2 max, so higher is better.

Higher is better, yes. And what did they find linking that to vitamin D?

[Kristin] (17:48 - 17:53)
So they found that there was an association between higher vitamin D levels and having a higher VO2 max.

[Regina] (17:54 - 17:55)
But how big was the effect?

[Kristin] (17:56 - 18:21)
They divided people into four groups based on their vitamin D levels. We call these quartiles. The low vitamin D group, the lowest quartile, had a VO2 max of about 37 on average.

And the high vitamin D group was about 41 on average. It was about a four-point jump, 4.3 to be precise in VO2 max. And I have to say, if I got four points on my Garmin in my VO2 max, then I do know that I'm getting fit.

So it's not a tiny effect.

[Regina] (18:21 - 18:35)
Okay. Good to know. So noticeable.

But what about all the ways that people who are high in vitamin D might just already be different than the people who are low in vitamin D, what we call confounders in stats jargon?

[Kristin] (18:36 - 18:52)
There are a lot of confounders here, Regina. The high vitamin D group was thinner on average than the low vitamin D group. They had better kidney function.

They had less diabetes, less hypertension. They are generally healthier. The authors did statistically adjust for these confounders.

And the effect didn't go away.

[Regina] (18:52 - 19:06)
It shrunk from a difference of 4.3 in VO2 max to a difference of 2.9. So it looks like the confounders are important because the effect got smaller when we accounted for those differences between the groups.

[Kristin] (19:06 - 19:41)
Right. But the effect remained statistically significant. So it didn't go away entirely.

And that's the key. The difference persisted even after the authors adjusted for confounding. So they're saying, hey, look, it's still there. So it might be causal.

Now, to be fair, they didn't come out and say, oh, this was definitely a causal relationship. But in their discussion section, all of these biological stories they're telling about calcium and phosphorus transport and muscle proteins, it's really, in essence, them explaining how you could sit on the couch, take a vitamin D pill and drop your 5K time.

[Regina] (19:41 - 19:54)
I'm still saying sign me for that one. But, Kristin, isn't there a simpler explanation for why vitamin D and VO2 max, the fitness level, would be correlated, would be associated?

[Kristin] (19:54 - 20:05)
You're on to it, Regina. This is why I love this in teaching, because to me, there is a huge elephant in the room. There is an obvious reason why vitamin D levels and fitness would be associated.

[Regina] (20:06 - 20:13)
So in all the things that they adjusted for statistically, did they control for differences in exercise between the two groups?

[Kristin] (20:13 - 20:26)
No, they did not have data on how much the groups exercise. So they controlled for age, race, sex, smoking, obesity, and comorbidities, but they didn't have data on exercise. So they did not account for differences in exercise.

[Regina] (20:27 - 20:42)
So we're talking about VO2 max, which has to do with athletics and sports performance and exercise and fitness, and exercise is tied both to VO2 max and vitamin D, and they're just completely ignoring that.

[Kristin] (20:42 - 21:24)
Strongly. The thing is, the elephant in the room, right? There's an obvious, you know, Occam's razor, obvious, simple explanation here.

VO2 max is very, very strongly influenced by how much you are exercising. Where does a lot of exercise take place? Outdoors in the sun, right?

I mean, of course, people sometimes exercise in a gym or a basement, but lots of people who are running, hiking, swimming, biking, all of those are taking place outdoors. So yes, you're going to have a higher VO2 max if you're outdoors exercising, and guess what? You get a lot more sun exposure, and you're going to have a higher vitamin D as well.

I 100% believe that there is an association between higher vitamin D and higher fitness. I believe that it's absolutely a real association, but I don't believe their explanation.

[Regina] (21:25 - 21:39)
Yeah, no, it's that causal thing, right? What is that simplest explanation? It's not your fast twitch muscles with the phosphorus, blah, blah, blah.

It's this confounder, and in this case, it's an unmeasured confounder.

[Kristin] (21:39 - 22:10)
They don't have any data on it, so they're not correcting for this in any way. And I think it's obvious that this is just due to people who go outside and exercise, have good VO2 maxes, and they have good vitamin Ds. But I love this for teaching because it illustrates the point that it's so easy to get sucked into some crazy biological story and to miss the obvious explanation, right?

We do this in a lot of cases, I think, with vitamin D. Can we pause for a minute, though, Regina? Can I make fun of a press release?

Will I offend you?

[Regina] (22:10 - 22:15)
You know, I used to write press releases for a journal, so I'll give you permission as long as it's not one of mine.

[Kristin] (22:15 - 22:23)
It is not one of yours. And, to be fair, most press releases, including the one I'm going to make fun of, don't have a byline, so I'm not making fun of a person.

[Regina] (22:23 - 22:27)
Okay. So we're going to enjoy it in a loving, humorous way.

[Kristin] (22:28 - 22:59)
That's a nicer way than say make fun of. Okay, yeah, we're enjoying it in a humorous way, yes. I will point out, the fact that the journal, when this article came out, the fact that they put out a press release, that's very interesting to me because it tells me that they thought there was something novel and exciting here.

They certainly would not have put out a press release if the study was, look, people who go outdoors and exercise, they have a good VO2 max and they have a good vitamin D. That's not interesting. They put out the press release because there was this underlying implication of like, ooh, maybe vitamin D causes you to be more fit.

[Regina] (22:59 - 23:02)
The hint and the promise of take some vitamin D.

[Kristin] (23:02 - 23:40)
That's why it even got a press release. But back to the humorous part, remember the average difference between the high and low vitamin D groups on VO2 max, it started at 4.3 and I gave the exact number for a reason. And after we statistically adjusted, it went down to 2.9. Remember those numbers. Those are differences in averages in VO2 max between the groups. Right, between the groups. But here's how it was reported in the press release.

This is a quote. The participants in the top quartile of vitamin D had a 4.3-fold higher cardiorespiratory fitness than those at the bottom quartile. The link remains significant with a 2.9-fold strength after adjusting for glucose.

[Regina] (23:40 - 23:47)
They completely missed the point here, they lost the whole plot. So they said 4.3-fold higher.

[Kristin] (23:48 - 24:02)
They're interpreting a difference in averages or means as a relative risk. But obviously, what would a 4.3-fold higher mean? Well, if we started at 37 VO2 max in that lower group, 4.3-fold higher would be?

[Regina] (24:02 - 24:11)
Well, 4 times 37 is around 150. So let's just say around 150. Right.

And you can't have a VO2 max of 150, I'll just tell you that right now.

[Kristin] (24:12 - 24:20)
The highest ever recorded, fun fact, Norwegian cyclist, Oskar Svendsen, his was 97.5. That's the highest we've ever seen.

[Regina] (24:20 - 24:23)
So you're like 95.5. I am not.

[Kristin] (24:24 - 24:43)
I am not nearly that high. Maybe at one time in my life, I was higher than I am now, but not up in that range. Interesting little aside, as long as we're talking about VO2 max here, have you heard of the Breaking2 project from Nike, Regina?

I have not. Nike did this whole project where the goal was to get someone to break two hours in the marathon.

[Regina] (24:44 - 24:49)
Oh, that was like a big, like publicity, kind of scientific goal.

[Kristin] (24:49 - 24:53)
Yeah, a publicity stunt, but also really cool. It is cool. Breaking two hours in the marathon is freaking fast.

[Regina] (24:53 - 24:56)
It is. So what would that be? Like, first of all, did they make it?

[Kristin] (24:56 - 25:14)
They did make it. So Kipchoge actually did it. He did 1:59 and change.

[Regina]
Just barely.

[Kristin]
Yes, but he did it. And it took two tries, but they did get it.

That's a 4.34 per mile pace. 4.34. I have never even run one mile that fast.

[Regina]
A mile in four minutes and 34 seconds?

[Kristin]
But 26.2 of them in a row.

[Regina] (25:14 - 25:16)
Oh my gosh. There's no way.

[Kristin] (25:16 - 26:01)
It wasn't an official world record because it was this controlled situation. They had pacers. It wasn't an open competition.

They delivered them water, really trying to optimize so that somebody could break two hours, which they did. But there was a lot of science behind this. And the fun part is that they had 16 athletes they started with.

Kipchoge is the one who actually broke two hours. But they did measurements on all 16 athletes and published the data. They measured VO2 peak.

Technically that's slightly different than VO2 max, but we're going to call it close enough. Reference point here. And what were they?

1. These were all male athletes.

[Regina]
Of course.

[Kristin]
For now. Yeah. For now.

And those ranged from 62 to 84, just to give you a sense of how high VO2 max can get. These are very elite runners.

[Regina] (26:02 - 26:03)
So you're not getting 150. That's impossible.

[Kristin] (26:03 - 26:12)
You're not getting 150. Getting back to the question. We don't know, though, if it was Kipchoge who was the 84 because they didn't tell us, you know, individual athletes.

So he's somewhere in that range.

[Regina] (26:13 - 26:23)
So I wonder how this got through, because when I wrote press releases for a journal, we had to send the press release to the authors to look over for facts checking in.

[Kristin] (26:23 - 26:29)
Hopefully that didn't happen here, because I would hope that this is such an obvious error that the authors, had they had the opportunity, would have caught it. Let's hope.

[Regina] (26:30 - 26:55)
Yeah. Yeah. All right.

Let's take stock and kind of see where we are on this whole epidemiological journey that you've been taking us on. We started out in a Petri dish. Realize we don't live in Petri dishes.

Ecological studies, not about jungles, groups of people, lots of problems with that because it's groups, not individuals. So then we moved on to individuals, talked about observational studies, but we have all these confounders.

[Kristin] (26:56 - 27:08)
And we do have statistical methods for trying to deal with confounding. They just don't work that well in the case of vitamin D, because vitamin D is in many ways a marker of good health.

[Regina] (27:08 - 27:18)
That makes sense, because if you're healthy, you're active, you're getting outside, your vitamin D is higher, you're health conscious, you're taking a multivitamin, your vitamin D is higher.

[Kristin] (27:18 - 27:32)
That's right. On the other hand, if you are a frail old person who is not getting outside and maybe malnourished, your vitamin D is going to be lower. And if you are ill, obese, smoke, have poor kidney function, all of these things can also directly lower your vitamin D.

[Regina] (27:32 - 27:43)
So if you look for association between vitamin D and pretty much any health condition you're going to find something there. You're going to find an association, but it doesn't mean it's the vitamin D that's causing you to be healthier.

[Kristin] (27:44 - 27:57)
Exactly. And Regina, it might be helpful for me to bring in another variable where I think this problem is a little bit easier to see. We both review for a journal that's about physical medicine and rehabilitation, so you are probably familiar, Regina, with the variable grip strength.

[Regina] (27:57 - 28:01)
Yeah. That's how strong you can squeeze with your hand. They love this variable and that discipline.

[Kristin] (28:02 - 28:18)
Yeah. And I think grip strength illustrates the problem well, because not surprisingly, your grip strength correlates with a lot of other health outcomes. So if you have a better grip strength, you are less likely to get cancer or heart disease or diabetes or infections or psychiatric illness, and you will also live longer on average.

[Regina] (28:19 - 28:34)
So it's not that grip strength, though, is preventing the disease, it's a reflection of good health, like you're saying. So if I were to improve my grip strength, you know, just sitting on the couch using those little, you know, hand springing things, that's not going to prevent cancer.

[Kristin] (28:34 - 29:23)
Right. We're looking at basically reverse causality here, right? The fact that you are in good health means that you have better grip strength, or the fact that you are in good health means that you have a higher vitamin D level.

Right. And it's not that the grip strength of the vitamin D itself is necessarily causing you to be healthier. Now, of course, researchers can use statistical models, statistical adjustment, these regression models we have, to try to isolate the effect of vitamin D alone, independent of all these confounders that correlate with your vitamin D level, the obesity, the age, frailty, exercise, diet, and so on.

But statistical adjustment is not all powerful, believe it or not. Statistical adjustment is actually something we're going to talk a lot about in this podcast, and I think vitamin D makes a good case in point of the limitations of statistical adjustment.

[Regina] (29:24 - 29:30)
Yeah, it's hard to tease apart. We can do a little bit of this with math, but it just gets so messy.

[Kristin] (29:30 - 29:42)
Right. I have this picture in my head, if you bear with me here, Regina.

[Regina]
Oh, no.

Yeah, okay. Imagine you're coming up on this big ball of string. Okay.

[Regina] (29:42 - 29:43)
There's no sex here.

[Kristin] (29:44 - 29:51)
I never can come up with good sex analogies. I'm going to have to rely on you for that.

[Regina]
I'll work on it.

[Kristin]
PG. Actually, G-rated.

[Regina] (29:51 - 29:53)
G-rated this time.

[Kristin]
Completely G-rated.

[Regina]
Okay.

[Kristin] (29:53 - 31:43)
But imagine that in this ball of string, we've got these little cards, and one of the cards says vitamin D, and other cards will say things like obesity and smoking, and vitamin D, that card is tied through string to all of the things that directly influence your vitamin D level. So that's going to be obesity. People who are obese tend to have lower vitamin D.

Smoking, people who smoke tend to have lower vitamin D. Again, your physical activity outdoors, your kidney function. Kidney function impacts vitamin D.

Your age, your frailty, whether you take a multivitamin. So a huge number of health factors, probably hundreds of health factors tied to vitamin D. All of these directly affect your vitamin D level.

Each one of those health factors is then themselves tied to a whole bunch of health outcomes, a whole bunch of diseases. So obesity is tied to heart disease. Your level of outdoor exercise is tied to your VO2 max.

Your level of outdoor exercise is tied to your cancer risk and to your mortality risk.

[Regina]
They have all these little strings attached to everything.

[Kristin]
So there's a huge number of conditions that are tied to all of these health factors that are then themselves tied to vitamin D.

And if vitamin D itself affects any of those things directly, like if vitamin D actually gets into your cells and phosphorus transport, blah, blah, blah, and affects physical fitness, then there would be a string also tied directly between vitamin D and physical fitness. Now, the problem is you come upon this ball of string, hundreds or thousands of cards and strings, and it's all in this big mess. And if you kind of squint your eyes, you might go, yeah, well, vitamin D is in the ball and physical fitness is in the ball, so clearly they must be tied together.

But whether or not there's actually a string, a causal relationship tying them together, that turns out to be really hard to tease out, right? We can try to do it with statistical adjustment, with math, but our math is really not that powerful.

[Regina] (31:44 - 31:52)
No, it's not that powerful. So vitamin D is linked with a lot of different things. So vitamin D is very promiscuous.

[Kristin] (31:53 - 31:55)
It's a promiscuous variable, yes.

[Regina] (31:55 - 32:00)
So it's always going out on dates.

[Kristin] (32:00 - 32:02)
Thank you for bringing it back to the sex, Regina.

[Regina] (32:02 - 32:12)
I am. I am. I'm picturing vitamin D as a promiscuous dude for some, I don't know why a dude, but okay, we're going to go with it, promiscuous dude.

And he is hooking up with everyone.

[Kristin] (32:14 - 32:16)
So untangling that sexual network really hard.

[Regina] (32:16 - 32:29)
Right, right. And everyone else is hooked up with everyone else. And we really want to know, like, you know, who does vitamin D really care about the most?

And we can't because everything is all right.

[Kristin] (32:29 - 32:41)
We try to do this statistical jujitsu to like isolate these individual strings. But the fact is it's just really hard to isolate one string out of this big ball, this big mess. It's hard to do that statistically.

[Regina] (32:41 - 32:43)
One sexual relationship.

[Kristin] (32:43 - 32:44)
Right, exactly. That's a whole mess.

[Regina] (32:45 - 32:47)
Okay, bring it back to that. Yeah. I'm not saying that.

[Kristin] (32:47 - 33:05)
I mean, sometimes we're going to talk a lot in this podcast about statistical adjustment and the limitations of that. Sometimes we have a better shot at untangling the string in some situations. Rock stars and hearing loss.

I don't know why. I'm trying to come up with one that's kind of more simple.

[Regina] (33:06 - 33:14)
As someone with hearing loss, but not, I'm not a rock star, but I do have hearing loss. I'm very interested in where this is going to go.

[Kristin] (33:14 - 33:17)
Right. But the more important question, yeah, have you dated a rock star?

[Regina] (33:19 - 33:30)
I had to think about it. What did I tell you about my dating history that I couldn't remember? I have not.

I've dated, no, I've dated drummers, but they weren't exactly rock stars.

And you?

[Kristin] (33:30 - 33:33)
I did date a minor rock star once.

[Regina] (33:33 - 33:34)
You did?

[Kristin] (33:34 - 33:34)
Yes, I did. Yes.

[Regina] (33:34 - 33:38)
No way. You have had a way more interesting dating life than I have.

[Kristin] (33:38 - 33:40)
Many, many years ago. Guitarist.

Yeah, he was very cute.

[Regina] (33:41 - 33:41)
Was he good?

[Kristin] (33:41 - 33:42)
He had long hair.

[Regina] (33:43 - 33:43)
Long hair.

[Kristin] (33:45 - 33:46)
Long hair and very cute.

Anyway, getting back to...

[Regina] (33:47 - 33:49)
Were you caught in reminiscence there for a moment?

[Kristin] (33:49 - 33:58)
For a moment. Yes, yes. My youthful days.

So anyway, rock star, hearing loss. So if you imagine trying to like piece out the causal structure in that one.

[Regina] (33:58 - 33:59)
Right, right.

[Kristin] (33:59 - 34:14)
With our little ball of string analogy. Rock stars might be tied to a bunch of things that do affect health, right? Not always.

I don't want to stereotype here, but maybe more common on average, like more alcohol than average, or more drugs, or more sex, or less sleep, or.

[Regina] (34:14 - 34:15)
Better lover.

[Kristin] (34:16 - 34:17)
Maybe. No comment.

[Regina] (34:18 - 34:20)
I don't have direct knowledge, so maybe you would like to share.

[Kristin] (34:20 - 34:22)
No comment. Could be, could be.

[Regina] (34:22 - 34:26)
Better in bed. Could be. It's a little hot in here right now, isn't it?

[Kristin] (34:29 - 34:29)
Anyway.

[Regina] (34:30 - 34:32)
They are promiscuous.

[Kristin] (34:32 - 34:46)
Could be. Could be, right? So you might imagine this little ball of string, and rock stars are, yeah.

[Regina]
But hearing loss.

[Kristin]
With hearing loss, we know is very strongly related to the amount of sound that rock stars are exposed to.

[Regina] (34:46 - 34:46)
Right.

[Kristin] (34:46 - 35:12)
So we have a pretty strong causal relationship there. And all these other strings that tie to alcohol, and drugs, and sleep, those aren't very tied to hearing loss, right? Maybe there's some small influence there, but not strong relationship.

So that might be a ball of string that we had, you know, a little bit better of a job.

[Regina]
We can separate it out a little bit more.

[Kristin]
Easier to separate with a statistical adjustment model, we could probably do a pretty good job.

So not all statistical adjustment is a lost cause.

[Regina] (35:13 - 35:18)
I think it's good to remember that. Yeah. And to really point out why it's especially difficult with vitamin D.

[Kristin] (35:18 - 35:23)
It's just especially difficult with vitamin D, I think. It's just a particularly promiscuous variable.

[Regina] (35:23 - 35:26)
I know. I like it. More mixed up in the string.

[Kristin] (35:26 - 36:17)
It's very hard to untangle. So my fear with the observational studies, and that's why I like that physical fitness one, because I think there's an obvious unmeasured confounder there that everybody can say, oh, yeah, I can see that. But, you know, it's easier to tell a story when it's cancer or heart disease.

We start coming up with these beautiful biological stories, you know, and kills cancer in a Petri dish and latitude studies, and it starts to sound really good. But the more obvious explanation in those cases might just be that people who are healthier have higher vitamin D levels, right? And it's really hard to tease out any small effect that vitamin D might have on cancer itself, right, if it has any.

This is why we need the next step in our little epidemiology lesson, right? We need randomized trials where we can actually control the variable and we can untangle it through study design rather than after the fact through math.

[Regina] (36:18 - 36:23)
To be able to control for all these elements in the room, the unmeasured confounders, all of that.

[Kristin] (36:23 - 36:31)
We can randomize people. And when we randomize people to take a vitamin D pill or a placebo pill, all of a sudden we break all of those, we cut all the strings.

[Regina] (36:32 - 36:41)
We cut all the strings. Oh, I like it. We can isolate.

[Kristin]
Yes, exactly.

[Regina]
Isolate our promiscuous dude in there. We have a course about clinical trials.

[Kristin] (36:41 - 36:49)
We do have a whole course on randomized clinical trials on Stanford Online, which highly recommended if you want to get really deep on how to do a clinical trial.

[Regina] (36:50 - 37:02)
We do some good storytelling in there to talk about what to do and what not to do. Because well done, randomized experiments, trials are like the gold standard. But poorly done.

[Kristin] (37:02 - 37:05)
They're not all well done. Take home message of our course.

[Regina] (37:06 - 37:08)
Yes, it is. And we talk about a lot of examples.

[Kristin] (37:08 - 37:38)
Right. And we talk about how to do them well. So if you are at all involved in clinical trials, highly recommended to take that course.

But vitamin D, one of the great things about vitamin D is this health factor that's very promiscuous, hard to study observational studies, even though we have tens of thousands of observational studies that have been done. I question kind of the whole ball of them. But the wonderful thing about vitamin D is it is easier than many things to study in a randomized control trial.

It is an easy variable to manipulate. And that's often not the case with things we want to study.

[Regina] (37:38 - 37:43)
We can assign it to people. It's hard to assign people, you are a rock star and you are not a rock star now.

[Kristin] (37:43 - 38:42)
That one's a hard one to randomize. Or, you know, for smoking, you can't assign people to smoke or not smoke. Or even like

[Regina]
Eating kale.

You have to eat your vegetables.

[Kristin]
You could do a randomized trial. But how many people are actually going to eat the kale?

Vitamin D is a nice one because we have an easy to manipulate variable. I can raise somebody's vitamin D level very easily. I can give them a vitamin D supplement.

Participants in randomized trials, it's an easy one to do. You don't have to eat kale. You don't have to exercise.

You just pop a pill every day. Much easier.

[Regina]
And you can check to see whether they've done it.

[Kristin]
And they have in good randomized trials. They randomize people to take a vitamin D pill or take a placebo pill. So it's blinded too.

You can easily blind. That's another nice thing about vitamin D. But then you can check and you can see, did their vitamin D levels go up?

And in these randomized trials, they're good volunteers. Their vitamin D levels go up. So we can isolate that vitamin D from the big ball of string and see, hey, if I sit on the couch and pop a vitamin D pill, does my physical fitness actually get better?

We can isolate that.

[Regina] (38:42 - 38:49)
So why do we even do any observational studies to begin with? Why don't we just jump?

[Kristin] (38:49 - 38:54)
Well, I would argue we've done too many vitamin D.

We probably should have gotten to the randomized trials faster. But of course, observational studies are easier to do.

[Regina] (38:54 - 38:55)
They are easier. They're cheaper.

[Kristin] (38:55 - 39:23)
They're cheaper. And we often piggyback on an observational study. So you might have had a cohort study designed for one thing.

And then you say, oh, as long as we've got that set up, I can easily measure people's vitamin D. Or maybe I had their blood stored and I get the vitamin D. Often, it's just easier, more accessible, easier to do, even the prospective cohort studies.

A randomized trial, you've got to blind people. You have to do very good protocol. Everything has to be well-planned ahead of time.

It's just harder and more expensive to do.

[Regina] (39:24 - 39:27)
The observational studies are good because they give you clues.

[Kristin] (39:27 - 39:28)
They give you clues.

[Regina] (39:28 - 39:40)
We keep talking about every step along this epidemiological journey gives you more clues. But maybe the problem you're suggesting is that we got stuck in this observational study for too long.

[Kristin] (39:40 - 39:50)
For too long with vitamin D, because vitamin D is just one that's a really hard nut to crack with any observational study. So we really need the randomized trials to definitively answer this question about causality.

[Regina] (39:50 - 39:55)
And the hype started probably when we were still in that observational study.

[Kristin] (39:55 - 40:24)
Back in 1980. Maybe not that soon, but certainly in the last two decades, there's been a lot of hype. And we talked about where that hype came from.

That study about physical fitness and vitamin D would not have gotten headlines other than the implication that maybe you can sit on the couch and have vitamin D and then run faster.

[Regina]
Causality.

[Kristin]
It's exciting for people. They like it. But there have been.

[Regina]
Okay, so there have been some good randomized trials.

[Kristin]
There's been some bad ones, too, but there have been some long term, well-designed, large randomized trials that I think we need to talk about now.

[Regina] (40:24 - 40:53)
Okay, so let's take a break and then we'll come back to the gold standard.

Welcome back to Normal Curves. Before the break, we were talking about whether upping your vitamin D can make you just overall healthier and happier.

And Kristin, you were talking about some experiments that have been done, some randomized trials. And you said they've already been done on vitamin D.

[Kristin] (40:53 - 41:09)
There have been some randomized trials already done on vitamin D. I'm going to start by talking about the biggest and longest trial, and in my opinion, the most important trial that's been done to date. And that was the VITAL trial.

And it was run out of Harvard. It has a factorial design. I love factorial designs.

[Regina] (41:10 - 41:12)
Let's talk a little bit about what's involved in them.

[Kristin] (41:12 - 41:23)
Right. That means that people get randomized twice. So in this case, you first got randomized to get a vitamin D pill or a placebo pill, and then you got randomized to get omega-3 fatty acids or a different placebo pill.

[Regina] (41:23 - 41:35)
So you could get both treatments, a vitamin D pill and an omega fatty acid pill, or you could get one treatment pill and one placebo pill, or you could get two placebo pills.

[Kristin] (41:36 - 41:49)
Exactly. It's two trials for the price of one. Now, one key caveat, Regina, we are assuming that omega-3 fatty acids and vitamin D have completely independent effects. They don't interact with each other in any way.

That's a key assumption, but it's reasonable here.

[Regina] (41:49 - 41:55)
We talk about factorial design and clinical trials in our course, our Stanford online course.

[Kristin] (41:55 - 41:58)
Yeah, it's a super important design because you can get this freebie, this free trial.

[Regina] (41:59 - 42:02)
OK, so how much vitamin D did they get, the active treatment group?

[Kristin] (42:02 - 42:05)
They got 2,000 IUs per day of vitamin D.

[Regina] (42:05 - 42:11)
That's about double what I take right now. That's what I remember, about 1,000.

[Kristin] (42:11 - 42:44)
Now, Regina, question for you. When you were at 92 nanograms per milliliter, were you taking more than 1,000 IUs per day?

[Regina]
You know, looking back, I was probably popping them like candy.

[Kristin]
I imagine. All right, the other interesting thing about the VITAL trial, which I really love, is they did not give them calcium. And in a lot of randomized trials, because vitamin D and calcium often come in the same pill, they often give them both.

And when you find an effect, you don't know if it's due to the calcium or the vitamin D or both.

[Regina] (42:45 - 42:49)
People change one variable at a time. It's not that hard.

[Kristin] (42:49 - 42:54)
Right. My kids learn that in basic elementary school science. One variable at a time, please.

[Regina] (42:54 - 43:00)
OK, 2,000 IU vitamin D and calcium. And who was taking this? Who was in the study?

[Kristin] (43:01 - 43:18)
There were about 25,000 people and that included 5,000 Black participants. And this is important because people who have more melanin they can't absorb as much UV light so they don't make as much vitamin D. So that is a group that's potentially at risk for having low vitamin D.

[Regina] (43:18 - 43:24)
So they oversampled that group so they would have a big enough number so they could do separate analyses on that.

[Kristin] (43:25 - 43:37)
So another good point about the trial. Now to qualify for the study, men had to be at least 50 years old. But Regina, we would not qualify for this study because women had to be 55 or older.

[Regina] (43:37 - 43:43)
At the time that we are recording this neither one of us is 55.

[Kristin]
We're not even close.

[Regina]
No, but we will be someday.

[Kristin] (43:43 - 43:57)
Someday, but I love that we are currently too young for this study, right? They wanted older people because their primary outcomes were cancer and heart disease and older people, of course, are more likely to develop these. This study lasted over five years.

It was a long study.

[Regina] (43:58 - 44:02)
25,000 people, five years. That's impressive. That's expensive.

[Kristin] (44:09 - 44:49)
Well, they did a sample size calculation. Yay. And their original target was actually 20,000 people.

They ended up recruiting 25,000.

[Regina]
They overshot.

[Kristin]
They overshot.

That never happens. I'm wondering if it's just there's so much hype about vitamin D that everybody wanted to sign up for this trial, right? Good for them.

But yeah, well done because usually in trials people can't get enough. They measured the participants' vitamin D before the study started. And only about 13 percent of the participants were under 20 nanograms per milliliter to begin with.

[Regina] (44:50 - 45:02)
We talked about 20 being that common threshold that people have used in the past to define low vitamin D, which I should point out, we have debunked in part one episode.

[Kristin] (45:02 - 45:09)
Right. But because that's been used in the past, that might be a group that we are especially interested in looking at. But it turns out we only have a few thousand in that group.

[Regina] (45:09 - 45:22)
That makes sense, though, because people who volunteer for clinical trials, they're probably already healthy. So they're going to have higher vitamin D to begin with than the average American. You're not going to have many that are low.

[Kristin] (45:22 - 45:40)
These volunteers definitely had higher vitamin D to begin with than the average American. And we have to keep this in mind when interpreting the results.

[Regina]
OK, what did they find?

[Kristin]
All right. They published their main results in the New England Journal of Medicine in 2019, and they found that vitamin D did nothing.

[Regina]
Oh, you are kidding.

[Kristin]
I'm not kidding.

[Regina] (45:41 - 45:41)
Nothing?

[Kristin] (45:41 - 45:58)
The primary outcomes, again, were invasive cancer and major cardiovascular events, which is defined as having either a heart attack, a stroke or dying of heart disease. Vitamin D did not reduce the risks of either of the primary outcomes. I'll put more details about the numbers in the show notes for anyone who's curious.

[Regina] (45:59 - 46:04)
Wow. So we're done. We got the big study.

That's it. And no more to do.

[Kristin] (46:05 - 46:09)
OK, actually, they did some additional analyses. I didn't just stop there.

[Regina] (46:10 - 46:19)
Oh, no. Was this a fishing expedition? They just looked for anything in their data.

Hey, low vitamin D. They got more hangnails.

[Kristin] (46:19 - 46:38)
No, no, they were better than that.

Again, this was a pretty good trial. They pre-specified ahead of time. They said, hey, we're going to do a series of secondary and tertiary analyses.

So it was not a case of torturing the data in order to find anything. But in the paper, they presented 31 of these extra analyses. Most of these 31, although not all of them were pre-planned.

[Regina] (46:38 - 46:46)
OK, so it's good that they just didn't run thousands of tests and throw their data on the wall to see what sticks, right? I always picture a big thing of spaghetti. They throw it on the wall and see what sticks on there.

[Kristin] (46:46 - 46:50)
Do people really throw spaghetti at the wall to see if it's done? Is that a real thing?

[Regina] (46:50 - 47:01)
That feels like a cartoon or something, right?

[Kristin]
I do not do that.

[Regina]
Yet. Never say never. OK.

But they still did run over 30 tests, right? 31 tests.

[Kristin] (47:02 - 47:05)
And that's the ones we know about. There could have been others that they didn't put in the paper.

[Regina] (47:05 - 47:19)
OK. And if you use this traditional criterion, P less than 0.5 for statistical significance, then even if vitamin D does nothing at all, we still expect one or two statistically significant results just by chance.

[Kristin] (47:19 - 47:19)
Just by chance, yeah.

[Regina] (47:19 - 47:23)
And so what did they find?

[Kristin]
They found two significant results.

[Regina] (47:23 - 48:04)
Well, there we go.

[Kristin]
OK, so let me tell you what they found. They saw that cancer incidence, the development of cancer, was lower in the vitamin D group compared with the placebo group when you looked only at people who were normal weight, not overweight or obese.

[Regina]
OK.

[Kristin]
They also found that for cancer mortality, this is death from cancer rather than the development of cancer, if you excluded the first two years of data, then the vitamin D group did better than the placebo group. It just made P is less than 0.05, just barely made statistical significance. And also, they are very transparent in the paper. They admit that this analysis was not a pre-specified analysis. They did not plan this ahead of time.

[Regina] (48:05 - 48:27)
OK, that makes sense. I like that they are transparent about it. It increases my trust in them.

Although, when you start talking about the specifics, like, OK, you have to exclude the first two years and it's only in these and this, it feels a little like superstition, you know, like as long as I don't step on the cracks in the sidewalk and I'm wearing my, you know, my lucky keychain and the sun is shining.

[Kristin] (48:27 - 48:41)
Then vitamin D will work.

[Regina]
Then it works.

[Kristin]
It does feel a little bit like that.

But, you know, again, they were transparent. And I think it's really good that they were transparent because it's reminding the reader, hey, these might just be chance findings. These could just be flukes.

[Regina] (48:42 - 48:48)
Take it with a grain of salt, especially if there's no compelling biological reason why it would be�

[Kristin] (48:48 - 48:51)
Well, you can always come up with a biological story, Regina.

[Regina] (48:51 - 48:51)
This is true.

[Kristin] (48:52 - 49:25)
Phosphorus transport and super twitchy muscle fibers, of course. Can we talk about one of my all-time favorite studies, Regina?

[Regina]
Yeah, OK, digression.

[Kristin]
Yeah, a little digression that illustrates the potential problem with subgroup analyses. There was a study done in 1980. They divided, it was about a thousand heart disease patients and they divided them into two groups.

We'll call them Group 1 and Group 2. And they labeled them as such in an Excel spreadsheet or some kind of, I don't know if they had Excel back then. Yeah, but they did have Excel.

And then they did nothing. And that was the study. They just labeled them and done.

[Regina] (49:26 - 49:30)
Oh, like a fake study. Right. I mean, it was intentional.

[Kristin] (49:30 - 49:31)
Intentional, exactly.

[Regina] (49:31 - 49:35)
They were trying to make a point.

[Kristin]
They were trying to make a point, yeah.

[Regina]
Also, fun and easy to run.

[Kristin] (49:36 - 49:54)
This is the easiest study in the world. This is why I tell everyone, become a statistician. What a great life because the whole point of the paper was to make a statistical point.

[Regina]
Right.

[Kristin]
But you didn't actually have to do anything. You didn't have to treat any patients.

[Regina]
Nothing messy.

[Kristin]
All the things we talk about in our course on randomized trials, all the logistics, that's all hard. All you had to do here was analyze some data.

[Regina] (49:55 - 49:59)
Uh-huh. Uh-huh.

Okay. What did they find?

[Kristin] (49:59 - 50:09)
So they found, not surprisingly, that the two groups did not differ. They were looking at survival. They did not differ in survival. Yay.

[Regina]
Okay. As expected.

[Kristin]
No supernatural stuff going on here.

[Regina] (50:10 - 50:13)
I�m relieved that putting them on one side of the spreadsheet or the other did not determine their fate.

[Kristin] (50:13 - 50:22)
Yes, exactly. But, Regina, you can probably guess where I'm going with this, having enough exposure to these kinds of studies. What do you think they did next?

[Regina] (50:22 - 50:34)
Yeah.

Slicing and dicing the data, torturing the data.

[Kristin]
We have so many different metaphors for this in statistics.

[Regina]
Pick a metaphor. Slicing and dicing the fish before you torture them.

[Kristin] (50:34 - 51:42)
Yes. Exactly.

That's exactly what happened. They compared Group 1 and Group 2 in 18 different subgroups. And guess what?

In one of those subgroups, Of course. It was patients with, I've got to read this, three-vessel disease and an abnormal left ventricular contraction. I hope I got that right.

In that subgroup, patients in Group 2 actually had a better survival than patients in Group 1, and the P value is less than 0.05.

[Regina]
And, of course, it had to be a fluke because they did not do anything different to Group 2.

[Kristin]
This is the point of the study. We know for sure that this was a chance finding because the patients were not treated any differently.

But the point is to illustrate just how easy it is to get a chance imbalance. If you look at enough subgroups, somewhere in one of those subgroups, you're going to, by chance, get one group having somewhat better survival than the other, and it's going to make statistical significance by chance. You can imagine, if this was a real study, the authors would have come up with some beautiful biological story, right?

Of course the treatment only works in those with three-vessel disease and an abnormal ventricular contraction. And let me tell you why. Phosphorus transport in twitchy, fast muscles, right?

And it would have sounded great. But, of course, it's just a fluke.

[Regina] (51:42 - 52:01)
Right. So, we laugh about a study like this because it's obvious and it's making a point. But often studies are analyzed and interpreted this way.

[Kristin]
They go in and look at the subgroups and they find something. And there's the beautiful biological story after the fact,

[Regina]
Yes. Uh-huh, uh-huh.

Oh, so this VITAL, back to the VITAL study.

[Kristin] (52:02 - 52:28)
yes, yes. So, that's what you worry about. Again, I want to give the authors a lot of credit because they were very transparent in telling the readers, hey, this could just be chance fluke findings.

They did a very good job of that. In fact, I made fun of a press release earlier and now I want to compliment a press release to balance out the universe. Good karma.

When that study was released, they put out a press release and in the press release, they called vitamin D supplementation robustly negative. I kind of love that.

[Regina] (52:29 - 52:34)
Robustly. I've been on some dates that have been robustly negative.

[Kristin] (52:35 - 52:54)
Yes, but that's a really good, like you're telling me this was like negative times two, right. And not only that, but in the body of the press release, the principal investigator of the study, this is Joanne Manson from Harvard. She is actually quoted as warning people do not over interpret those post hoc findings. They could just be chance.

[Regina] (52:55 - 52:57)
I know Joanne. I've interviewed her.

[Kristin] (52:57 - 53:00)
Oh, I've interviewed her too. What did you interview her about?

[Regina] (53:00 - 53:05)
About walking. She ran a study on whether walking is a good exercise for the LA Times.

[Kristin] (53:05 - 53:12)
Oh, I didn't know she did other things besides vitamin D, but I guess that makes sense. I actually did interview her for Allure on something about vitamin D.

[Regina] (53:12 - 53:13)
Oh, on vitamin D. Okay, you win. That's better.

[Kristin] (53:14 - 53:36)
You have to love the transparency. The problem is, of course, that despite Joanne's excellent warning, lots of people didn't heed her warning.

[Regina]
I bet they didn't.

[Kristin]
Right. A lot of the media coverage on the VITAL trial honed in pretty quickly on those post hoc analyses.

So they said things like, oh, it didn't do anything overall, but wait, if you're normal weight, then it might do something.

[Regina] (53:36 - 54:01)
You know, I think this is human nature. Hope springs eternal. We really want something.

[Kristin]
Well, you really want vitamin D to work, Regina.

[Regina]
We all want a talisman, right, that will keep us happy and healthy and living forever. And if there's any hint of that in these studies and the results, then maybe we just go out on that shaky limb anyway.

[Kristin]
Right. It's that glass house full approach again.

Yeah, absolutely.

[Kristin] (54:02 - 54:29)
But Regina, I want to point out one other study here, another large randomized trial of vitamin D from Australia, again run for over five years. It's called the D-Health Trial. Okay.

They had about 20,000 people, 10,000 on vitamin D, 10,000 on placebo. This was just vitamin D and placebo, no omega-3s. And the participants got a little bit of a different vitamin D supplementation.

They got one dose a month of 60,000 IUs of vitamin D.

[Regina] (54:30 - 54:34)
Oh, interesting. So 60,000 divided by 30, oh, that's about 2,000 a day.

[Kristin] (54:34 - 54:54)
It's about 2,000 a day, but rather than spreading it out, they got it all at once.

[Regina]
Oh, that's kind of efficient when you think about it.

[Kristin]
Because you're body can store vitamin D, so why not just give it out at once?

For this trial, the primary outcomes were death of any cause as well as cancer mortality specifically. Main result: vitamin D, no effect on mortality, no effect on cancer mortality.

[Regina] (54:54 - 54:55)
Oh, that's so sad.

[Kristin] (54:56 - 55:32)
Yeah. But the reason I bring this trial up, it's one of the other really big and long-term trials, but the other thing that's really interesting is they published their results after VITAL. So guess what they went in and did?

They did this post-trial analysis where one of their outcomes was cancer mortality and they said, we can dump the first two years of data and see if we do that, do we find an effect too? So maybe there is some kind of lag here. So they tried that and guess what they found?

[Regina]
Oh no, nothing?

[Kristin]
No, they found the opposite.

[Regina]
Oh no.

[Kristin]
The vitamin D group had higher mortality for cancer specifically.

[Regina] (55:32 - 55:34)
That's not good news for vitamin D.

[Kristin] (55:34 - 55:43)
What this makes me think is this could just be, they both could just be chance, right? They're mucking around in the data, they could both be fluke findings because they're in the opposite direction.

[Regina] (55:44 - 55:51)
Right, and this is actually consistent with something being a fluke finding because one time it's going to go in one direction and the next time it's going to go in the other.

[Kristin] (55:51 - 56:10)
Right, in both cases the P value is really close to 0.05. It could also be, and they wrote this in the paper for D-Health, they did mention that maybe this is real because maybe the dosage matters. Maybe there's something about taking 60,000 IUs at once that's bad for cancer and spreading it out is good for cancer. I mean, that could be, right, again, biological explanation.

[Regina] (56:10 - 56:14)
I'm glad you mentioned that because I was about to take 60,000 at the beginning of the month.

[Kristin] (56:15 - 56:47)
But, you know, it makes me skeptical. I'm interested in preventing cancer mortality. I have a personal interest in it at this point and I am not popping vitamin D pills because I'm looking at these results and saying, well, you know, it went in one direction in one study but actually it went in the opposite direction and I don't want to take a chance that it's bad for me and I suspect that this is just chance findings.

All right, Regina, getting back to the VITAL trial, VITAL didn't just look at cancer and heart disease. They did a whole bunch of what we call tack-on studies because they already have this big study going on. They might as well look at some other outcomes.

[Regina] (56:47 - 56:51)
Why not? So the participants get to decide if they want to be in these other studies?

[Kristin] (56:51 - 57:09)
They could opt into these tack-on studies, ancillary studies. For example, one of them was looking at, you'll like this, Regina, depression. And to be in the depression study, people had to fill out some extra questionnaires about their depression.

Not everybody did it but it was about 18,000 out of the original 25,000 people who participated in the depression study.

[Regina] (57:09 - 57:13)
That's a big number still. Still a big number, yeah. Those are the nice volunteers.

[Kristin] (57:13 - 58:01)
Nice volunteers but guess what? Vitamin D did not prevent depression.

[Regina]
Oh, no.

[Kristin]
And there's been a bunch of other tack-on studies as well and so far here's what we know or what they've found. Vitamin D did not prevent falls, atrial fibrillation, migraines, macular degeneration or knee pain, and vitamin D did not improve body composition, cognitive function or stroke outcomes. There was only one of these tack-on studies that's shown even any signal for vitamin D and this was for autoimmune disease.

It didn't quite make statistical significance but they put a positive spin on it in their paper in the British Medical Journal anyway. But, Regina, they've looked at so many outcomes at this point. Could this just be chance?

[Regina]
It could be chance.

[Kristin]
Yeah. Easily could be chance.

I want to now kind of end on what we started with which is bone.

[Regina] (58:01 - 58:08)
Mm-hmm. Vitamin D impacts your bones, right? We talked about this.

Osteomalacia and rickets. Look how much I've learned.

[Kristin] (58:08 - 58:37)
Yeah. So, if we were going to expect to see any benefits for vitamin D, we might expect it to pop up in the skeleton and they looked at, in VITAL, bone mineral density and fractures and in 2020, they released the results for bone density. Guess what?

Vitamin D did not improve bone density or bone structure and in 2022, they released a study on fractures and vitamin D did not reduce the risk of fractures even in those who started with lower vitamin D and even in those who had osteoporosis.

[Regina] (58:38 - 58:51)
This is weird because if we suspect that vitamin D is actually strongly associated with bones, isn't this just showing the limits of all of these studies? Could it be a sign that the trials themselves aren't reliable?

[Kristin] (58:51 - 59:26)
You want to negate the entire trial now, Regina? Well, I actually think that what it's showing is maybe we just don't need huge levels of vitamin D. I actually believe this.

There have been studies that have shown if you take a calcium and vitamin D pill together that that might reduce your risk of fractures or benefit your bone density but remember, those studies had calcium and it might just be the calcium. It might not be the vitamin D. With vitamin D, it might be that you need some minimal amount to be able to absorb the calcium but beyond that, you might not get any benefit for bones.

[Regina] (59:27 - 59:36)
So it might be one of these cases where more is not better. You just need to not be the vegan in the basement who never leaves your house.

[Kristin] (59:37 - 1:00:00)
That's right. I found a great article in the New York Times by Gina Kolata. This came out in 2022 right after the fracture results from VITAL came out and this article had some great quotes. But first of all, Joanne Manson, the principal investigator of Vital, here's her quote.

She says: This will surprise many but we seem to need only small to moderate amounts of the vitamin for bone health. Larger amounts do not confer greater benefits.

[Regina] (1:00:00 - 1:00:03)
It's like she's talking to me. Directly to me.

[Kristin] (1:00:03 - 1:00:06)
She's talking to you. You do not need to be a 92 nanograms per milliliter.

[Regina] (1:00:06 - 1:00:10)
She's like, Regina, stop. Just put it down. You don't need it.

[Kristin] (1:00:10 - 1:00:45)
Right. I mean, if your vitamin D is 4 nanograms per milliliter, if you're the vegan dude, that's bad. That's definitely bad for your bones.

But, you know, maybe you just need to get a certain amount. The IOM ballparked that around the average person needs about 16 nanograms per milliliter and maybe you just don't need to be higher than that. Remember, in VITAL, most people started with vitamin D levels above 20 nanograms per milliliter so they were probably all fine and even in that few thousand people who had vitamin D levels below 20 nanograms per milliliter, they didn't find any benefit for them for bones either but probably most of them were in that range that the IOM said was fine. You know, if you're around 16, you're fine.

[Regina] (1:00:46 - 1:00:49)
Kristin, it's like you're telling me Santa Claus does not exist.

[Kristin] (1:00:50 - 1:01:14)
There's no vitamin D fairy.

[Regina]
I'm so sad.

[Kristin]
I just love this article, though, because there's some great outside expert quotes in this article.

Here's one. This scientist, Dr. Stephen Cummings, he says: Providers should stop screening for vitamin D levels or recommending vitamin D supplements and people should stop taking vitamin D supplements in order to prevent major diseases or to extend life.

[Regina] (1:01:14 - 1:01:14)
Whoa.

[Kristin] (1:01:15 - 1:01:23)
And Dr. Cummings, actually, I worked with him on one of my very first research papers as a graduate student because he's an expert in bones and I did graduate student work on bones.

[Regina] (1:01:23 - 1:01:28)
Oh, interesting. So this guy knows bones.

[Kristin]
Yes.

[Regina]
And he's like no on the vitamin D.

[Kristin] (1:01:28 - 1:01:46)
Yeah, and I think he's right. You know, he said this back in 2022 and, of course, the Endocrine Society, their new guidelines, 2024, pretty much say the same thing that we talked about in part one. Another quote I love in this article, another expert is quoted as saying he has become a vitamin D therapeutic nihilist.

[Regina] (1:01:48 - 1:01:49)
I want that on a t-shirt.

[Kristin] (1:01:49 - 1:01:52)
I think I'm a vitamin D therapeutic nihilist, Regina.

[Regina] (1:01:52 - 1:02:01)
You are definitely a vitamin D therapeutic nihilist and I know what I am getting you for your next birthday. Don't be surprised. You got to wear it.

[Kristin] (1:02:01 - 1:02:18)
But she also quoted people in this article and this is interesting. She went out and interviewed some physicians who treat patients and they were quoted as saying, hey, you know what, I'm still going to give my patients vitamin D just in case it does something. It's that wishful thinking.

[Regina] (1:02:19 - 1:02:19)
Yeah. Just in case it works.

[Kristin] (1:02:19 - 1:02:26)
People aren't ready to drop this. They really believe that vitamin D does something and it's hard to take in new evidence and drop that idea.

[Regina] (1:02:27 - 1:02:35)
Yeah, I bet it's like the people at the labs that do all the testing or maybe the pharmaceutical companies, the supplement companies.

[Kristin] (1:02:35 - 1:03:41)
Sure. They're going to have some negative things to say about VITAL and D-Health and there were some other randomized trials, smaller ones for vitamin D that also have found null results, right?

[Regina]
Deniers.

[Kristin]
They are vitamin D randomized trial nihilists.

[Regina]
I like that.

[Kristin]
And some of the criticisms, you know, we should think about them.

They're legitimate. For example, people have criticized VITAL and D-Health by saying, even though these were very long for randomized trials, over five years, that might not be long enough, especially for an outcome like cancer. Cancers can take years, even decades, to develop.

It is possible that had they followed the participants even longer, maybe they would have seen some kind of impact on cancer.

[Regina]
Okay, possible.

[Kristin]
Doesn't explain the lack of findings on all the other outcomes that don't take years and years to develop, but, you know, that's an open question with cancer.

Another criticism people have levied on these studies is, remember, that at the start of the study, a lot of the volunteers were already pretty good in vitamin D. Average was 31 nanograms per milliliter. There's a lot of people who said, well, but if you just looked at, you know, if there had been more people who were low in vitamin D, then it would have made a difference.

[Regina] (1:03:41 - 1:03:45)
Yeah, but I thought they analyzed the subgroups, right? Didn't they isolate it?

[Kristin] (1:03:45 - 1:04:05)
They did, actually, yeah. They looked in people who were lower than 20 nanograms per milliliter, and even in that group, vitamin D didn't do anything. But that's why I want to bring up one more clinical trial because there was a clinical trial in the UK where people did start with much lower vitamin D levels.

[Regina]
Oh, okay.

[Kristin]
Turns out people in the UK have pretty low vitamin D.

[Regina] (1:04:06 - 1:04:14)
That is because they hang out in pubs all day. That is my guess. And they're pasty white, and it's gray and raining all the time.

[Kristin] (1:04:14 - 1:04:21)
Yeah, I guess so. So this was a randomized trial done on COVID, and you probably heard about vitamin D and COVID during the pandemic.

[Regina] (1:04:21 - 1:04:25)
Yeah, vitamin D was supposed to reduce your risk of being infected.

[Kristin] (1:04:25 - 1:05:16)
There was lots of that narrative going around in the media. Now, there were a few studies that were actually fraudulent on vitamin D and COVID. We're not going to talk about those.

Those were retracted, gone. But finally, eventually, we did get, we couldn't have them early in the pandemic, but we did eventually get big randomized trials. So there was a randomized trial of 6,000 people done in the UK.

It was run from May of 2020 till about the end of 2021, which means it was before the vaccines for the most part, but we didn't get the results until about 2022, a little late. But they randomized 6,000 people. 3,000 people didn't get anything, not blinded.

3,000 people got a vitamin D test. And if their vitamin D was below 30 nanograms per milliliter, then they got a supplement, one of two doses.

It turns out, 86% of the people in this study were below 30 nanograms per milliliter, and two-thirds of them were below 20 nanograms per milliliter, what Michael Holick would call deficient.

[Regina] (1:05:17 - 1:05:18)
Mm-hmm. Well, there you go. Pasty Brit.

[Kristin] (1:05:19 - 1:05:26)
Right. So plenty of people that were low, most of the people were low in vitamin D. So this argument that, okay, well, that's the group where this is going to work.

[Regina] (1:05:26 - 1:05:27)
Mm-hmm.

[Kristin] (1:05:27 - 1:05:35)
Well, guess what? No benefits for vitamin D testing and supplementation did not prevent upper respiratory infections, and it did not prevent COVID.

[Regina] (1:05:36 - 1:05:36)
Mm-hmm.

[Kristin] (1:05:37 - 1:05:48)
So this argument that, you know, if people were just low enough, we would have seen something in a randomized trial. We didn't see it, at least for COVID. And certainly, COVID is not one that takes years to develop either.

[Regina] (1:05:49 - 1:05:57)
Kristin, I think we have exhausted vitamin D at this point. What do you say?

[Kristin] (1:05:57 - 1:05:58)
I think we have exhausted everybody on the topic of vitamin D.

Yes.

[Regina] (1:05:58 - 1:06:21)
Nothing left to be said anymore. So let's go ahead and wrap it up. We are going to evaluate the strength of evidence behind the claim that we're looking at today, and that claim is that upping your vitamin D will make you healthier and happier.

And we do this with our rating, our smooch scale rating, one to five, very scientific, of course.

[Kristin] (1:06:21 - 1:06:21)
Very, very.

[Regina] (1:06:22 - 1:06:32)
One smooch means little to no evidence for the claim, and five means really strong evidence. So, Kristin, I'm going to let you go. Kiss it or dish it.

What do you say?

[Kristin] (1:06:32 - 1:07:06)
Regina, you can probably guess that I am going with one smooch on this. Right? There's all this hype and all these claims that vitamin D does everything from, again, you know, improving menstruation to preventing cancer.

And I think it's all hype and based on no actual evidence because observational studies, statistical adjustment, just isn't powerful enough to tease this out. And now we have these big, well-done randomized trials, and guess what? They find nada.

Few subgroup analyses, post-hoc, I'm not convinced that that isn't just chance.

[Regina] (1:07:06 - 1:07:07)
Yep, yep.

[Kristin] (1:07:07 - 1:07:08)
How about you, Regina?

[Regina] (1:07:08 - 1:07:18)
I am going to go with two smooches on this for a couple of reasons. One, I feel like we have absence of evidence here. Like, we haven't proved that it doesn't do anything.

[Kristin] (1:07:18 - 1:07:19)
Hard to prove a negative in science.

[Regina] (1:07:19 - 1:07:28)
It is hard to prove a negative. And I do tend to be a glass-half-full kind of person as you have often pointed out.

[Kristin] (1:07:28 - 1:07:31)
You're taking the Le Mar skin cream approach as we talked about last time.

[Regina] (1:07:31 - 1:07:44)
Yes, yes. Okay, fair enough. That is me.

And I think that everyone's body is different. So, just because it's not working in these big clinical trials, but that might just be my bias in here. It's still low.

It's not five.

[Kristin] (1:07:44 - 1:07:50)
It's not four. It's not three. So, I have done some work in convincing you that maybe this is a lot of bunk.

[Regina] (1:07:51 - 1:08:07)
Yeah, yeah. It has been really eye-opening, I have to say. All right.

Let's now share our morals of the story today. So, just like Aesop's fables, you get a moral at the end. What did we learn?

We have our methodological morals. Methodological morals. In today's story, right? You go first. You go first.

[Kristin] (1:08:07 - 1:08:16)
Regina, mine for today is going to be: Variables with too many entanglements make observational studies a fool's game.

[Regina] (1:08:17 - 1:08:27)
Oh, I like that one. Fool's game. When your variable is entangled and promiscuous.

Yes. And it sleeps around. Vitamin D.

[Kristin] (1:08:28 - 1:08:32)
Yeah, statistical adjustment just not powerful enough to tackle that. How about you, Regina?

[Regina] (1:08:33 - 1:08:40)
I am going to go with this: If your intervention works only when you torture your data, it's probably a false confession.

[Kristin] (1:08:41 - 1:08:42)
I like it. Yes.

[Regina] (1:08:43 - 1:08:44)
There's subgroup analysis.

[Kristin] (1:08:44 - 1:08:46)
Subgroup analysis. Great.

[Regina] (1:08:46 - 1:08:54)
All right. So, this has been a delightful journey over two parts, and we are just wrapping it up. I'm a little sad to leave vitamin D now.

[Kristin] (1:08:54 - 1:08:57)
Right. I feel like... Everything you ever wanted to know about vitamin D.

[Regina] (1:08:57 - 1:09:08)
More than I ever wanted to know, but it has been really eye-opening, and you've changed the way I think about things and probably a lot of other people, too. So, thank you so much.

[Kristin]
Thanks, Regina.