Pheromones: Is sexy sweat the key to genetic diversity?

[Regina] (0:00 - 0:01)
I think you're going to like this, Kristin

[Kristin] (0:02 - 0:02)
Really?

[Regina] (0:03 - 0:03)
And by like, I mean hate.

[Kristin] (0:05 - 0:36)
So I might have some criticisms, is what you're saying.

[Regina]
Maybe, maybe a couple.

[Kristin] Welcome to Normal Curves.

This is a podcast for anyone who wants to learn about scientific studies and the statistics behind them. It's like a journal club, except we pick topics that are fun, relevant, and sometimes a little spicy. We evaluate the evidence, and we also give you the tools that you need to evaluate scientific studies on your own.

I'm Kristin Sainani. I'm a professor at Stanford University.

[Regina] (0:37 - 0:42)
And I'm Regina Nuzzo. I'm a professor at Gallaudet University and part-time lecturer at Stanford.

[Kristin] (0:42 - 0:48)
We are not medical doctors, we are PhDs, so nothing in this podcast should be construed as medical advice.

[Regina] (0:48 - 1:01)
Also, this podcast is separate from our day jobs at Stanford and Gallaudet University. Kristin, today we are going to look at a classic paper in the scientific literature, commonly known as the Sweaty T-Shirt Study.

[Kristin] (1:02 - 1:03)
Ooh, interesting nickname.

[Regina] (1:03 - 1:14)
It is. And along the way, we are also going to learn about things like pheromone dating parties, pheromone dating websites, and choosing your dating partner by sniffing up his armpits.

[Kristin] (1:14 - 1:15)
Okay, that sounds interesting.

[Regina] (1:15 - 1:23)
It's a good paper, weird but fascinating. And a ton of studies have then taken that and extrapolated even further.

[Kristin] (1:23 - 1:31)
All right, so the idea here is that we all might be choosing our mates based on how they smell. Am I getting that right?

[Regina] (1:32 - 1:55)
Absolutely. The claim is this, that women prefer the smell of men whose genetics are different from them. And we're going to take a closer look at that. But we're also going to touch on a few topics that people may have heard about in stats class, things like bar charts, correlated observations, but people are probably not learning in stats class about sexy body odor.

[Kristin] (1:56 - 2:10)
Well, correlated observations, that is something that I teach. I have a lot to say about that. But yeah, I don't teach it in the context of sexy body odor.

I use the example of eyes because you have two eyes, so those eyes are correlated within a person, right? That's kind of boring.

[Regina] (2:11 - 2:13)
Armpits. We've got two armpits.

[Kristin] (2:13 - 2:27)
Oh, well, I could switch examples and that might wake a few students up.

[Regina]
Body odor, it always works.

[Kristin]
Yes, definitely.

But Regina, I want to know, what goes on at one of these pheromone dating parties that you mentioned? Did you ever go to one?

[Regina] (2:27 - 2:30)
I have not. Sadly, I have just read about these.

[Kristin] (2:30 - 2:31)
Okay. Yeah. So how does it work?

[Regina] (2:31 - 2:38)
I can tell you the whole concept, though. So these are patterned off of the sweaty t-shirt study.

[Kristin] (2:38 - 2:40)
Okay, the study that we're going to talk about upcoming.

[Regina] (2:41 - 2:54)
So here's the idea, men, women, both, you sleep in a clean t-shirt for two nights. Okay. Yeah, no deodorant.

Just like all you. And then you bring it into a bar, a party at a bar, in a sealed plastic bag.

[Kristin] (2:54 - 2:58)
So you're not allowed to wear any deodorant during those whole two days?

[Regina] (2:59 - 3:04)
Yep. Yeah, or at least not at night. I don't think that there were a lot of rules here.

[Kristin] (3:06 - 3:13)
Okay. I'm thinking of this like a scientist, let me put my scientist hat aside for now. You sleep in your t-shirt at night and it picks up your scent.

[Regina] (3:13 - 3:32)
I did read about one person who let their dog sleep in their t-shirt and they brought that in. Oh, that's interesting. So clearly this is not a well-controlled study.

[Kristin]
I want to know what happened with the dog t-shirt, though.

[Regina]
Like how many people? Yeah. I would probably smell the dog and say, yep, I want that one.

[Kristin] (3:32 - 3:34)
I love my dog. I mean, yes.

[Regina] (3:35 - 3:45)
Yes. That was crazy. Okay.

Yes. So uncontrolled study, uncontrolled experiment. They recommend that you sleep in it for two nights and just to kind of infuse it with your own body odor.

[Kristin] (3:45 - 3:55)
And then you're bringing the t-shirt in. You're allowed to shower before the party. You are allowed to shower, probably encouraged to shower before the party.

So you bring this awful, gross t-shirt in and what are we doing with the t-shirt?

[Regina] (3:55 - 3:56)
You bring it in a sealed plastic bag.

[Kristin] (3:56 - 3:58)
Oh, nice. That's very hygienic.

[Regina] (3:58 - 4:03)
And the organizers take it and they give it an anonymous label and gender.

[Kristin] (4:03 - 4:05)
Oh, it is kind of like a little science experiment then.

[Regina] (4:05 - 4:13)
It is kind of, although I think we could probably improve on it. But they throw it all, all the bags on a big table in the middle of the bar.

[Kristin] (4:13 - 4:14)
Okay. You're at a bar.

[Regina] (4:14 - 4:30)
You're at a bar, first of all. Right. I mean, what other way are you going to smell t-shirts?

You need a few drinks before you do this. So you're wandering around. You're having drinks.

You're chatting people up. And at the same time, you're sticking your head in these plastic bags.

[Kristin] (4:30 - 4:35)
Oh, you go over and smell. And you're smelling the sweaty t-shirts. And if you find one you like, then what happens?

[Regina] (4:35 - 4:48)
Then what happens? Then you walk with the bag up to the organizer's table up front. And they take a photo of you holding the bag up.

And then they project that up onto a big screen so everyone can see.

[Kristin] (4:48 - 4:49)
This is not a party for introverts.

[Regina] (4:50 - 4:51)
Definitely not.

[Kristin] (4:53 - 5:03)
And yeah. Okay. And so you're holding the bag with the number.

So the person who owns that bag, that they can see. And are they getting to smell your bag then?

[Regina] (5:03 - 5:10)
They are not. So it's not double blind.

[Kristin]
Oh, goodness.

[Regina]
I know. See, I told you. We can improve on the state of the science.

[Kristin] (5:10 - 5:20)
This is not a good science experiment. Okay. But they're just, if they like your look.

If they like your look, sure. Then they're going to go find you. So what if your picture gets projected and then nobody comes and finds you?

[Regina] (5:20 - 5:24)
I know. That would be crushing.

[Kristin] (5:24 - 5:26)
Okay. But this is in the background. We had a lot of alcohol. So maybe you don't care. So

[Regina] (5:27 - 5:30)
maybe, right. Okay. You just did not even notice.

[Kristin] (5:30 - 5:43)
So at least the person who picked the bag, they are basing their choice without seeing the person, without knowing their salary. Right. Just on this primal instinct.

Well, maybe we should all be picking that way. Who knows, right?

[Regina] (5:43 - 6:09)
Is this better? This is actually the epiphany that the woman had who started a lot of these parties. Oh, okay.

So she started a business where she was doing this. So she said that she had always been dating around. She was in her 20s dating based on, you know, salary and resume and looks.

Until one time she dated someone she just had an animal attraction to. And she said it went so much better. Interesting.

And she was obsessed with the way he smelled.

[Kristin] (6:10 - 6:12)
Oh, interesting. And she knew about the sweaty t-shirt study?

[Regina] (6:13 - 6:19)
She had read about the sweaty t-shirt study. So she set up these parties to share her epiphany with the entire world.

[Kristin] (6:19 - 6:22)
That's where the organizers are from. She provided the organizers. Oh, interesting.

[Regina] (6:22 - 6:26)
So it was like in different cities, London, New York, LA.

[Kristin] (6:26 - 6:33)
So was it successful? I mean, did she match a bunch of people? And are they married and living happily ever after now?

[Regina] (6:34 - 6:39)
I don't think they had a longitudinal follow-up.

[Kristin] (6:39 - 6:41)
They didn't have outcomes. What was the primary outcome?

[Regina]
What was the primary outcome?

[Kristin]
Marriage, right?

[Regina] (6:41 - 6:52)
Or maybe just hooking up for the night. Yes. We don't know.

Sadly, the world will never know unless we start one up again. But that is the idea behind the pheromone dating site.

[Kristin] (6:52 - 6:57)
And this was based, again, on that sweaty t-shirt study, which we're going to get to. I'm looking forward to talking about the study itself.

[Regina] (6:57 - 7:06)
But before we do that, I think it's probably easiest if we talk about the pheromone dating site that I alluded to earlier.

[Kristin] (7:06 - 7:06)
Okay.

[Regina] (7:06 - 7:07)
Because there's more science there.

[Kristin] (7:07 - 7:09)
So another business model off of the t-shirt study. Wow.

[Regina] (7:09 - 7:18)
There's another one. This one has more science, so you're going to like it.

So these pheromone dating sites, I wrote about one, actually, back in 2008.

[Kristin] (7:19 - 7:23)
This is when you were writing the science of sex column for the LA Times.

[Regina] (7:23 - 7:34)
The mating game. The science behind mating, dating, and sex. Yes.

Not Regina's sex life. I always feel like I need to point that out. Yes.

So I dug up my old LA Times story.

[Kristin] (7:34 - 7:34)
Oh, I want to hear it.

[Regina] (7:35 - 7:51)
Here's just a quick quote. Okay. The dating site, quote, offers to find you a lover who smells good.

If all goes well, you'll get a lusty good smell, the kind that makes you bury your face in your mate's pillow the next morning to catch the lingering scent.

[Kristin] (7:51 - 8:05)
Well, I love your writing, Regina. That's very evocative. Also promising a lot.

And question for you, are you sending a sweaty t-shirt to the dating site and then they're like cutting it into little pieces and sending it to a potential mate? Like, how does this work?

[Regina] (8:06 - 8:07)
Logistics are very important here.

[Kristin] (8:08 - 8:08)
Exactly.

[Regina] (8:08 - 8:11)
This is the cool thing, though. There's no smell. It's all genetics.

[Kristin] (8:11 - 8:12)
Okay. Whoa. Wait.

What?

[Regina] (8:13 - 8:17)
Uh-huh. No actual smell. All of this thing is just genetics.

You send in a sample of your spit.

[Kristin] (8:18 - 8:18)
Oh.

[Regina] (8:19 - 8:24)
And they use a few particular genes from your DNA to suggest your matches.

[Kristin] (8:24 - 8:31)
Okay. So they're using genes that they think are related to pheromones that people might be smelling. Uh-huh.

Okay.

[Regina] (8:31 - 8:47)
It's like, um, instead of the DIYing it by going around snipping up people in a bar. Which could be confounded by, like, the alcohol you're drinking and things. Yes.

Right. This is like cutting straight to the chase and saying, okay, there's genetics behind these pheromones.

[Kristin] (8:47 - 8:54)
And are they matching you, like, sight unseen? You never get to see a picture or know the age or anything of the person?

[Regina] (8:55 - 9:00)
This site does not exist anymore, but from what I can understand is they just suggested matches and then it went on like a traditional thing.

[Kristin] (9:01 - 9:02)
So you can still, you can still screen.

[Regina] (9:03 - 9:03)
You can still screen.

[Kristin] (9:03 - 9:07)
I want one where they're just like, nope, this is the person you're compatible with. Go meet them and that's it. Done.

[Regina] (9:07 - 9:11)
That would actually make a great reality show.

[Kristin] (9:12 - 9:18)
That would make a great reality show. Oh, Regina. Oh, my God. We're onto something.

All right. But you said that the dating site no longer exists, so it went defunct.

[Regina] (9:18 - 9:24)
It went defunct. So maybe it didn't work. But seriously, if we started it as a reality show, love at first whiff.

[Kristin] (9:24 - 9:30)
Love it first whiff, oh, my God, we shouldn't be giving this away. We shouldn't be giving this idea.

We should be keeping this in a secret box somewhere.

[Regina]
I think we just trademarked it.

[Regina] (9:32 - 9:43)
This is the idea. So you also got a lot of perks. Like you said, they were promising a lot.

So they were not just promising you someone who, like, who smells.

[Kristin] (9:43 - 9:50)
Right. So it's not just that you are going to get somebody you like the smell of. You're also going to get somebody who, what?

[Regina] (9:50 - 10:00)
Better sex.

Oh. Increased fertility. Oh.

Healthier kids. Less cheating. And more orgasms.

For women only. But that's okay.

[Kristin] (10:01 - 10:19)
The men appreciate that anyway. The men benefit from that. Wow.

So they're saying if you pick your mate this way. Yes. Then it really gets down to all the animalistic pheromones.

All about the sex and mating reproduction. That will all go well. Now, he might not have any money.

He might be on the street, but the sex will be good.

[Regina] (10:19 - 10:22)
But the sex would be good and you'd be happy.

[Kristin] (10:22 - 10:29)
I'd like to know, like, if there were success stories from this. But it's defunct. So maybe it didn't work or maybe this just wasn't a good business model.

[Regina] (10:29 - 10:32)
I'm wondering if people didn't like sending in their spit.

[Kristin] (10:32 - 10:32)
Oh.

[Regina] (10:32 - 10:40)
Because when you tell people how you met, it was like, we met through drool. That does sound kind of gross. Maybe it's just not so sexy.

[Kristin] (10:40 - 10:47)
All right. So, Regina, this company was matching people based on genetics. But how were they matching them?

How was that? What were they looking for?

[Regina] (10:47 - 11:08)
Yeah. So it's interesting. They were looking to see if you and your potential mate had different genes on just a few genes in one particular area of the chromosome.

And that area is called MHC, major histocompatibility complex. You might have also heard this called HLA, human leukocyte antigen.

[Kristin] (11:08 - 12:29)
And those are different names for the same genes, actually. And these are immune genes. They have something to do with the immune system.

Immune system. And Regina, when you mentioned for today's podcast, we were going to be talking about HLA genes. I remembered that I wrote an article back in 2011 where I talked about HLA genes.

[Regina]
You did.

[Kristin]
Yes. And the context was I was writing.

I think this is the only time I've written about sex. I was writing about sex. I was writing about ancient sex, though.

[Regina]
Old people having sex.

[Kristin]
Sorry. Archaic humans.

Sex during evolution of modern humans because modern humans and archaic humans like Neanderthals were on the earth for a period of time at the same time. And they actually intermated. They had sex with each other.

And we know this because you can find little pieces of Neanderthals like teeth and pinky fingers. And then people have sequenced the DNA from those samples. And you can show that there are some genes that we inherited from Neanderthals.

[Regina]
Hmm. In the modern gene pool. And these are the HLA genes?

[Kristin]
Yes. I was writing about research from Stanford. And one of those genes that we inherited from Neanderthals is a specific HLA gene, which is showing how important these HLA genes are because we kept this gene throughout evolution.

And it must be benefiting us in some way, maybe because HLAs, it's good if they're diverse.

[Regina] (12:29 - 12:36)
I am so jealous that you wrote about Neanderthal sex because now I want to. That is on my bucket list.

[Kristin] (12:37 - 13:04)
And because the gene I was talking about was an HLA gene, I had to give a little context at the beginning of the article. So I said what HLAs are and I'm going to read you one of the sentences from my article:

HLA diversity is so important that it may even influence mate selection. Studies show that people are attracted to the sense of prospective sexual partners with disparate HLA types. I think I wrote about the T-shirt study.

[Regina] (13:04 - 13:05)
That is. Oh, absolutely.

[Kristin] (13:06 - 13:28)
Right. I mean, it's exactly what we're talking about. I had completely forgotten that I had written that.

I didn't maybe do my due diligence as a journalist because I just said studies show. Everybody knows. I did not go back to the original source.

I must have read that on the Internet. Like there's a bunch of studies that show this and I just took that at face value. So maybe we're doing a little penance today.

Maybe I could have fact checked that better.

[Regina] (13:28 - 13:35)
You know, it's fascinating that you brought this up because people do reference this sweaty T-shirt study all the time. It's everywhere.

[Kristin] (13:35 - 13:44)
I referenced it without even knowing that that's what I was referencing. But that's great that we're going to actually go back and, you know, 13, 14 years later here, I'm going to actually learn what was the science behind this.

[Regina] (13:44 - 13:47)
What was the science, right? Or is there any science behind this?

[Kristin] (13:47 - 14:04)
The other thing I wrote about HLAs in the background, just to give people a sense of what genes we're talking about. This is probably the context where most people know of HLAs and that's organ transplantation. Because when we talk about matching people for organs, what we're really matching is those HLA genes.

[Regina] (14:05 - 14:27)
Be the match.

Yes, exactly. It's because these genes, to get into the science behind why that's important. The genes control how the immune system is recognizing those things floating around our body.

What is supposed to be there, what is self, versus what is not self, not supposed to be there. There are born invaders, bacteria, viruses, fungi, parasites. Right.

Or someone else's organ. Right.

[Kristin] (14:27 - 14:40)
Normally, you want your immune system to react to those things. But in the case of organ transplant, you don't want the body to see that as foreign and to reject it. So you want to be as similar as possible to the organ donor in terms of your HLA genes.

[Regina] (14:41 - 14:43)
It's hard to find an exact match.

[Kristin] (14:44 - 14:44)
It is, yeah.

[Regina] (14:45 - 14:52)
If you're not related to them. I think in the general population, finding an exact match, it's like one in 100,000. Right.

[Kristin] (14:52 - 14:53)
It's quite rare.

[Regina] (14:53 - 14:53)
Yeah.

[Kristin] (14:53 - 14:55)
That's why often family members are the best donors.

[Regina] (14:55 - 15:02)
So with dating, though, we want kind of the opposite. We want someone with a different gene. Right.

[Kristin] (15:02 - 15:14)
We don't want somebody similar because you don't want to be marrying somebody in your family. Unlike for organ transplantation, you're looking for as diverse as possible. Genetic diversity is good in evolution. Hybrid vigor.

[Regina] (15:14 - 15:21)
Hybrid vigor. So this is getting to like the immune system.

You want to give your kids the best possible immune system.

[Kristin] (15:21 - 15:22)
Right.

[Regina] (15:22 - 15:38)
And it's kind of interesting how the genes do it. Like, why do you want someone with different genes? I had to go look this up.

So the genes are these six major HLA genes. And my favorite is HLA-DR. This gene comes in like 11,000 different variations or flavors.

[Kristin] (15:38 - 15:40)
You're talking about alleles. Alleles.

[Regina] (15:40 - 15:42)
Right, yes. I like to think of them as ice cream flavors.

[Kristin] (15:42 - 16:05)
Yes, slightly different versions of the gene. And the HLA genes have the most diversity of any genes in the human body.

[Regina]
Highly polymorphic.

Right, I like using the technical term, you know, if you compare that to, say, our eye color gene or something. There's probably very few versions of that. But the fact that we have so much diversity in these HLA genes tells you that diversity in those genes must have been important in evolution.

[Regina] (16:05 - 16:35)
Important, which is why we're still hanging on to the Neanderthal genes there. And so it works because, okay, you've got these 11,000 different alleles, you know, flavors. They each specialize in recognizing a slightly different foreign invader, which is just mind-blowing.

So there are some that are really good at detecting the bacteria that causes leprosy. Oh, interesting. And another one, hepatitis C.

And another one, HIV, actually recognizing the HIV virus.

[Kristin] (16:35 - 16:41)
Right, so if you carry that particular allele, you might be less likely to get HIV even if you were exposed.

[Regina] (16:41 - 16:49)
So you can see why you want your partner to have different alleles, not the same. That makes total sense. Kind of like an investment portfolio.

[Kristin] (16:50 - 17:20)
Diversify, diversify. Okay, Regina, so we want to choose a mate that has very different HLA genes from us. But if I were guessing, I would say, like, okay, when you're looking for a mate, often we, you know, opposites attract.

We might be looking for somebody who looks very different from ourselves. I definitely went for the exotic types when I was dating. And I thought maybe I'm doing that because, you know, hybrid vigor.

So you're saying, though, it's not just looks. You're saying that we might be able to actually smell each other's HLA genes.

[Regina] (17:20 - 17:27)
Like, how does that work? Smell, it is fascinating, especially because researchers are not sure how we smell each other's genes.

[Kristin] (17:27 - 17:29)
It sounds a little far-fetched.

[Regina] (17:29 - 17:54)
Is there anything to base this on? The idea that maybe these HLA genes are, you know, creating these chemicals and then it's coming out in our sweat. What they do have evidence for is this whole thing in mice.

That mice can smell each other's pee and they can smell the different genes in their pee and that they're choosing to mate preferentially with mice that have different immune system genes.

[Kristin] (17:54 - 17:57)
Oh, okay. So there's some evidence in mice that this is true.

[Regina] (17:57 - 18:03)
They saw this in animals and no one had studied it in humans yet. And that's where you get the sweaty t-shirt study.

[Kristin] (18:03 - 18:05)
That's the origin of the sweaty t-shirt study.

[Regina] (18:06 - 18:08)
Which I think we are now ready to talk about.

[Kristin] (18:08 - 18:36)
I can't wait to hear about it, but let's take a short break first.

[Kristin]
Welcome back to Normal Curves. Today, we're looking at the claim that women prefer the smell of men whose genetics are dissimilar to them.

And we're now going to talk about the study behind this claim, which is the sweaty t-shirt study. And I'm really looking forward to hearing the details of this study.

[Regina] (18:37 - 18:45)
Yeah, I think it's one of those studies that people have heard about. Maybe they remember conclusions, but if you press them, they can't really tell you the details.

[Kristin] (18:46 - 18:51)
Or guilty, wrote about and didn't look up any of the details. Took at face value, yes.

[Regina] (18:52 - 19:08)
Understandable, understandable. Spoiler alert right here. Let's just say it was not as rigorous as I was hoping it would be.

Let's just say that. So I'm going to hit the highlights here. Some places, you're just going to have to trust me.

Kristin, I know you want details.

[Regina] (19:09 - 19:24)
We'll put more info in the show notes. Dig in more. Okay, because it had problems with how they designed the study, the different ways they analyzed the data, how they reported their methods, the results, the conclusions they made.

It was a whole bit.

[Kristin] (19:25 - 19:36)
Well, I mean, first of all, I'm shocked that there was a bad study, right? Those exist, right? I'm not surprised that the study was bad.

I'm disappointed, though, since this is widely cited. I know.

[Regina] (19:36 - 20:03)
It's not all horrible, but let's get into some of those details. Sadly, the paper does not have the word sweaty or t-shirt in the title. I know, I was hoping.

It's called MHC Dependent Mate Preferences in Humans.

[Kristin]
Such a boring academic title.

[Regina]
Published in 1995.

Lead author is a guy named Klaus Wedekind, an evolutionary biologist at the time. He was a lecturer at the Zoological Institute at the University of Bern in Switzerland.

[Kristin] (20:03 - 20:04)
Okay, Switzerland.

[Regina] (20:05 - 20:08)
So, let's get a little bit into methodology.

[Kristin] (20:08 - 20:08)
Right.

[Regina] (20:08 - 20:10)
First of all, sample size.

[Kristin] (20:10 - 20:12)
Yes, how big was this study? I want to know.

[Regina] (20:12 - 20:32)
93 students.

[Kristin]
Okay, not the worst sample size ever. 49 women, 44 men.

[Regina]
Okay. So, remember those numbers. They were all about 25 years old on average.

And remember from the study goals, there were two things they needed to measure that we're going to talk about. First of all was what male body odors the women preferred.

[Kristin] (20:32 - 20:32)
Right.

[Regina] (20:33 - 20:38)
And they also needed to look at the different HLA gene versions that the men and the women had.

[Kristin] (20:38 - 20:46)
Right. So, I'm guessing that the preference for smells was through sweaty t-shirts because it's called the Sweaty T-Shirt Study. So, they brought in sweaty t-shirts like the pheromone parties.

Uh-huh.

[Regina] (20:46 - 20:52)
Men wore a new cotton t-shirt for two nights. And unlike the pheromone party, this was actual science.

[Kristin] (20:52 - 20:53)
Yeah.

[Regina] (20:53 - 20:59)
So, they had a list of instructions. No smoking, no drinking, no sex, no scented soap.

[Kristin] (20:59 - 21:02)
So, they didn't want to contaminate their natural smell. That makes sense. Sure, right.

[Regina] (21:03 - 21:03)
Animal smell.

[Kristin] (21:03 - 21:08)
And the women didn't have to wear the t-shirts because this is just women smelling men. Right. But the women had a protocol to follow.

[Regina] (21:08 - 21:15)
Oh, okay. The women had to use a nasal spray for two weeks before they experiment to keep their nose moist and virus-free.

[Kristin] (21:16 - 21:19)
Okay. And the women had to have the full sense of smell and their smell to be protected prior to the study.

[Regina] (21:20 - 21:26)
Okay. Also given a copy of a novel to read to sensitize their smell perception. A book.

[Kristin] (21:26 - 21:26)
Okay. What's the novel?

[Regina] (21:26 - 21:35)
It's called Perfume, the Story of a Murderer. Okay.

From 1985. Wow. Have you read it?

[Kristin] (21:35 - 21:35)
I have not read it. No. Have you?

[Regina] (21:36 - 21:38)
I actually downloaded it and started reading it.

[Kristin] (21:38 - 21:40)
Really? Okay. I was really curious.

Is it good?

[Regina] (21:41 - 21:43)
Oh, my God. It's so weird.

[Kristin] (21:43 - 21:43)
Really?

[Regina] (21:44 - 21:49)
It's about a guy in 18th century France who is a super smeller so he can smell things down to the molecule.

[Kristin] (21:50 - 21:50)
Okay.

[Regina] (21:50 - 21:54)
And he murders women to capture their smell and make perfume from them.

[Kristin] (21:54 - 22:03)
Oh, well, I think I'm not sure I'd want to be in this study after reading that book. What message is that sending to the women? Yeah.

Was that just supposed to prime that they're aware of their sense of smell?

[Regina] (22:03 - 22:15)
They're aware of their sense of smell and how important it is. Okay. So the men, the guys had infused their lovely scent into the t-shirts for two nights.

Each woman got six t-shirts to smell.

[Kristin] (22:15 - 22:15)
Okay.

[Regina] (22:15 - 22:27)
Three were from men that were classified as genetically dissimilar, the exotic, you know, genes. And three were from men who were classified as genetically similar. And the women sniffed each t-shirt.

[Kristin] (22:28 - 22:35)
Regina, how did they determine which men were genetically exotic or genetically similar? What was the classification scheme?

[Regina] (22:35 - 22:49)
Yeah, they were not transparent about that, actually. I know. Like, how much overlap in genes do you need before you're defined as genetically similar?

Did you have to be eligible to be, you know, an organ donor for them? Or just kind of similar?

[Kristin] (22:50 - 22:52)
So they didn't give any details in the paper?

[Regina]
No, they did not.

[Regina] (22:53 - 22:58)
And they were only able to look at three of the six HLA genes. So that is not great either.

[Kristin] (22:58 - 22:59)
That's not great.

[Regina] (22:59 - 23:07)
Yeah. Yeah. When you make a judgment call like they're doing here, you really need to report it.

Transparency is key here.

[Kristin] (23:08 - 23:09)
Right. Otherwise, we can't replicate it.

[Regina] (23:09 - 23:10)
Right.

[Kristin] (23:10 - 23:11)
We don't know if it's valid.

[Regina] (23:11 - 23:25)
Right. Another thing they weren't transparent about, Kristin, they did not report all their results, actually. The women rated three things.

How intense the body odor was, how pleasant it was, and how sexy it was.

[Kristin] (23:26 - 23:32)
And I'm guessing that we care the most about how sexy the smell was to the women, right? The sexiness.

[Regina] (23:32 - 23:38)
Actually, they never reported the results on sexy.

[Kristin]
What? Are you kidding?

[Regina]
They never reported them.

[Kristin]
We don't get the data on that at all?

[Regina] (23:38 - 23:38)
Only pleasant.

[Kristin] (23:39 - 23:53)
OK, so this is making me very suspicious.

[Regina]
I told you. I told you.

[Kristin]
You usually are going to report all your variables, especially since sexiness is the relevant one here. We're not going, oh, I'm going to date that guy because he's pleasant. Because he's pleasant.

[Regina] (23:53 - 23:54)
We do not.

[Kristin] (23:54 - 23:56)
We date men that are sexy.

[Regina] (23:57 - 23:58)
Right. Exactly.

[Kristin] (23:58 - 23:59)
So, OK.

[Regina] (23:59 - 24:02)
And there's only three of them, so they could have reported all of them.

[Kristin] (24:02 - 24:25)
They could have. It's not like it takes up a lot of space in the paper. But it makes me worried that something happened like the following.

I don't know this for sure, but this is where my hackles go up. That maybe they analyzed the data for pleasantness. They analyzed the data for sexiness.

And pleasantness just made statistical significance. And sexiness just missed statistical significance or something like that. And they're like, oh, I don't want to.

We don't want to show the one that wasn't a positive or something.

[Regina] (24:25 - 24:41)
And another thing they were not transparent about, Kristin, they didn't actually report any numerical summaries of their data. They used bar charts to show the average value on these ratings, like how pleasant the odor was. But they never gave the exact numbers.

[Kristin] (24:41 - 24:47)
So you had to, what, take a ruler, draw a line across to the y-axis and guess or estimate those numbers?

[Regina] (24:47 - 24:51)
That is basically what I had to do just to recreate the numbers for us to talk about.

[Kristin] (24:51 - 24:52)
That's annoying.

[Regina] (24:53 - 25:02)
And even worse than that, Kristin, can we do a little statistical detour here about bar charts and what they are actually supposed to be used for?

[Kristin] (25:02 - 25:02)
Sure.

[Regina] (25:02 - 25:09)
Because it is not this. This is an inappropriate use of bar charts. Old man shakes fist at clouds. That is me.

[Kristin] (25:10 - 25:13)
Yes, I know this is one of your pet peeves, Regina.

[Regina] (25:13 - 25:20)
Can you explain why for our audience? The data we have here are numbers. They're ratings on a scale.

[Kristin] (25:20 - 25:30)
Right. These are what we call numerical data. They are numbers that you can add, subtract, multiply or divide.

But bar charts are not meant for numerical data. They are meant for categorical data.

[Regina] (25:30 - 25:51)
They are not meant for numerical data. Categorical data would be something that, believe it or not, has a category. It's like how many people have blue eyes versus brown eyes or green eyes.

Now, bar charts make sense for categorical data because the height of the bar corresponds to something, to how many people are in each category.

[Kristin] (25:51 - 25:54)
Right. Like this many people have blue eyes, this many people have green eyes.

[Regina] (25:54 - 26:03)
Right. And you can almost even picture each of those people stacked up standing on top of each other, right? And that's the height of the bar.

It has a physical correspondence.

[Kristin] (26:04 - 26:14)
Right. But the bar doesn't mean much when you're talking about numerical data. Typically, people report the mean and they just make a bar that starts at zero and ends up at the mean, which actually makes no sense.

[Regina] (26:14 - 26:27)
No sense. Because why is it starting at zero, right? The height and zero don't correspond to anything physical.

Like, what if you had a mean that is negative, right? Would you have an upside-down bar chart?

[Kristin] (26:27 - 26:34)
I want to see an upside-down bar chart. Right. The only thing they're displaying here is the mean, but that's just a single number.

[Regina] (26:34 - 26:37)
It's like we could erase the entire bar and just give a point.

[Kristin] (26:37 - 26:39)
Exactly. So just put the number in a table.

[Regina] (26:40 - 26:44)
But for some reason, people get so excited about bar charts, don't they?

[Kristin] (26:45 - 26:55)
People love their bar charts and they're misused all the time. I think maybe it's like the easiest chart to get in your canned statistical analysis program. But now our audience will know better.

[Regina] (26:55 - 27:31)
They will know better. All right, Kristin, let's take stock for a moment, shall we? We have talked about the things they were not transparent about and all the red flags there.

So a little recap. It was they did not tell us how they defined their groups. They dumped one of their variables, their sexy variable, and they did not report numeric summaries.

Right, yes. All right, let's talk now about their study design and how that affected the structure of their data and the implications for that. And this is going to bring us to correlated observations.

[Kristin] (27:31 - 27:38)
Oh, goody. You foreshadowed that we were going to talk about this today. I might have to get out my soapbox.

[Regina] (27:39 - 27:42)
I'm surprised you don't have it out already. Ready to go. How do you teach about correlated observations?

[Kristin] (27:43 - 27:55)
All right, Regina, I start with talking about the unit of observation. Like what is the thing that you are measuring? You have to define that first.

Are you measuring a country? Are you measuring a person? Or maybe you're just measuring an armpit.

[Regina] (27:55 - 28:08)
Right, and each unit of observation is getting a row in the data set. So if you're measuring a person's belly button, there's only going to be one row of data per person because we only have one belly button.

[Kristin] (28:08 - 28:11)
What are we measuring about the belly button? Is it we are smelling the belly button?

[Regina] (28:11 - 28:25)
Not how they smell, how much lint. But if you're measuring a person's armpit separately, left armpit, right armpit, how much lint you have in your armpit, there are going to be two rows of data for that person.

[Kristin] (28:26 - 28:42)
Right, and if there are multiple rows of data that belong to the same person, these are going to be correlated. Because obviously, like if we're measuring, let's say, smell in the armpits, the smell of your left armpit is likely to be very similar to the smell of your right armpit. Those are not independent.

[Regina] (28:42 - 28:58)
Especially if they're full of lint. Anyway, so, OK, we are talking here about correlated observations, though, and I think it's important that we maybe take a moment to talk about how that's different than correlated variables because they kind of sound the same.

[Kristin] (28:58 - 29:24)
Oh, yes, this confuses everybody because the names sound too similar. Correlated observations arise when rows of data are related, whereas correlated variables mean that columns of data are related. For example, if I've measured height and weight, these values are going to be stored in different columns and those columns are obviously going to be related.

But this is not correlated observations. This is correlated variables.

[Regina] (29:25 - 29:34)
Very nice. I like that one, actually. All right, so correlated observations here.

Let's talk about why it's so important to handle them correctly.

[Kristin] (29:35 - 30:09)
Many statistical models assume that your observations are independent. So if you feed correlated observations into these models, you are violating an assumption of the model and you will get the wrong answer. Like if our study involves 50 men but 100 armpits and we pretend as if we have 100 men, not just 100 armpits, we are artificially inflating our sample size and the results might look more impressive than they actually are.

And a lot of people get this wrong, hence my soapbox. Regina, also I should mention correlated observations also sometimes called dependent observations or dependencies in the data.

[Regina] (30:10 - 30:21)
Let's take this back to the sweaty T-shirt study. We got the preliminaries out of the way. We understand why it's important.

In this study, there were actually three sources of dependencies or correlated observations.

[Kristin] (30:21 - 30:22)
That's complicated.

[Regina] (30:23 - 30:37)
Complicated little creature there. And let's just walk through all three. First of all, first one.

Each woman rated three genetically exotic men and also three genetically similar men. So we have six observations for each woman.

[Kristin] (30:38 - 30:43)
We have replicates. So six rows of data per woman. How did the authors handle that?

[Regina] (30:44 - 31:06)
They collapsed those six numbers basically down to two numbers. So they took the three, her ratings for the three genetically exotic men and averaged those together and then took her ratings for the three genetically similar men and averaged those together. And so instead of six rows per woman, now we only have two rows per woman.

[Kristin] (31:06 - 31:10)
Got it. So that handles the correlated nature of the replicates. Collapses three rows into one.

[Regina] (31:11 - 31:21)
That's the first source of correlation. Three man replicates. Now the second source of correlation is that we are still stuck with these two rows per woman.

[Kristin] (31:21 - 31:22)
Right.

[Regina] (31:22 - 31:26)
Her average odor rating for each of the two groups in there.

[Kristin] (31:26 - 31:33)
Right. So this is a design where women are serving as their own controls because every woman rates both genetically similar and genetically exotic men.

[Regina] (31:38 - 31:39)
So they compared each woman to herself.

[Kristin] (31:39 - 31:45)
Oh, great. Okay. So they handled those two sources of correlation correctly.

What's the third source though, Regina?

[Regina] (31:45 - 32:03)
Yeah, this one's trickier. It's not as much fun. And this is because of how they designed the experiment.

The rows of the data are also connected by the man, not just by the woman. Because the men were each smelled by multiple women.

[Kristin] (32:03 - 32:04)
Yes.

[Regina] (32:04 - 32:07)
And it was not a balanced design.

[Kristin] (32:08 - 32:08)
What do you mean by that?

[Regina] (32:09 - 32:17)
Remember, there were 49 women and only 44 men. So they had to use some men more than others.

[Kristin] (32:17 - 32:18)
Right.

[Regina] (32:18 - 32:24)
Some men got sniffed more than others. There's a man shortage, which kind of sounds like some party.

[Kristin] (32:25 - 32:39)
Okay, so this is super complicated. This is making my brain hurt. With this weird design they have, you really would have to use a fancy statistical model in order to account for that correlation by man.

So did they use one?

[Regina] (32:39 - 32:42)
They definitely did not use a fancy statistical model.

[Kristin] (32:43 - 33:15)
So this is problematic. And, you know, Regina, to illustrate this, I'm going to make up an extreme case. Sometimes you have to go to the extremes.

It makes it easier to picture in your head. So let's imagine that in the sweaty t-shirt study there is one man who is just genetically very exotic. He's from Switzerland, but somehow he has a very unique lineage, very unique genetic profile, and he pops up as the most genetically exotic man for all 49 women.

So every single woman ends up smelling his t-shirt.

[Regina] (33:15 - 33:22)
So his rating is going to be very high.

[Kristin] (33:22 - 33:27)
Let's imagine also that he just smells really great. This is a good smelling man.

[Regina] (33:28 - 33:41)
Truffles. In my mind, he smells like truffles and like ocean. Chocolate, coffee, leather, tobacco, roses.

I love it. Let's create a perfume and make a fortune here.

[Kristin] (33:42 - 33:48)
Do we have to murder him to make this perfume? Like in the book? That book I'm still wondering about.

[Regina] (33:49 - 33:50)
We're just going to fantasize.

[Kristin] (33:51 - 34:16)
All right. Bottom line, he smells great. It has nothing to do with his genes or the match of the genes with these different women.

He is just an objectively good smelling guy and most women who smell him are going to rate him highly. But 8, 9, or 10 out of 10 basically. This means that every woman in the study has at least one high score in her genetically exotic pile.

He's averaged into the exotic scores 49 times more than anyone else.

[Regina] (34:16 - 34:19)
Outsized influence this one guy has.

[Kristin] (34:19 - 34:26)
Yes. This one man could be driving the entire finding. With him in the data set maybe you find a big difference between genetically exotic and similar men.

[Regina] (34:27 - 34:30)
Yeah, but if you were to drop him maybe it just goes away entirely.

[Kristin] (34:30 - 34:49)
Yeah, he could be driving the entire effect and obviously the issue is we are counting one man as if he represents 49 independent men. When we ignore the correlated nature of the observation we end up over counting him and this can skew the results. This is why they should have used a more sophisticated model here or even better a better study design.

[Regina] (34:51 - 35:07)
Of course I should clarify this is an extreme example probably not what happened Right, probably not what happened in the study. More subtle but it doesn't show why it's important in there and the point is they had a weird data structure that they did not really deal with.

[Kristin] (35:07 - 35:22)
This example is meant just to drive the intuition. We actually don't know because they weren't transparent. We don't know how many men were rated by how many women.

We don't know what effect this had on the results but we do know that they failed to account for something that could be important. Right, right. So Regina tell me more about the results.

[Regina] (35:22 - 35:35)
Right, the results. We're ready for the results. First of all remember they never reported on sexiness.

Right. They presented results just for how intense the body odors were and how pleasant they were.

[Kristin] (35:35 - 35:35)
Okay.

[Regina] (35:36 - 35:59)
And for intensity there were actually no significant effects at all. I know. So I'm just going to talk about pleasantness.

Okay. Simplifies things. They presented their results in a weird way but based on what I could see of the results it looks like overall there were actually no significant differences in how women rated the pleasantness of exotic or similar men.

[Kristin] (36:00 - 36:09)
Wait, wow. So no effect but why then does everyone cite this paper I guess including me if they didn't actually find a difference?

[Regina] (36:09 - 36:20)
I know. Good question. And the answer to that is to get the results that they highlighted you know the exciting results and the discussion they did two weird things with their data.

[Kristin] (36:20 - 36:24)
Uh-oh, I'm foreseeing some data shenanigans.

[Regina] (36:25 - 36:35)
Uh-oh is right. First one, they analyzed their data from two different perspectives meaning they looked at how women rated the smell of men in the two genetic groups.

[Kristin] (36:35 - 36:35)
Right.

[Regina] (36:35 - 36:37)
Right, that's the women's perspective.

[Kristin] (36:38 - 36:40)
That makes sense that's kind of what we talked about already.

[Regina] (36:40 - 36:49)
But they also analyzed the data from the man's perspective meaning they recalculated the averages for each man.

[Kristin] (36:49 - 36:51)
Okay, explain what do you mean by that?

[Regina] (36:51 - 37:17)
I know, I know. Let's do a hypothetical like you did before. Let's say one of the men was rated by five genetically exotic women different genes and three genetically similar women.

Sure. He would have now eight rows of data that belonged to him. So we're going to take those five exotic scores and then the three similar scores and compare those two averages just like we did.

So we kind of inverted the whole thing.

[Kristin] (37:17 - 37:22)
Right. Like the man is serving as his own control now. Oh, that is weird.

[Regina] (37:22 - 37:22)
It is weird.

[Kristin] (37:23 - 37:32)
Because I mean the study was set up about the women. It's the women smelling the men. This doesn't make a lot of sense and it's not going to give us the same answer as from the women's perspective.

[Regina] (37:33 - 38:18)
Exactly. And okay, ultimately the results were more exciting only in one direction, only from the man's perspective. Oh, I see.

So that's what they highlighted. Of course. I know, I know.

So that leads us actually into the second word. Oh, there's another one. There's another one.

They did not stop. They did something I found quite suspicious. They split the women into two groups.

Women on hormonal birth control, the pill, and women not on the pill. And then they analyzed each group separately, reported each group separately. And when you split the women into two groups like this, and only when you analyze it from the man's perspective, that's when you get the exciting results.

[Kristin] (38:18 - 38:26)
But the women are on the pill, not the men. So how do you do this from the point of view of the men? The men are not on the pill.

[Regina]
I know. I told you it was a mess.

[Kristin]
Well, first of all, what were the exciting results?

[Regina] (38:27 - 38:33)
Right. All right. Women not on the pill preferred the smell of men who had different MHC genes.

[Kristin] (38:33 - 38:35)
Okay, that was the expected result.

[Regina] (38:35 - 38:49)
Right. that. Yeah, went in the right way, but women on the pill had the reversed preference.

They preferred men with similar MHC genes, so that preference completely flipped. The opposite of what was expected.

[Kristin] (38:50 - 38:59)
All right, Regina, I'm wondering though, was this stratification dividing it up by women on the pill versus not on the pill, did they plan to do this ahead of time?

[Regina] (38:59 - 39:04)
Yeah, right. They were not transparent about it. Not surprising.

I suspect it was not.

[Kristin] (39:04 - 39:04)
Okay.

[Regina] (39:04 - 39:13)
They didn't mention anything about it in the intro section or citing literature about this. And also part of that though, the clue is the sample size.

[Kristin] (39:13 - 39:17)
Oh, what was the sample size out of the 49 women? And how many were on the pill?

[Regina] (39:17 - 39:20)
Yeah, 18 on the pill, 31 not on the pill.

[Kristin] (39:20 - 39:48)
Right, so I see what you're getting at because if they had planned this, this is an experiment and if they cared about pill versus not pill, they would have gone out and recruited about the same number in each of those groups to maximize the statistical efficiency. So this makes me suspicious as well, Regina. This is like you really have to twist the data through a lot of gymnastics to get some exciting result, which makes me think that it might not be robust.

And this is one of the things that you're supposed to do as a statistician. You're supposed to try to break your analysis.

[Regina] (39:49 - 39:49)
You are.

[Kristin] (39:49 - 40:03)
Throw it on the floor and see if it shatters because if your result only holds up when you twist the data in this very particular contortion, right, it's probably not robust. It's probably just a statistical artifact.

[Regina] (40:03 - 40:17)
It might be an artifact. I am wondering if there were some post-talk analyses going on. And by post-talk, after the fact, meaning they were not pre-planned these analyses.

Maybe when they analyzed everything as this one big group, nothing came out significant. Right.

[Kristin] (40:17 - 40:28)
I mean, this is what I'm thinking too. This is what might happen. We didn't find anything, so let's start twisting our data and looking for something.

Can we pull something out of the data?

[Regina] (40:28 - 40:31)
Split it by pill and not on the pill.

[Kristin] (40:31 - 40:34)
Right. It often ends up to be just a statistical artifact.

[Regina] (40:35 - 40:39)
Something that is just a fluke in there. And of course we're going to drop sexiness.

[Kristin] (40:41 - 40:54)
I'm really curious about the sexiness variable, that rating, because why did they drop it? They gave us the intensity one, which was not significant. Was it like the sexiness went in the opposite direction or something so they couldn't make a good story?

[Regina] (40:55 - 41:00)
We will never know. That was 1995. But we know how these things go.

[Kristin] (41:00 - 41:09)
Right. We don't actually know what happened behind the scenes. We don't know what the authors did.

But the two of us have enough experience that we can kind of guess that something like this probably went on behind the scenes.

[Regina] (41:09 - 41:13)
We know what happened in the dark recesses of the research basement.

[Kristin] (41:15 - 41:16)
Shenanigans. It's not pretty.

[Regina] (41:17 - 41:26)
So I think that leads us very nicely actually to talking about their conclusions and their discussion section. And you are going to get a kick out of this.

[Kristin] (41:26 - 41:41)
I can't wait. But first a short break.

[Kristin]
Welcome back to Normal Curves.

We are talking about the Sweaty T-shirt study and we were about to hear the conclusions of that study.

[Regina] (41:42 - 41:43)
I think you're going to like this, Kristin.

[Kristin] (41:44 - 41:44)
Really?

[Regina] (41:44 - 41:45)
By like I mean hate.

[Kristin] (41:46 - 41:48)
So I might have some criticisms.

[Regina] (41:48 - 41:55)
Maybe a couple. Let me read you the first line of the discussion section: The contraceptive pill seems to have a strong influence on odor preference.

[Kristin] (42:00 - 42:27)
Oh wow. So that is an overstatement. They actually don't have data to support that statement and this is a little subtle but they haven't compared the women on the pill directly to the women not on the pill which is the analysis you would need to do to make that statement.

It's a little bit of a subtle statistical point. It's called a test of interaction. We'll discuss it in a future episode maybe but suffice it to say they don't have data to support that and it's the first sentence of their discussion.

[Regina] (42:27 - 42:39)
The first sentence. Okay next line. This indicates that steroids which are naturally released during pregnancy could change body odor preferences leading to a preference for odors which are similar to those of relatives.

[Kristin] (42:40 - 42:52)
Okay wait a minute. How did we get to pregnant women? I thought the women in the study were not pregnant.

They were not pregnant. They were on the pill but being on the pill is not the same as being pregnant and I can tell you that from personal experience.

[Regina] (42:53 - 43:02)
They're going off there. Then they go off on pregnant mice actually. They have a few lines about pregnant mice and what the pregnant mice like to do.

[Kristin] (43:02 - 43:16)
So we have wandered from women on the pill all the way over to pregnant mice. This is what I get really upset about more often when I'm reading discussion sections authors wander too far from their data. They are like wandering out into the wilderness here at this point.

[Regina] (43:16 - 43:19)
Yeah. I think these people might be galloping into the wilderness.

[Kristin] (43:21 - 43:24)
Way far from their data. Straying from their data. Yes.

[Regina] (43:24 - 43:36)
Okay. Then they go on to say, therefore, the contraceptive pill seems to interfere with natural mate choice. Oh, no way.

Wow.

[Kristin] (43:36 - 44:18)
It's a good one, right? This is a little anti-pill here. Is this like their political agenda behind this?

[Regina]
Pills are destroying the very fabric of marriage. Well, and also those women today.

[Kristin]
Mate choice was not an outcome here.

It was how pleasant the men smelled. So we've gone from the pleasantness of the men smell, not even sexiness, to mate choice. Because we're all running around saying, oh, that man is pleasant, I'm going to marry him.

[Regina]
He has a pleasant smell, so that's enough. Right. But yet, this study gets cited all the time.

[Kristin]
Right, you said it's widely cited, and I apparently might have cited it myself when I said that studies show. Please tell me there's other data that support my lying studies show, or did I just get it wrong?

[Regina] (44:18 - 44:35)
First of all, I did the same thing. I referenced this paper when I was writing about the pheromone dating parties for the LA Times. So you and I are both doing a little mea culpa here.

There have been follow-up studies, but before that, I'd like to dwell on the splash that this made.

[Kristin] (44:35 - 44:37)
Did this get some media attention?

[Regina] (44:38 - 44:42)
It did. In fact, there was one in New Scientist by our colleague Peter Aldhous.

[Kristin] (44:42 - 45:04)
Oh! Peter was my professor at UC Santa Cruz in the science writing program, although I completely cut his class.

[Regina]
You cut his class?

[Kristin]
Yeah, I was on a boat in Antarctica for that quarter, so I skipped his class entirely. That is such a great story. We're going to have to save that one for later.

[Kristin]
Okay, we'll talk about that, but question for you. Was Peter credulous or skeptical of this study when he wrote about it?

[Regina] (45:05 - 45:13)
Credulous, I'm afraid. Let me read you his lead. Women are attracted to the odors of men who have different immune system genes from their own, but only if they're not taking the pill.

[Kristin] (45:13 - 45:17)
Okay, so he bought it hook, line, and sinker. So he didn't miss anything by cutting his class.

[Regina] (45:19 - 45:21)
Next time I see him I'm going to tell him that.

[Kristin] (45:21 - 45:22)
I'm kidding, I'm kidding.

[Regina] (45:23 - 45:31)
But he did add this bit of information, investigative journalism. I think you'll like it. He was talking about the reversal of preferences on the pill.

[Kristin] (45:32 - 45:36)
Oh, where the women on the pill like the similar men better than the dissimilar men, yes.

[Regina] (45:36 - 45:40)
Here's his quote, Wedekind is still mystified by this result.

[Kristin] (45:41 - 45:48)
I am mystified by that quote because in his discussion section he sounds pretty sure of himself. He did not sound mystified. interesting.

[Regina] (45:48 - 45:56)
Yeah, so it just adds to my suspicion, they didn't plan this. Right. Because if they had planned this pill then why would they be mystified by the result?

[Kristin] (45:57 - 46:04)
If he was mystified by it he clearly didn't expect it ahead of time so it wasn't a preplanned analysis. Yeah, adding to our suspicion that it's post hoc.

[Regina] (46:04 - 46:08)
At least Peter didn't gush though. Here's one story in the news section of an ecology journal.

[Kristin] (46:09 - 46:09)
Okay.

[Regina] (46:09 - 46:10)
And they were downright giddy.

[Kristin] (46:10 - 46:10)
Oh really?

[Regina] (46:11 - 46:25)
Let me read you a quote: A mate chosen by a woman on the pill might smell sweet at the time but when she goes off the pill to reproduce she might likewise go off his scent. Could the rise in divorce rates be correlated with mate choices made on the pill?

[Kristin] (46:26 - 46:42)
The rise in divorce rates? Wow. So we've gone from a few women on the pill and this weird analysis to, now we can explain why divorce is on the rise.

Wow, that is wandering very far from the data. Yeah, they're like ocean maybe. Yeah, out in the ocean in the middle of Antarctica.

[Regina] (46:45 - 46:48)
Luckily, not everyone loved it. Oh, I'm glad to hear that.

[Kristin] (46:48 - 46:48)
Yep, yep.

[Regina] (46:49 - 46:54)
A year later, there were two researchers who wrote a letter to the editor in the Ecology Journal.

[Kristin] (46:54 - 46:54)
Okay, fine.

[Regina] (46:54 - 47:26)
Phil Hedrick and Volker Loeschcke, and they picked up on some of the same things we did.

[Kristin]
Oh, yay!

It was really good. They said, experimental design, data need more scrutiny, sample size is too small. They also picked up, like we did, on how the results differed depending on whether you're using it from the point of view of men or women.

Yeah, that weird thing, yep. And they said, you need to explain that. And they pointed out, you know, humans are not mice.

Right, yes, true, true. Though with mice, it's easy because you can have a clone and you can manipulate genes.

[Kristin] (47:27 - 47:37)
So making inferences based on the mice studies doesn't make a lot of sense. Well, that's great. I know.

Did people read their letter to the editor? And, well, clearly not, since this is a widely cited study.

[Regina] (47:38 - 47:38)
I know.

[Kristin] (47:38 - 47:41)
I'm guessing it kind of got buried, as sometimes happens with criticisms, yep.

[Regina] (47:41 - 47:47)
You know who did read it in response, however, was Wedekind and one of the other authors of the T-shirt study.

[Kristin] (47:47 - 47:48)
Right, I imagine they weren't happy about it.

[Regina] (47:48 - 47:59)
Yeah, they were not happy. It's kind of funny to read. Reading between the lines, not a quote, but they kind of accused those letter writers of fundamentally misunderstanding statistics and science.

[Kristin] (47:59 - 48:02)
Well, they criticized us. So, of course, they misunderstood everything.

[Regina] (48:03 - 48:19)
And Wedekind defended their study design, which doesn't really have a lot in its defense. Right, exactly, yes. And they seemed a little miffed at how the letter writers were questioning their data analysis.

So they said, OK, we're going to send you our data so you can reanalyze it. They sent the data.

[Kristin] (48:20 - 48:24)
I know. Did the letter writers ever do anything with that data? Can we get a hold of that data?

[Regina] (48:24 - 48:34)
No, I couldn't find that they published on it. I searched and searched, so I emailed them.

Oh, you did? I emailed them. Oh, wow.

They're both emeritus faculty now.

[Kristin] (48:34 - 48:35)
Right, this was a while ago.

[Regina] (48:35 - 48:38)
Oh, this was 1996 when you and I met.

[Kristin] (48:38 - 48:39)
Did they remember this?

[Regina] (48:39 - 48:48)
Oh, they did. One's in Arizona, another is in Denmark. And I said, hey, you know, it's 30 years later and all, but great letter to the editor back in 96.

[Kristin] (48:48 - 48:58)
You know, I think if I'm an emeritus professor someday and somebody says, that article you wrote 30 years ago, I loved it. I'd be happy.

[Regina] (48:58 - 49:10)
And it was genuine because it was really well done. And I said, and then, by the way, do you still have that data that we were going to send you?

[Kristin]
Did they have the data?

[Regina]
They did not have the data. One said, oh, I don't remember if we analyzed it or couldn't figure out how to analyze it.

[Kristin] (49:10 - 49:11)
Oh, they don't remember.

[Regina] (49:11 - 49:39)
And the guy in Denmark is like, oh, I'll go look for it. I mean, it's 30 years later. They didn't have cloud storage then.

[Kristin]
They didn't have cloud storage back then, yeah.

[Regina]
So we ended up having this really nice email conversation on a Saturday night, you know, talking about, it was interesting, the geneticists, so talking about where this whole field has gone since then and what Wedekind has been up to. So he published two more studies on the topic, but mostly he works on fish biology now, apparently.

[Kristin] (49:39 - 49:41)
Oh, well, maybe that's a good thing.

[Regina] (49:43 - 49:50)
So they sent me the two papers, the 1997 paper was kind of a replication.

[Kristin] (49:51 - 49:53)
Oh, well, good for him for trying to replicate the findings.

[Regina] (49:53 - 50:01)
Yes. OK. Study design, a little different.

OK. He worked with six t-shirt wearers. Only six this time.

[Kristin] (50:01 - 50:01)
Only six.

[Regina] (50:01 - 50:03)
A mixture of males and females.

[Kristin] (50:03 - 50:07)
Oh. Are we smelling both ways then? Men smelling women, women smelling men?

Yep.

[Regina] (50:08 - 50:09)
121 sniffers.

[Kristin] (50:09 - 50:10)
OK. OK.

[Regina] (50:10 - 50:11)
Also a mixture of male and female.

[Kristin] (50:12 - 50:12)
Oh, interesting.

[Regina] (50:12 - 50:20)
OK. And used some of the same t-shirt wearers and snippers as in his 95 paper, his original paper.

[Kristin] (50:20 - 51:03)
Oh, some of the same study subjects. Some of the same people. Yes.

Oh, that is a red flag for me. I always worry about when I see that because that makes me think that they're on the friends, family, staff and grad students plan, meaning, hey, I'm just going to pick study subjects from whoever I can find sitting, you know, at the desk next to me. And that's really problematic because that is not the way that we should select study subjects.

The people who are working in the lab, for example, the grad students, they may have a vested interest in how those results come out. And also, it's just not a really good generalizable sample. It's not a representative sample.

No. So don't like that right away.

[Regina]
So not only that, but it had a lot of statistical issues as well.

Same, similar to the other paper.

[Regina] (51:03 - 51:10)
Yeah. So just want to give one example. They found no significant effect for T-shirts being rated by women on the pill.

[Kristin] (51:11 - 51:11)
OK.

[Regina] (51:11 - 51:19)
No significant effect for T-shirts being rated by men. No significant effects for T-shirts being rated by women not on the pill.

[Kristin] (51:19 - 51:22)
So none of it replicated then is what you're telling me.

[Regina] (51:22 - 51:28)
But why stop there, Kristin? Just because you have three non-significant results doesn't start combining subgroups.

[Kristin] (51:28 - 51:28)
Oh, no.

[Regina] (51:28 - 51:36)
Yeah, right. Because if you combine the men and the women not on the pill, then you get significance. But wait a minute.

[Kristin] (51:36 - 51:45)
Why in the world would you combine men with women not on the pill, like women who are menstruating and ovulating? How is that combinable with men?

[Regina] (51:45 - 51:50)
Right. If anything, they'd be combining the women or maybe the women not on the pill with it.

[Kristin] (51:50 - 51:53)
That's weird, yes. That's the only way they could get significance, I guess.

[Regina] (51:53 - 51:56)
I know. They dropped sexiness. Oh, again, no sexiness.

[Kristin] (51:57 - 51:58)
Oh, all the same problems.

[Regina] (51:58 - 52:03)
The analyses had even smaller numbers, smaller sample size, just a little bit of a dumpster fire.

[Kristin] (52:03 - 52:14)
It sounds like it. Yes. Okay.

So it really, that's their quote-unquote replication. But if you're hoping to make the whole thing more robust by pointing to that replication, you've gone awry.

[Regina] (52:15 - 52:15)
Okay.

[Kristin] (52:15 - 52:22)
But I'm sure that other people have tried to replicate this result since 1995. What are some of the other studies that have been done on this topic since then?

[Regina] (52:22 - 52:33)
A number of people have tried to do this over the 25 years. There's so many of them that now people are able to do a meta-analysis where they combine all the results. So do you want to explain a meta-analysis?

[Kristin] (52:33 - 52:54)
Sure. So meta-analysis just means you take a bunch of papers and you're just taking the summary data from those papers, not necessarily the underlying original data, and combining it to try to get some sense of the overall effect. So maybe one study found something, another didn't, but science isn't just about one paper.

So let's pool all that data and see overall, do we find that women prefer the genetically dissimilar men?

[Regina] (52:54 - 53:16)
Mm-hmm. So there was a really nice one in 2020, a big meta-analysis, odor preference and genetics, and they found 10 nice studies to include. And what did they find?

First, I want to give a nice call-out to them because this Wedekind 1997 replication paper— It was included in this meta-analysis. It was included, but they didn't fall for this combining the subgroups thing.

[Kristin] (53:16 - 53:21)
When they extracted the data, they kept the women on the pill or not on the pill separate from the men.

[Regina] (53:21 - 53:25)
Mm-hmm. And they had to use a little digital tool to be able to get the results out.

[Kristin] (53:25 - 53:28)
Oh, they extracted the data from those bar charts, too. Yes.

[Regina] (53:28 - 53:29)
So it was well done. I just wanted to tell them.

[Kristin] (53:29 - 53:37)
Right. So we have 10 studies in this meta-analysis. Two of those are the Wedekind studies, the 1995 and the 1997, but they are calling the 1997 results not significant.

[Regina] (53:37 - 53:38)
Mm-hmm.

[Kristin] (53:38 - 53:38)
Right.

[Regina] (53:38 - 53:38)
Yep.

[Kristin] (53:38 - 53:40)
Okay. They pooled the data. What did they find?

[Regina] (53:40 - 53:42)
Yeah. Nothing. Nada.

[Kristin] (53:43 - 53:45)
No evidence. So no finding. No evidence for this claim.

[Regina] (53:45 - 53:51)
The only significant effect out of those 10 studies was Wedekind 1995, that original study.

[Kristin] (53:51 - 54:02)
That was it. So that original study found a significant effect. The 1997 study, they're calling it out and saying, actually, that was a failure to replicate as we looked at it.

And then eight other studies found nothing.

[Regina] (54:03 - 54:08)
Nothing.

[Kristin
So there's nothing here.

[Regina]
There's nothing here. I'm really sad about this. I'm kind of crushed.

[Kristin] (54:08 - 54:09)
You wanted this to be true. I did.

[Regina] (54:10 - 54:10)
Yeah.

[Kristin] (54:10 - 54:11)
This is such a fun result.

[Regina] (54:11 - 54:11)
Yeah.

[Kristin] (54:11 - 54:19)
People love to talk about it. But that first study was just so bad that it's not surprising that it didn't replicate, actually.

[Regina] (54:19 - 54:20)
They got lucky.

[Kristin] (54:20 - 54:21)
Yeah.

[Regina] (54:21 - 54:24)
And they published that and people could not replicate.

[Kristin] (54:24 - 54:35)
Now, we're not saying that smell doesn't influence mate choice, right? Our claim that we're looking at today was that it's mediated through these HLA genes. But, of course, people still do choose mates based on smell.

[Regina] (54:35 - 54:46)
Mm-hmm. Mm-hmm. In fact, some studies suggest...

Uh-oh. So we don't know if we're right or not, but studies suggest that people are doing this, that our body odor is reflecting things like our personality.

[Kristin] (54:47 - 54:47)
Really?

[Regina] (54:47 - 54:50)
Our mood at the time. Yes. Our health, our general health.

[Kristin] (54:50 - 54:52)
It certainly reflects, like, did you eat garlic last night?

[Regina] (54:53 - 54:57)
But even beyond that. So maybe we can do an episode on that.

[Kristin] (54:57 - 55:37)
We should. Yeah. We might want to dig into some of those studies that suggest.

See if they're real, but I'd be curious to do that. All right, Regina, this is the part of the episode where we're going to wrap everything up and we're going to rate the strength of the evidence for the claim that we're looking at today. And the claim is that women prefer the smell of men who are genetically dissimilar to them.

And how do we rate strength of evidence on this podcast? We use our trademarked, highly scientific smooch rating scale. One to five smooches, kind of like Amazon stars.

One smooch means little or no evidence for the claim. And five smooches means very strong evidence for the claim. What say you, Regina?

Kiss it or diss it?

[Regina] (55:38 - 55:53)
I'm going to need to diss this one. Diss it. I'm going to give it one smooch.

I am so disappointed in this original study. It was way weaker than I had expected or hoped. And there's just nothing there.

[Kristin] (55:53 - 56:15)
One smooch. Yeah. I have to agree with you, Regina.

This is a clear one smooch from me, too. This is so widely cited that you would think that there's something behind it. So it's really disappointing that this study is so bad.

And it hasn't been replicated at all despite attempts to do so. Yeah. So I'm going one smooch, no evidence.

Maybe this could be true in mice, but not humans. Right.

[Regina] (56:15 - 56:20)
I might still use it as a screening tool for my men on dates. But nothing to do with the genetics.

[Kristin] (56:21 - 56:48)
You might still throw a pheromone party.

[Regina]
I might.

[Kristin]
But not to do with the genetics necessarily.

The other thing we like to do in this podcast is because we're not just talking about this claim that we're looking at today. We're talking about how to evaluate evidence for a claim like this. And so we'd like to give some methodologic morals.

And these are a little like Aesop's fable morals. So, Regina, do you have a methodologic moral for us today?

[Regina]
I have many, actually.

[Kristin]
Oh, I'm going to let you have more than one today.

[Regina] (56:49 - 57:00)
So I'm going to pick one. It's a small point in today's story, but it's something I really want people to remember. So repeat after me.

Bar charts are not for numerical data.

[Kristin] (57:00 - 57:03)
Bar charts are not for numerical data. Got it.

[Regina] (57:03 - 57:06)
Good job. Good job. Can I have a sub moral for that one?

[Kristin] (57:06 - 57:07)
Of course. Yes.

[Regina] (57:08 - 57:24)
Always report your summary statistics. Don't make readers guess by looking at your crappy charts. Oh, hallelujah to that one.

Oh, very nice. Can I have one more?

[Kristin]
You get one more today. Absolutely.

[Regina]
Okay. Don't go ranting about off-topic crap in your discussion.

[Kristin] (57:24 - 57:34)
Oh, yeah. Agree with that one too, Regina.

[Regina]
Okay. You?

[Kristin]
So mine is going to be those who ignore dependencies in their data are destined for flawed conclusions.

[Regina] (57:34 - 57:35)
Oh, I love it.

[Kristin] (57:35 - 57:44)
Yeah, it's true. Something to always keep in mind when you're analyzing data. Dependent data.

Well, this has been a fascinating episode. I've learned so much. Thanks so much.

[Regina]
Thanks, Kristin.