Maybe you heard about that study that showed men don’t want to date smart women. It certainly plays into our stereotypes of how men behave, and it got a lot of press as a result. But if you actually dig into the paper, you’ll find there are some serious problems with those results.
Let’s take a closer look.
Lead author Lora Park of the University at Buffalo and her colleagues were testing how a concept called psychological distance affected the glorious mess that is human attraction. The general idea is that people tend to judge humans they don’t know against abstract ideals, but rate people they have actually met against more concrete evidence, like the way those people make them feel.
The group had preliminary data that suggested most young men would be happy to date a woman who was smarter than they were — at least in the abstract. The question was whether that would remain true when the woman was right in front of them, instead of an imaginary ideal.
To find out, the researchers recruited single college-aged heterosexual men, manipulated their perception of how they fared on an exam relative to another person, then asked how interested they would be in spending time with that person. In some experiments, the other person didn’t actually exist; in others, the part was played by a confederate who was helping with the study. But in each variant, the young men were given a set of questions to complete, and then told they had done either better or worse on the exam than the person that they were expecting to talk to.
In their first experiment, they asked 151 men to take a short maths test — which they were told measured intelligence — at the same time as a partner in the other room. (The partner didn’t actually exist.) After the test, the men learned whether they had been paired with a man or a woman, and whether their partner had done better or worse then they had on the exam. When asked to rate how much they’d like to interact with their partner, they showed more interest in meeting the women who had outscored them than either men with higher scores or anyone with a lower score. That matched the data Park and her colleagues had collected in their pilot study.
In their second experiment, 81 men took the test in the same room as a young woman. After the test, an examiner announced aloud how well each of them had done on the exam. After hearing the scores, each man was asked to rate the attractiveness and desirability of his partner and move his chair so he could sit facing her for a simulated date. The researchers used the distance between the chairs as a measure of how interested he was in her.
This time, differences in test scores didn’t really affect how men rated their partner’s physical attractiveness, but men tended to sit further away from women who had outscored them and were less “romantically interested” in them. But men had to respond positively to two questions before the researchers would count them as “romantically interested.” That was true for 20% of the men in the study, but we’re not told how many men answered yes to one question but not the other, which means we have no way to judge whether romantic interest was expressed in other ways.
In a slightly different version of this experiment, men were simply asked whether they wanted to interact with their partner instead of scoring them romantically, and some men never learned what their partner scored. This made it even more complex than the other experiments, because instead of two, there were three possible situations: “woman gets higher score,” “woman gets lower score,” and “woman’s score remains unknown.”
This time, Park and her team reported that men were less interested in interacting with a woman who had outscored them than when she had either a lower score or when her score was unknown. Men also placed their chairs further away from women with higher scores than when they only knew their own score. But the researchers also found no significant difference in where men placed their chairs when they were talking with women who had either higher or lower scores than their own. So, sometimes they didn’t care when she appeared more intelligent, and sometimes they did?
The last two experiments muddy the waters still more. These trials varied whether the woman taking the test was in the same room as the participants or out of sight in the next room. In one trial, the researchers found that the men they tested were less interested in interacting with the woman when she outscored them if they were in the same room during the test. Outscoring him when she was in the other room had no effect on his interest.
In the second trial, when the men were in the same room as the woman, her scores had no effect on their interest in her, but when she was in a different room, men were more interested in her when she appeared more intelligent.
Confused yet? It’s a pretty inconsistent picture. And that’s probably the effect of the study’s limited datasets. For example, one part of the second experiment tested 81 men, but when you split them up by whether they ‘scored’ higher or lower than their female partner, there are really only about 40 men assigned to each situation. And the second part of that experiment tested 73 men, who were divvied up three ways. That takes an already small dataset and makes it tiny.
The third experiment has a similar problem, because it takes a modest number of men (71 for the first version, 134 for the second) and divides them up four ways. In all, Park’s datasets are small and statistically underpowered. And that means positive results can pop out of a small group that would get washed out as random variation in a larger group.
That’s bad enough, but the paper also engages in the wince-worthy description of what they call “marginally significant” or “near significant” data. Especially in datasets these small, those should be classed as “not significant” (and I’ve treated them as such in my description).
Worst of all, since the experiments recreate the horrid feeling of publicly learning how badly you did on a test (and since every man in the experiment was told he got 12/20 correct, he didn’t exactly get a great result), it’s not entirely clear whether it actually affected the way men felt about the women in the trial, or instead made them embarrassed about themselves, possibly making them think she was out of their league.
Park and her team touched on this in their last set of experiments by asking their subjects to fill out a survey where they rated themselves against stereotypically masculine traits. In the first, smaller test, they found that when the men were tested in the same room as the woman they felt less masculine, and were less interested in interacting when she outperformed them. The effect evaporated in the second, larger trial, along with the apparent rejection of intelligent women.
Image via Miramax