There’s a formula to how you experience science: First, splashy press releases come out about a study, like ducklings being capable of abstract thought. News outlets write about said study (including some of the best science writers). Many of those writers will vet those studies with several scientists to ensure they are accurate. You will read the story, think “hmm!” and then live the remainder of your life thinking that ducks are “smart like apes.”
Image: Antone Martinho
But the story should never end there.
Scientists are now debating that duckling study, which was published in the journal Science last summer. The charges might sound minor — some researchers took issue with the statistics and methods, and thought some conclusions were too strong. But science is more than just groundbreaking discoveries, and lots of the action occurs after other researchers in the field have had time to react to an initial, thought-provoking paper. So what’s the story now? Well, maybe ducks aren’t as smart as we thought. But maybe they are.
“We rarely ‘know’ anything in science for sure,” Jean-Michel Hupé, a researcher at the Brain and Cognition Research Center at the University of Toulouse in France, told Gizmodo.
The first paper, if we’re being honest, had news headlines written all over it. A team of researchers from the University of Oxford exposed the most adorable newborn ducklings to a pair of objects that were either the same or different shapes, and a pair of objects that were either the same or different colour. Baby ducks playing with blocks! Then, they exposed the ducklings to another set of shapes. “The original paper was this somewhat surprising finding, even to us,” Antone Martinho, one of the original study’s authors from the University of Oxford, told Gizmodo. Ducklings exposed to objects of the same shape seemed to continue to prefer objects of the same shape.
“If you imprint them on identical red spheres and give them a choice of two blue spheres or a purple and an orange sphere, they prefer the two blue spheres,” said Martinho. “They preferred sameness.” But conclusions like that aren’t so exciting, so naturally, the story you hear is that ducklings are actually geniuses, when really, with one paper all we get are statistics on where some ducklings walked.
Image: Antone Martinho
Once the paper ran its course in the news, other researchers like Hupé took a look at it, too. Science is one of the most important academic journals, after all. “Because Martinho and Kacelnik were providing the data, I decided to use them for an exercise of statistics (I was preparing my teaching for Master students in September),” said Hupé.
He reanalyzed the data in a way he thought would be more accurate, and found the P-value, a number that shows how likely the result could have been gotten by chance, was way higher. In other words, he thought their results were much more random and less “statistically significant” than the initial analysis. “I discovered the issues right away when reanalyzing the data.” Science published his comment today.
Who knew ducklings could be so controversial?
Other scientists took issue too, essentially saying that the experiment’s results weren’t as robust as the authors claimed. “For many ducklings, the distribution of approach behaviour is simply random and provides no information about real preferences,” wrote scientists Jan Langbein and Birger Puppe from the Institute of Behavioural Psychology in Dummerstorf, Germany. They also thought that newly hatched duckies would simply pick the block set that would be most “beneficial for survival,” and Science published their comment today, too.
Martinho heard these comments soon after the paper came out, and had what he said was “productive scientific discussion” with all of the commenters.”We’ll be performing the experiment again soon and a lot of [our] decisions did grow out of the productive comments the commenters made,” said Martinho. But he stands by their original paper. “We’re confident in the results and had a productive discussion with both sets of commentary.”
Image: Antone Martinho
Hupé agreed that the conversation was cordial, even friendly, but worries that others have already cited the paper without any doubts and without waiting for more data to come out. “Published results, especially in Science, have authority,” he said.
All this is to say that science keeps happening, even after we put out exciting stories with loud headlines begging you to read them. Sometimes, no matter how adorable a result might be, we need more data before we can establish anything a truth. And maybe scientists need to look at more rigorous mathematical ways to determine that they have proven something.
“A revolution is needed,” said Hupé, “which should lead to probably about 10 times fewer papers published.”