If you know science, you know it isn’t perfect. Outside eyes review papers their peers wrote before those papers get published, results must be reproduced to establish truths, and even then, stuff can still contain mistakes.
That’s the “X” in arXiv (Image: Ryan F. Mandelbaum/NASA/NSF)
Physicists are now discussing a new paper published on the arXiv preprint server last week. Papers landing on the arXiv have been moderated but not reviewed by others, so it isn’t clear whether they’re error-free or even worthy of publication in a scientific journal. This makes a new Forbes column from theoretical physicist and science blogger Sabine Hossenfelder especially interesting — the public got to see the reality of science through live peer-review.
Hossenfelder dug up a new analysis of the Laser Interferometer Gravitational Wave Observatory (LIGO)’s now-famous gravitational wave events and wrote her column up last Friday. She explains the paper’s findings:
A team of five researchers — James Creswell, Sebastian von Hausegger, Andrew D. Jackson, Hao Liu and Pavel Naselsky — from the Niels Bohr Institute in Copenhagen, presented their own analysis of the openly available LIGO data. And, unlike the LIGO collaboration itself, they come to a disturbing conclusion: that these gravitational waves might not be signals at all, but rather patterns in the noise that have hoodwinked even the best scientists working on this puzzle.
Essentially, LIGO’s two observatories detect gravitational waves by splitting a laser beam, sending it down several kilometre-long tunnels, and merging the light waves together again. A passing gravitational wave changes the shape of space by a teeny amount, enough to move the laser beams’ light waves in and out of alignment with one another. But the movement is barely perceptible, a shift smaller than an atom. So any amount of noise, even folks running at the site, can be detected.
The new paper says that the two detectors (one in Louisiana and another in Washington) could have heard noise at the same time interval that a gravitational wave would have taken to pass between them, thus clouding the signal. This would cast doubt on the robustness of the two gravitational wave events they looked at, one that occurred in December of 2015 and another in January of 2017.
But this paper wasn’t peer reviewed, and Hossenfelder points that out. The Danish team’s analysis wasn’t as robust as LIGO’s, either. Hossenfelder even gets a source from LIGO into her article — the experiment is familiar with the Danish team’s work. The LIGO source told Hossenfelder that they “respectfully respond that [they] have talked at some length with the group in the past and do not agree on the methods being used and thus with the conclusions”.
I reached out to Salvatore Vitale at MIT who works at LIGO as well to see if the new paper holds any water. “No,” he said, “they screwed up basic things,” listing some specifics of their analysis. The LIGO collaboration ultimately responded via physicist Sean Carroll’s popular blog, Preposterous Universe, in a guest post by postdoctoral physicist Ian Harry, which you can read here.
People took this several ways. One physicist blogger, Mark Hannam at Cardiff University, was upset that Hossenfelder covered the arXiv paper. Others were happy to get to see the scientific process unfold in real time.
The truth is, science writers cover arXiv papers all the time. This is one of the places where ideas get discussed in physics, and it prevents researchers from having to wait around for months before results get published. Science writers should know that arXiv papers require a lot of outside eyes to properly report. Hossenfelder does — her column ultimately ended with the requisite heavy dose of scepticism towards the Danish group.
Another astrophysicist, Jason Wright from Penn State, summarises his own opinions in his own blog:
As I said, if I were the LIGO team I would be annoyed by the episode, but the things that would annoy me the least are that the world showed an intense interest in my work, that I had to explain my science to that interested audience, and that I got to show up a gadfly on a big stage.
Basically, he’s saying that physicists questioned a huge, famous, probably future Nobel prize-winning science organisation’s results. Scientists are supposed to question one another’s results. Now, we got to see scientists take a hard look at a criticism and peer review it publicly.
And if Harry’s post on Carroll’s blog isn’t convincing, Carrol points out that “Happily, there are sufficient online tools that this is a question that interested parties can investigate for themselves”. Essentially, the more eyes on the LIGO data, the better.
Anyway, that’s what’s happening in physics this week. It’s messy and you got to see what normally happens over emails and behind-the-scenes play out in real time. Maybe you liked it, maybe you didn’t, but this is what it looks like.
[Forbes]