Science and medicine are supposedly based on rigour — a rigour where theories are only correct if you can replicate results. It turns out, though, that the software used to analyse medical images of your brain gives wildly different answers if it’s run on Mac or PC.
The finding is completely crazy and casts doubt over the way many scans — such as CT and MRI — are analysed in routine practice.
A team of German researchers took data from 30 brain scans and analysed them using a package called FreeSurfer — one of the major medical image analysis programs used to measure the size of different parts of the brain.
They ran the software on Windows and on Macs running different versions of Mac OS, each time using the software to measure the size and thickness of various structures of the brain. They found that not only did Windows and Mac OS installs of the software throw up different results — but the version of Mac OS also had an impact.
Across most sections there was at least a 2-5 per cent variation in the answers. But in the parahippocampal and entorhinal cortex, the answers diverged by as much as 15 per cent. A 15 per cent variation just because of a Mac OS update. The results appear in PLoS One.
Sadly, it’s not clear why the problem arises. Indeed, that’s something medical professionals and software developers need to work on — and quickly. It’s a terrifying to think a single piece of software that can be used to measure parts of the brain — including tumours — can throw up different results after a simple update.
That means that not only might different hospitals choose to treat patients differently, but the same hospital could in theory change its diagnosis as a result of an IT upgrade.
As NeuroSkeptic points out, it would be wrong to lay the blame solely at the feet of the software developers behind FreeSurfer, because this is probably a widespread problem across other imaging platforms too. But regardless of whose fault it is, something needs to happen to address the problem. [PloS One via NeuroSkeptic]