Not long ago, decrypting DNA was an expensive undertaking that could run into the hundreds of thousands of dollars. Now, for $200 you can spit into a test tube and find out about your ancestry, your risk of developing Alzheimer’s, and even how likely you are to smell asparagus in your pee.
But how accurate are those results?
A new study published by the diagnostics company Ambry Genetics in the journal Genetics in Medicine casts some doubt on the testing methodologies of direct-to-consumer genetic tests like those offered by AncestryDNA and 23andMe, though it doesn’t call out any specific companies in its paper. Ambry, which processes DNA for doctors and research institutions, looked at variants that consumer test results identified that were associated with a risk of disease, and found that as many as 40 per cent of those test results were false positives. In other words, a consumer test indicated that a troubling genetic variant was present in a person’s DNA, but when Ambry’s genetic sequencing lab analysed samples from the same people, they found that it wasn’t.
Now, before you go discounting everything you’ve learned from DNA testing, there are a few crucial caveats here. The first is that the study authors were looking at the raw data from consumer testing companies, not just the information that those companies provide to consumers in all those colorfully illustrated reports.
At present, 23andMe only offers genetic health risk reports for a handful of conditions, and in order to provide those reports the company had to go through a lengthy FDA approval process to demonstrate that the results of the testing methodology could be reproduced 99 per cent of the time by testing methods used in clinical labs. But if you download your raw data from 23andMe, there is information on a lot more genes than 23andMe mentions in its reports. And third-party companies like Promethease offer to take that raw data and tell you a lot more. You can even take your raw data from companies like AncestryDNA that don’t even offer health analysis, and receive information about your health through a different service.
Ambry Genetics is a clinical testing lab, and often when patients go to their doctor afraid that they are at risk for something like cancer because of a variant detected in their raw data, doctors will ask Ambry to confirm those results.
“As all these samples were coming, we through noticed a lot of samples wound up being negative,” Stephany Tandy-Connor, a genetic counselor and lead author on the study, told Gizmodo. “We wondered how extensive that really was.”
It’s not that 23andMe is telling consumers that they’re at risk for cancer when they’re not, said Tandy-Connor. It’s that consumers don’t understand what they are getting when they download their raw data. To confirm the presence of a troubling variant, clinical labs use one method of sequencing, and then if something shows up, they use another to confirm it. Most consumer tests genotype DNA rather than sequence it, and they typically just use one method to do so. That method is cheaper and more efficient than clinical sequencing, but it’s also less robust, looking at far fewer letters in a gene to determine whether a variant is present. For the health risks that 23andMe includes in its reports, the company has proven that the letters it’s looking at are just as good at detecting a variant as the more robust tests. But for other variants, there is no such guarantee.
A 23andMe spokesperson told Gizmodo that this is why customers receive a warning that their raw data is not necessarily accurate nor appropriate for medical use before downloading it.
“This data has undergone a general quality review; however, only a subset of markers have been individually validated for accuracy,” the warning says.
Ambry Genetics sequenced data from 49 patients who had visited their doctor with concerns about troubling variants that showed up in the raw data from different consumer reports. They looked at 26 genetic variants associated with different conditions, many of them cancers. Only 60 per cent of the variants detected in the test were confirmed by Ambry’s test.
“We want this to be educational, for the public and for clinicians that may not be savvy in the genetics space,” Tandy-Connor said. “Patients can ask for raw data. But the raw data is dirty. You can’t bank on it. If something comes up that is worrisome, you need to have it clinically confirmed.”
23andMe, for its part, also encourages consumers to speak to a doctor if a troublesome health condition appears in their results. Tandy-Connor noted that the company plans to do a bigger study to get a better sense of the size of the problem.
Even for health conditions that 23andMe does report on, said Tandy-Connor, consumers need to be aware that the information they’re getting is not necessarily comprehensive.
She pointed to 23andMe’s genetic health risk reports for Parkinson’s disease. The test looks at one variant in each of two genes associated with the disease, LRRK2 and GBA. But there are other pathogenic variants in both of those genes that 23andMe does not look at, as well as additional genes clinically associated with Parkinson’s. Tandy-Connor echoed the concerns others in the medical community have expressed that consumers who take the test are not aware of the limitations of the reports.
“It’s important to be aware of what you’re getting, and what you’re not getting,” she said. “None of these companies have comprehensive testing.”