If you've ever watched a prime-time crime drama like CSI, you know that DNA evidence is often the linchpin that makes a case. Match a suspect's DNA to DNA found at the scene of a crime and it's certain they're the culprit. The thing is, it's not always that simple. Most people think of DNA testing as a monolithic, infallible technique. But there are many different kinds of tests -- and many different ways of interpreting them. Sometimes, somewhere between the process of collecting evidence at the scene and processing it in the lab, something goes awry.
Illustration: Jim Cooke/Gizmodo
For Chen Long-Qi, a bad DNA test derailed his life.
It was early in the morning on May 25th, 2009 and Chen was hanging out in a Taiwan warehouse he rented for work, drinking with friends. At around three in the morning, they were joined by two women. According to Chen and his lawyers, Chen left shortly after to pick up his wife from work, and sometime between 4 and 6 a.m., the two women were raped. While the victims had not accused Chen of rape and no one had placed Chen at the scene when the assaults took place, he was ultimately convicted and sentenced to four years in prison. DNA evidence had linked him to the crime.
Five years later, Chen was exonerated when a second DNA test that found he was not a match after all. In the years he lived as a convicted rapist, he had lost his wife, his business and most of his life. He refused to go to prison, living instead a lonely life as a fugitive, overcome by depression and shame.
Chen was what's known as a coincidental match. Investigators originally tested 17 genetic markers on the Y-chromosome from a mixture of several people's DNA found at the crime scene, and his DNA was a match. But when they tested a larger number of markers, the match didn't hold up. Chen's DNA, it turned out, was not evidence of a crime, but instead evidence of a statistical anomaly that we rarely consider when assessing DNA evidence: that false positives do happen.
"All DNA [evidence] is not the same and that's very hard to explain people," said Greg Hampikian, a Boise State University professor and director of the Idaho Innocence Project who assisted with exonerating Chen. "Even the experts don't understand all the time."
Since DNA profiling led to its first US courtroom conviction in 1987, it has played an increasingly large role in the hunt for justice. The FBI's DNA database alone has grown to include more than 12 million profiles, which contribute to tens of thousands of investigations each year. And there are a variety of different tests performed on DNA used as criminal evidence, even just within the US. The gold-standard of DNA testing, for which the likelihood of an accidental match is one in a billion, is the autosomal STR test. But different tests can be more efficient, appropriate, or cost-effective given different scenarios. Which test investigators chose -- in addition to how it's interpreted -- can greatly affect the outcome of a case.
So while courts are rarely sceptical of DNA evidence, a growing number of academics are calling into question how DNA evidence is handled in criminal investigations. And stories like Chen's have turned Hampikian and others into advocates for greater scrutiny when it comes to DNA evidence.
The one consistently unassailable fact about DNA evidence is our faith in it: One 2005 Gallup poll found that 85 per cent of Americans consider DNA evidence to be very or completely reliable. Another series of studies published in 2008 found that jurors believed DNA evidence to be 95 per cent accurate. Recent research out of Australia found that sexual-assault cases involving DNA evidence were twice as likely to reach trial and 33 times as likely to result in a guilty verdict; homicide cases were 14 times as likely to reach trial and 23 times as likely to end in a guilty verdict. Scientists in the field sometimes refer to a bias towards DNA evidence as the "CSI effect."
DNA can be incontrovertible proof that a crime has been committed -- or in the case of the many exonerated thanks to DNA testing, proof that it wasn't. But in Chen's case, the DNA evidence that mistakenly matched him to the crime was weighted more heavily the testimonies suggesting he was innocent.
To understand how that happened, it's important to understand a bit more about how, exactly, labs match DNA samples. Forensic analysts don't examine a suspect's entire genome, but rather a few key places on it where populations are typically diverse, referred to as "markers." In Chen's case, the lab examined 17 different Y-chromosome markers. However, this particular Y-chromosome test is not as specific as the most common forensic DNA testing, autosomal short tandem repeat testing, or STR. Both tests look at what's known as short tandem repeats, genetic locations on a person's genome that contain a snippet of DNA that is repeated multiple times. The number of those repeats at any location can vary greatly from person-to-person. But whereas the Y-chromosome test looks only at 17 positions on one chromosome, the 13 marker test looks at 13 sites across multiple chromosomes, greatly diminishing the odds of an accidental match. The odds that any two people (except identical twins) will match at all 13 markers in an autosomal STR test is somewhere around 1 in a billion.
According to Hampikian, many crime labs still use the test that mistakenly implicated Chen, though he said some labs are gradually gravitating towards the 23-marker Y-chromosome test that eventually exonerated him.
In another case that illustrates the powerful effect of DNA evidence, Hampikian used DNA evidence to help exonerate Christopher Tapp, a man who until this spring had been in prison since 1998 serving a 25-years-to-life sentence for murder, though his DNA did not match the crime scene sample. When police reopened the decades-old murder in 2015, DNA testing nearly ensnared another innocent man. Police in Idaho Falls combed the records of Ancestry.com for close matches to DNA at the scene, landing on a man named Michael Usry who matched 34 of 35 markers that belonged to the killer. Like in Chen's test, they were looking at only the Y-chromosome, only this time they examined 35 markers instead of the more standard 17 or 23.
When analysts look only at Y-chromosome STRs, they are essentially looking at the whole Y-chromosome. The genetic profile from the Y-chromosome is called a haplotype, and identical haplotypes are passed from father to son, so a population potentially contains many men with the same exact Y-chromosome profile. Had they used a more standard test, Usry would have likely also been a coincidental match and might have been convicted. Instead, police began looking for a complete match among his relatives and wound up getting a sample from Ursy's son. A standard Y-STR test would have nabbed him, but ultimately the 35-marker test cleared his son's name, too. A DNA test gone wrong doesn't just mean that the wrong person is incriminated -- it also robs victims of receiving the justice they deserve.
Chen's case highlights another error that Hampikian and others are increasingly concerned about: DNA mixtures.
In Chen's case, DNA evidence came into play because the two victims in the case had been drinking, and were unable to identify their assailants. Chen's friends both admitted to having sex with the women, but claimed it was consensual. Meanwhile, Chen's wife told police that her husband had picked her up from work around 4 a.m., and a timestamp from her job confirmed it. One of Chen's friends corroborated Chen's assertion that he had left the warehouse around 4 a.m.
When investigators decided that they would test DNA from semen on the underwear of one of the victims, they decided to run it against the DNA of all three men that had been at the warehouse. The DNA on the underwear was a mixture, meaning that the DNA of more than one person was intermixed and it was impossible to tell which snippets of DNA belonged to which person. But because the DNA of more than one person was intermixed in the sample, it meant there was a higher likelihood that a suspect would match the 17 markers tested because there were more than one combination of markers that would fit the bill.
"It's like taking the scrabble tiles for my name and your name and mixing them in a bowl to pull out hundreds of names," said Hampikian. "That makes it especially confounding."
According to calculations later done by Hampikian and his team, there was a 1 in 741 chance that Chen would match the DNA in question -- meaning the DNA on that underwear could hypothetically match thousands of different people in a city of 23 million. The crime lab that ran the test concluded that Chen "or men who share the same paternal line cannot be excluded" as suspects. On this evidence, Chen was convicted of gang rape and sentenced to prison in 2012. The two other men were also a match, and were convicted.
The exonerated Chen Long-Qi (second from left) along with the director of the Taiwan Association for Innocence (far left), Greg Hampikian and Chen Long-Qi's wife. Credit: Greg Hampikian
Chen refused to go to prison, telling his attorneys that he would not "voluntarily walk into jail for something I did not do." He also contacted the Taiwan Association for Innocence, which immediately began working on an appeal, seeing obvious flaws in Chen's conviction. They argued for the DNA to be retested, this time using a test that looks at 23 genetic markers instead of 17. Of those six new genetic markers included in the new test, Chen was only a match for four. The new evidence excluded him as a possible source for DNA found in the mixed sample. What's more, all of the genetic markers in the sample could be explained by the other two men who were convicted. On that evidence, the court granted a new trial and overturned his conviction.
Dan Krane, an expert in DNA evidence at Wright State University, told Gizmodo that in his opinion, the DNA evidence from Chen's case should have never been used in the first place, because it was a mixture of several people's DNA.
"There is no accepted, reliable way of attaching a statistical weight to a mixed YSTR DNA test," he said. "As soon as you see it's mixed, you have to throw up your hands and say, 'Well, that's too bad.' That would have solved the problem right out of the gate here."
Michael Coble, a forensic scientist at the National Institute of Standards and Technology told Gizmodo that closer examination might have allowed the forensic scientists in Taiwan to tell whether the DNA mixture seemed to be largely from one suspect, but otherwise there are limits from the information to be gleaned.
"Often times any DNA evidence gets the seal of approval, but it's really the interpretation that matters," he said.
In a 2013 survey the National Institute of Standards and Technology, which Coble works for, asked 108 labs to interpret a made-up DNA sample with four people in it. They also provided the DNA profile of a fake suspect who wasn't included in the sample. Seventy per cent of the labs found the fake suspect to be a match.
When Gizmodo reached out to the FBI, the agency said that while it does rely on DNA mixtures as evidence, it only does so if a crime lab can pick out one primary person who contributed to a sample. (The FBI also confirmed that its labs still use the 17-marker Y-STR test used in the case against Chen.) In a new case study of Chen's case published in the journal Forensic Science International: Genetics, Hampikian argues that such evidence should only be used to exclude suspects, not to charge or convict them. Krane said it should be thrown out altogether.
"DNA profiling should be binary," Krane said. "There shouldn't be a grey area."
Especially when it's someone's life on the line.
"I think there are maybe a hundred experts who get it and thousands who don't," said Krane. "And there [are] prosecutors who don't want to understand. And I think there's an awful lot of defence attorneys whose eyes glaze over when DNA is mentioned. It's hard enough to teach this in a classroom."
Ballistics, fingerprinting, and arson analysis were all once considered solid scientific evidence designed to extract the potential for human bias in investigation. Today, it is well-established that the results of those methodologies are not always sound. In forensic science, DNA evidence is gradually attracting more scepticism, and scrutiny to ensure that the conditions under which it was collected and processed were sound. In one 2008 study, researchers wrote that there is "a mystical aura of definitiveness often surrounds the value of DNA evidence," but all DNA evidence is not created equal. DNA evidence can be conclusive, but only when good DNA samples are tested correctly using the appropriate test.
DNA's mystical hold over public imagination, Hampikian said, should make all of us very uncomfortable.
Chen, Hampikian said, is one of the few cases that caught the attention of experts like him looking to right the wrongs of DNA testing.
"The way that we rely on tests has to be reevaluated constantly," he said. "We are making mistakes today and we won't know about them for a long time."