The problem with DNA

Since the advent of DNA testing, it's solved cold cases, connected crimes committed in different jurisdictions and even freed innocent men from death row. Genetic fingerprinting has in many ways revolutionized forensic science and is often the best weapon in an investigator's arsenal.

However, popular crime shows such as "CSI," "Forensic Files," and "Law and Order" make solving crimes and catching criminals look easy-particularly with DNA evidence (see "Judicial Notebook"). In reality, juries sometimes gloss over the possibility of lab mistakes and mixed evidence-even the similar DNA of siblings-which can all lead to false positives and land innocent people in prison.

"It's not so much a question of whether they understand the technology," notes psychologist Jonathan J. Koehler, PhD, a business professor at the University of Texas at Austin and an expert in behavioral decision-making. "But do they know how to interpret reported matches?"

Math class for juries

Research indicates that people generally aren't very good at interpreting probabilities, and they are easily swayed by the way statistics are presented, Koehler explains. For instance, a 2004 study published in Psychological Science (Vol. 15, No. 8, pages 540-546) found that mock jurors were more impressed by a match with the probability 0.1 in 100 than with one in 1,000, even though they are mathematically identical. Koehler argues that the fractional component in the first statistic (0.1 in 100) discouraged jurors from thinking about others who might match by coincidence. This made the DNA evidence seem relatively strong. In contrast, jurors who received the DNA match statistic as one in 1,000 were more likely to think about others in a large population who might match by coincidence, and this made the evidence seem weaker.

"Jurors have trouble appreciating the power of a DNA match when they can imagine another person matching," Koehler explains. "When you say two in 2,000 or two in 2,000,000, people think about the other guy-what about him?"

Since everyone's DNA-except that of identical twins-is unique, the chance of a coincidental match is often around one in 10 billion, which sounds really impressive, given that the population of the earth is about 6.5 billion. But one in 10 billion is a statistical probability of coincidence-not proof that only one person on the earth could have this profile. What's more, that number does not factor in other potential problems, such as lab errors, says psychologist William Thompson, PhD, JD, professor and chair of the School of Social Ecology at the University of California, Irvine.

In the courtroom, jurors tend to think that match statistics such as one in a billion cover all possibilities for error, but they only refer to one area, such as the chance of a coincidental match, and there are other mitigating factors such as the chances of a false match, Thompson explains.

"The problem with DNA is that it is perceived as providing a unique and infallible ID," he warns. "But very often there are disagreements among experts about whether results are a match."

Reasonable doubt?

In the glitzy labs of crime dramas, no one argues about whether the DNA is a match. The test results are final. However, in the real world, it's not so cut and dried.

"Ultimately it comes down to the judgment of the examiner, which is influenced by many factors," says Koehler.

Lab technicians run a sample through DNA processing software, which produces an image similar to an EKG readout, with a series of lines with peaks and valleys, which represent the number of repeating DNA sequences at certain points on the genome. If the profile is taken from a single source ofDNA, there's not usually much need for interpretation, explains Thompson, who is also an attorney. Errors are more common when the DNA is "mixed," meaning from more than one person or the evidence is degraded by time or improper storage, he continues. Then experts must try to separate out the different sources.

The people who are sorting all of this out often know who is supposed to "match," which can influence the process.

"Analysts think they should know all about the cases and evidence instead of just evaluating in a blind way," says Thompson, who is also an attorney. "Almost no crime labs do blind scoring and they can often become emotionally involved, and this leads to bias."

Many public and private labs periodically test technicians' DNA-matching proficiency, and publish the results. From looking at such tests in the 1990s, Koehler has estimated that labs produce false positives in one or two instances out of every 100 tested samples.

Judges rarely arm jurors with this information, and sometimes ask them to decide which expert to believe based on highly technical testimony, says Thompson, who often serves as a trial consultant. He recently consulted on a case where the dispute over a match was due in part to confusion over whether there were two or three contributors to the DNA sample.

"What does a jury do with that?" he asks.

Jurors often get too much information, and not enough instruction on how to analyze it, says David H. Kaye, JD, a law professor at Arizona State University.

For instance, in cases with good samples, probability estimates should be thrown out the window, Kaye says.

"What do we care if it's one in one million or one in one sextillion?" he asks. "Why not have an expert say, 'It's my opinion that this came from the same individual?'"

Koehler would like to see attorneys present a match without probability statistics but with lab error rates.

And while most of the time a match is a match, says Thompson, sometimes an overeager lab, a messy crime scene and a jury that's watched too much crime television can lead to a mistaken conviction.

"They think they understand it better than they do," he says.