What can we do?

Editorial

Does always mean always (ie. justice)? To those of us learning the pledge of allegiance by rote, always means always. The trouble is, in real societies justice does not naturally work that way. A few local societies, like the Amish and Quakers in southeastern Pennsylvania, USA, do. Unfortunately they are the very rare exceptions, not the rule. This page looks into the basis for one of the greatest equalizers ever developed for use in the justice system. When used properly, DNA, is an equalizer, for each of us, except identical twins sports a unique genome. it has freed literally hundreds of individuals wrongly convicted of crimes. That happens when a perpetrator leaves traces of him/herself at the crime scene or on a victim that does not match the person wrongly convicted of the crime on heresay, circumstantial evidence, or out of simple mistake.

Nevertheless, DNA can be misused as evidence in criminal justice systems more interested in closing cases, preserving face, or keeping their jobs simple, than in justice for the accused. The accused may be you or me. We may live in a scientific age, but our justice system has only partly caught up. The reader might benefit here to review "Statistics On The Table" by Stephen Stigler, for a simple overview of where statistics fits in modern society.

The problem is not the DNA. Well check that, the DNA of prosecutors and judges--who actively work to preserve the myth of infallibility in the court room--follow their own genetic dispositions enabled by their DNA. They, with vital help from prior court precedents, outdated FBI policies, state and federal attorneys general, and equally important, the general ignorance of lawyers, judges, jurors and the accused regarding statistical matters, in effect conspire to distort justice, even in capital cases. Misuse of DNA in the court arises simply because it is unfamiliar to all involved in the case. For sure, statistical concepts are difficult for the typical judge, prosecutor, defense attorney, jury, or the accused to understand. (Even for practicing statisticians, it may take years before they are comfortable dealing with probabilities.)

For one example, the most basic concept is the term significance. Most people take it as the strength of an effect. In fact it is the probability that an effect does not even exist! The most common level is set at 0.05, 5%, or one in twenty. Properly used, one sets the level of significance to fit the situation. How sure do we want to be should set the level of probability of error, not convention. This is only the first tier effect. Is 5% adequate in capital cases? Certainly not. But then how do we calculate significance? Do we use the numbers or do we acquiesce to authority--of an agency that has a conflict of interest built in--that refuses to release the logic behind their numbers--the FBI for example.

Misuse happens often because of that very situation. But there is a rub. In all good faith:

• District Attorneys ask for indictments based on DNA samples too fragmentary to be reliable.
• Prosecutors push forward as if a given lab result, however weak, was approved by God.
• Judges routinely exclude pertinent scientific evidence because earlier precedents did so.
• Defense lawyers are unaware of the fragility of DNA evidence in many cases, and seek to plea bargain instead of finding creative ways to challenge weak evidence.
• The accused are typically most disadvantaged of all and are nearly always unaware.
• Jurors are hardly more aware than the accused.

Statistics do not lie, its users do--and they can distort evidence in a variety of ways, in all innocence, as noted above. Being too trusting of a system that is not transparent can work against us. Injustice is an all too common result. The FBI is anything but transparent.

What follows relies heavily on the Dec 18 2009 issue of Science and a follow-up article by Michael Bobelian in the March/April issue of The Washington Monthly. Bobelian studied and reported on this issue under the sponsorship of the Nation Institute. Since the issue is how and why statistics are distorted, we will spend time on the meaning of statistics themselves, skipping the math. Simplification of the usual textbook style to some basic easily-understood principles is of primary importance. We will also illuminate the human side of this problem briefly.

Taking the DNA statistical issue first, there are three uses of the lab-DNA data:

• A perpetrator leaves DNA at a crime scene that is fresh and not mixed with any other DNA, and there is other substantial evidence, both real and circumstantial, that an accused is guilty. The DNA from the crime scene matches that of the accused in all of the standard 13 markers. In this situation, DNA evidence is about as certain as mere mortals can create. The odds of the evidence being wrong is astronomical, about one in several trillion.
• A person, wrongly convicted on circumstantial or improper evidence, can be exonerated on the basis of DNA. Here the exoneration is typically supported by astronomical odds. Because DNA typically degrades over time, not all 13 markers may be available, which reduces the odds. This and the foregoing types may be regarded as sound and scientific. They are hot hunts and provide justice when the DNA evidence is sound and solid enough to prove justice or its miscarriage in a practical sense.
• There is another use--one that usually does not rise to an equivalent level of justice. This happens in the case of cold hunts. For example, a case that remains open for years is often closed as unsolvable. If it happens that DNA evidence becomes or is still available, a closed case may be reopened to the extent that state and government data bases of DNA may be searched for a match. In these cases the DNA is often degraded. The fewer the markers, the less the certainty. California recognizes this and requires a minimum of seven markers of the 13 available before a search of the state's data base can be done. But a California case, resting on just 5.5 markers, brought conviction when a judge did not permit statistical arguments or evidence on actual frequencies in cold cases to be introduced. Is this a miscarriage of justice? We think so. The odds of a miscarriage of justice were not even one in twenty, they were more like one in three! Fortunately, that case is on appeal and may be reversed.

Now, before going on, inferential statistics require certain conditions: Samples must be random and representative of the population they are supposed to represent. All data bases themselves are samples. To be a random sample, every individual of a society must have an equal chance of being sampled. That condition is not met when cold hunts are used where a data base of convenience is used that does not represent real marker frequencies. The data bases available at state and national levels are data bases of convenience, not truly representative of our entire society. Nevertheless, they are large, and that seemingly gives them some validity.

But there is a further and very critical rub. Marker frequencies are hardly random in populations, but are assumed to be. We have not found any studies of the marker frequency patterns in any of the large data bases available to the justice systems in the world. Populations actually consist of an admixture of family trees arising within local societies that in our age are only somewhat blended, and data bases reflect that fact. Family origins matter, and societies are not uniformly blended in regards to family trees. This lack of randomness, and how significant it is, is not yet known. It is certainly not yet common knowledge.

Data from Arizona and Illinois are available to test the FBI procedures that courts are expected to use. In testing for how many matches were found for nine markers, the statistical expectation on the FBI bias is the product of the probability and the data base size. The FBI position is testable and was tested. In Arizona, in a data base of 65,000, where about two matches were expected, 122 matches were found. A Chicago data base of 233,000 profiles, where about seven matches were expected on FBI procedures, a whopping 902 were found. There is something badly wrong in paradise if FBI statistics bear so little relation to reality.

The FBI will not allow their data base to be published for comment or testing. Their reasoning is that it would violate individual confidentiality and would also clog up their computers if they answered every such request. In the first place it is not necessary to release individual names to do effective searches. Secondly, a single search would only take minutes, and would not have to be repeated more than a few times to accurately assess actual marker pairing frequencies. Politics, not justice is the problem.

For the record, combination theory alone, with no other assumptions, provides expectations within less than three multiples for both the Arizona and Chicago results, in sharp contrast with the FBI differences of a hundred multiples or so. This is a huge discrepancy, but it is easily accounted for by the lack of randomness and presence of family trees in the data base. The FBI results would be laughable, if lives and reputations were not at stake.

Why bureaucrats take these positions is stunning on the one hand, but business as usual on the other. History is replete with similar situations. In our time, global warming is a similar form of denial. So also for form coming ahead of substance, myth presiding over science, the existence of corporations too large to fail, that "fact" that women cannot be trusted with their own morality and ethics, that racial differences are significant and grounds for maltreatment, that Homo sapiens is unique in all ways that count and no other species matters, that "America First" is God's command. Is any of that justice? Hardly.

The Science article is a letter edited by Jennifer Sills. The letter has 39 signers with Dan Krane of Wayne State University the lead author. The other 38 signers include individuals, universities, institutes, government officials, and public defenders.

Obama promised an open government, but so far that promise has been far too selective. The FBI has done and still does great work. But justice for all is not their byword. Denial at worst, defensiveness at best, seems to be endemic. And so we will believe--until proven otherwise.