At a press briefing in February 2002 about the lack of evidence of weapons of mass destruction in Iraq linked to terrorist groups, then-Secretary of Defense Donald Rumsfeld offered insightful perspective:
“There are known knowns; there are things we know that we know,” Rumsfeld said. “There are known unknowns; that is to say, there are things that we now know we don’t know. But there are also unknown unknowns – there are things we do not know we don’t know.”
Forensic facial reconstruction is a powerful investigational tool to identify crime victim remains, particularly when DNA is not available. It is also used in archeology; for example, a reconstructed face of King Tut was displayed on the cover of National Geographic in 2005. Psychological profiling is useful for investigating serial murders, bombings and other egregious criminal events. However, because of the degree of subjectivity underlying forensic facial reconstruction and psychological profiling, both forensic techniques are typically inadmissible in criminal trials.
Another effective investigational forensic technique is polygraphy. In 2003, a National Research Council (NRC) report stated that, although well below perfection, results from use of the polygraph exceed chance guess rates for detection of untruthful statements. It has been a useful forensic tool for inducing confessions from crime suspects. Although there are numerous studies concluding that false confessions are a major contributing factor in wrongful convictions that have lead to exonerations, the polygraph remains an effective investigational tool, though like facial reconstruction and psychological profiling, it is not suitable as evidence of guilt in judicial proceedings.
For decades, firearms examiners have testified in court that firearm/toolmark identification, the practice that links scratches (called “striations”) and impressions on bullets and cartridge cases to one, specific weapon, is virtually infallible. Yet there is much anecdotal evidence that suggests the possibility that the forensic practice of firearms identification, known as specific source attribution, may have a significantly higher rate of error than claimed by expert witnesses. Notwithstanding, under increasing pressure from the mainstream scientific community, examiners have recently revised their wording in court testimonies. Instead of “absolute certainty” and “bullets concluded to have been fired from this specific firearm,” they now claim conclusions to a “practical certainty.”
We find this revised position incredulous. It is an example of an unknown unknown: The examiners do not know that they do not know fundamental statistical principles and other essential sciences required as foundational support for the pattern-matching opinions they render, that specific source attribution is inherently probabilistic, and that the revised wording is nothing more than semantic manipulation.
A recent audit of the Detroit crime labs Firearms/Toolmarks Unit found serious errors in 10% of cases reviewed. Had the audit been conducted by the mainstream scientific community rather than by the firearm/toolmark examiner auditors, additional errors may well have been uncovered. The unit was subsequently closed. If the evidence in those cases were the only evidence presented at the trials of 100 defendants, it could translate into 10 wrongful convictions.
In a recent trial in the state of New York, a firearm/toolmark examiner concluded that the defendant’s gun positively fired the fatal bullets in an April 1988 homicide. Yet records showed the firearm had not even been shipped from the original U.S. distributor (the firearm was of foreign manufacture) in Greenville, S.C. until four months after the murder.
In yet another serious misattribution, a police officer in Missouri was killed with what investigators originally believed was his own weapon, since it was missing from the scene. A suspect was apprehended and a firearm in his possession was sent to a crime lab for comparison with bullet evidence from the shooting scene. The firearms identification expert declared a “match” and the suspect was convicted. Later, the officer’s weapon was recovered. The same expert who examined the original suspect’s firearm decided he liked the second submission (the officer’s gun) as a better “match.”
If we assume a practice accuracy of 90 percent, firearm/toolmark identification may be adequate, or even good, as an investigational technique. But it is objectionable when represented or perceived as incontrovertible evidence of guilt or as forensic “science” rather than the art or trade that it is. Juries should be made expressly aware that opinions of provenance are just that: opinions, and that such opinions are subject to currently unquantified error.
Firearms/toolmarks examiners are sometimes also asked to testify about bullet damage to structures, including whether a specific caliber of weapon was the cause. Most often, it’s inappropriate for a firearm examiner to give such testimony. The science involved in terminal ballistics (impact mechanics) is immensely complex and highly mathematical, and requires appropriate advanced credentials in fields such as physics, materials science, metallurgy, mechanical engineering and statistics for uncertainty assessment. Asking firearms/toolmarks examiners to understand and explain the material deformation mechanisms of threat/target interactions, and to give an opinion on what caused damage because they’re familiar with firearms and have also seen many bullet impacts, is akin to designating a skilled driver as an “expert” witness on whether vehicle damage was caused by a collision with another vehicle, a pedestrian, a deer, a guard rail, a stop sign, a utility pole, a fire hydrant, a telephone booth, or some other object, and on the scientific bases for that conclusion.
Except in the area of DNA evidence, too many members of the forensic science community are, as the British might say, too clever by half when it comes to providing expert testimony. What is good investigational forensics often results in scientifically unsupportable testimony to jurors and jurists, who are asked to recognize true science from educated conjecture. Thus, for both forensic experts and judicial proceeding participants (judges, prosecutors, defense attorneys and jurors, alike) there are too many unknown unknowns as the system currently functions (again, aside from the domain of DNA science). The system is badly in need of reform and external oversight by the mainstream scientific community.
William A. Tobin is a scientist who spent 27 years as an FBI agent, the last 24 in the FBI Laboratory, and since then has worked on legal cases for both defense attorneys and prosecutors as a forensic metallurgist/materials scientist. He was the principal critic of the forensic practice of comparative bullet lead analysis (CBLA), which has since been discontinued by the forensic community, and has published numerous forensic papers.
Distinguished Professor Cliff Spiegelman, a member of the Texas A&M University faculty since 1987, was a panelist for the National Research Council of the National Academy of Sciences’ committee convened to investigate the challenge to CBLA. He is a leader in the field of statistical forensics and best-known for his joint forensic work related to the President John F. Kennedy assassination bullets, which earned an award from the American Statistical Association (ASA). Spiegelman is an elected fellow of the ASA and the Institute of Mathematical Statistics.
To find a previous article by these authors on the science of forensic evidence, go to this story at statesman.com/insight.