In 2010, an influential national organization of scientists devoted to ensuring that forensic labs employ only the latest and best methods of analyzing DNA evidence published a new set of guidelines. In essence, the Scientific Working Group on DNA Analysis Methods concluded that analysts should use the quality of genetic samples, rather than the quantity of evidence gathered at a crime scene, to decide if they could produce a confident genetic profile.
Despite their increasing importance in criminal investigations, forensic labs in the U.S. aren’t regulated, but rather voluntarily adhere to a set of evolving standards. So the working group, along with the federal government’s National Institute of Standards and Technology, set about spreading the new best practices.
Over the next five years, the organizations held numerous training sessions, conducted surveys and visited forensic labs. They stressed that using the older methods could lead a lab to overstate the confidence of its findings — which could result in improper criminal prosecutions. By 2015, every lab across the country that John Butler, a forensic science expert for the national institute, was aware of had adopted the improved method, he said.
All, that is, except one: the DNA lab on Springdale Road operated by the Austin Police Department.
“They were doing calculus problems still using two plus two equals four and algebra math,” Butler said. “Their statistical tools were not sufficient to address the task at hand.”
To outsiders, the Austin lab’s implosion was speedy. In June, then-Police Chief Art Acevedo said the DNA division would close temporarily to allow analysts time to correct problems found in a recent review. In December, officials announced the lab was being shuttered indefinitely.
Yet interviews with those who worked there, documents detailing the facility’s past performance and recent court filings suggest that many of the lab’s problems had been simmering for years.
From the start, “they never supported the things I needed to set up the DNA lab,” said Donna Stanley, the former DPS analyst originally hired to get the Austin lab off the ground more than a decade ago. Front-line analysts said they raised the issue of the lab’s inadequate methods to their supervisors, only to be ignored. Unwilling to trust its work, prosecutors and defense attorneys had requested reviews of the lab’s DNA analyses.
At the same time, the watchdogs supposedly guaranteeing lab quality standards failed to notice basic shortcomings. The American Society of Crime Laboratory Directors Lab Accreditation Board, the industry’s nonprofit accrediting body, bestowed passing audits on the Austin facility without noting its analysts were clinging to a DNA statistical method considered scientifically unacceptable across the country.
“One would think it would have been picked up,” said Lynn Garcia, general counsel for the Texas Forensic Science Commission, formed a decade ago to monitor the science increasingly being used to convict criminals. It has requested an explanation of the lapse from the accreditation agency. “It raises questions about what the auditors were doing.”
The accreditation board did not respond to phone or email messages seeking comment. But Garcia said that in response to the forensic science commission’s questions, an officer explained that under older standards the board did not test if a lab’s scientific processes were appropriate for analyses. Newer standards are supposed to ensure a lab’s methods are sound, Garcia said.
Yet the accreditation board’s latest audit of the Austin lab, in 2015 — apparently conducted under the new standards — still concluded its “methods were fit for their intended use.”
Over the past decade, a surprising number of forensic facilities in Texas and across the country have endured high-profile failures, ranging from stolen or missing evidence, to shoddy science and even faked results. Taken together, the scandals have raised doubts about the scientific conclusions used to secure tens of thousands of prosecutions. Of the 302 Texans who have been exonerated of their crimes, 142 — nearly half — were convicted at least in part based on false or misleading forensic evidence, according the National Registry of Exonerations, a database maintained by the University of Michigan’s law school. (Most were drug cases.)
Experts say the repeated failures point to the urgent need for more comprehensive oversight to replace a system unable or unwilling to ensure that sound science is being used to fairly prosecute the correct people. One survey found half of the failed labs in recent years were officially accredited.
Time after time, the lab scandals have been dismissed as the shoddy work of a lone misfit without bothering to identify the systemic failures that caused them, said Erin Murphy, a New York University law professor and author of “Inside the Cell: The Dark Side of Forensic DNA.”
“When other disciplines engage in high-stakes activities” — airlines and hospitals, for example — “they understand it’s essential to understand the root causes that result in mistakes,” she said. “In crime labs there isn’t that much of a culture of root cause analysis — a sense that we need to re-examine our labs from top to bottom.”
In recent years, forensic science has struggled to redefine itself as evidence emerged that lab work confidently used to solve crimes in the past was anything but scientific. Once considered rigorous, verifiable science, bite marks and traditional arson investigations have been debunked as little more than tea-leaf reading.
Even ballistics and fingerprint analysis, while still valuable tools, are “more like art appraisal” than hard science, said University of Houston Law Center professor Sandra Guerra Thompson.
Yet even with DNA analysis — a true science — Thompson, whose research resulted in her 2015 book, “Cops in Lab Coats: Curbing Wrongful Convictions through Independent Forensic Laboratories,” said she found fundamental problems with the way forensic labs traditionally operated. For the most part, “these are police organizations that have a goal; to produce evidence to assist police officers in investigations, and prosecutors in securing convictions,” she said.
Historically, she said, that had resulted in a range of flaws, including a law enforcement tilt by supposedly unbiased lab analysts. A 2009 U.S. Supreme Court opinion noted that the “neutral scientific testing” police labs claimed as their standard was neither predictably objective nor reliable: “Forensic evidence is not uniquely immune from the risk of manipulation.” That year the National Academy of Sciences published an influential paper challenging much of the science, interpretation and oversight of forensic work.
Thompson also discovered many labs struggled with inadequate funding from police chiefs more interested in buying new equipment and beefing up patrols. That, in turn, often led to the hiring of undertrained analysts who didn’t understand the science behind their work. Unqualified supervisors either let mistakes pass or failed to recognize them.
“There has been a fair amount of fraud,” said Thompson. “But mostly it’s incompetence.”
The failings have become more pronounced as DNA analysis has become more ambitious. Today, crime scene investigators collect genetic evidence that might include only a few hundred human cells and represent the biological codes of three or more people. Many lab technicians simply are unprepared for the advanced work necessary to analyze such evidence, Murphy said.
At the same time, regulation of the profession — its facilities and workers — remains mostly voluntary. The result has been a steady string of scandals.
In recent years, New York City has been forced to revisit hundreds of rape cases because of poorly interpreted DNA evidence; a Boston lab’s credibility was rocked when an analyst admitted to falsifying hundreds of drug tests; and a Florida facility was humiliated after a supervisor pleaded guilty to stealing drug evidence.
Texas hasn’t been immune. Forensic labs in Houston, San Antonio, El Paso and in the Department of Public Safety have all been forced to re-examine thousands of cases due to problems ranging from clerical errors and theft to “dry-labbing” — the reporting of results without performing any actual testing.
‘Dug their heels in’
The lack of meaningful regulation meant the Austin lab’s fundamental deficiencies were discovered only by happenstance.
In the spring of 2015, the FBI alerted labs that its database used to determine if DNA found at a crime scene matched a particular person contained errors. The forensic commission decided to survey public forensic labs in Texas to see how they were responding, Garcia said.
The answers from Austin were alarming. When the commission’s DNA experts, including University of North Texas Health Science Center professor Bruce Budowle, reviewed the lab’s work, “He saw something that was different than all the other labs,” she said.
His review of a random selection of some of the facility’s cases showed a surprising lack of understanding of foundational DNA analysis among examiners. Worse, even after the commission’s experts — including Butler, who’d written a book about the subject — pointed out that the lab’s methods were incorrect, supervisors there continued to use them.
“They still kind of dug their heels in,” Garcia said.
Further investigation revealed that it wasn’t the first time lab supervisors had been alerted to the problem, either. “Multiple analysts had raised concerns” with DNA unit managers over the years, Garcia said. “But they never did anything with it. They just kept doing the same thing.”
It took commission investigators only two days on-site to uncover the serious failings other auditors had missed or ignored. In one case the commission’s experts reviewed, the Police Department’s DNA analysis “contradicted both the victim and the suspect’s accounts” of what had happened.
Other failures ranged from mundane — the lab was using a chemical reagent against manufacturer and industry recommendations, which could compromise analysis — to examiners’ fundamental understanding of their work.
“Refresher training in basic molecular biology, forensic genetics and statistics will be a critical component of moving the laboratory forward,” the report concluded. It noted that Austin DNA lab technical leaders — supervisors — were underqualified for the oversight and expertise they were expected to provide.
Austin police have pledged a review of the lab’s work, including selected retesting and, if necessary, reopening cases to ensure that any errors didn’t produce mistaken convictions. Early estimates of the cost to taxpayers have been placed at up to $14 million.
Issues noted in 2008
City officials also have vowed a thorough examination of the lab’s history to explain how the DNA unit veered off its rails. Interviews and a review of past reports show that some of the same problems that led to collapses at other labs also appear to have plagued Austin: uneven funding, poorly trained and overworked technicians, and supervisors ill prepared to run a high-tech science facility.
When Mark Gillespie arrived to assume leadership of the Austin police Forensic Science Division in 1996, he found the facilities scattered throughout the Seventh Street building. “I didn’t realize how bad it was,” he said.
After DPS started telling the Police Department it could no longer handle all of its DNA work, Austin began making plans for its own unit. A bond measure to build a new forensic facility passed in 1998. In the meantime, Gillespie said, the unit was located in a former basement storage room.
Gillespie, now the owner of an investigations business in Cedar Park, said he mainly recalls the fights to convince police brass to fund the new unit’s work. “They were always taking money away for patrol and ops,” he recalled. “We had positions removed from our DNA unit several times.”
The person he hired to get the DNA lab off the ground, Donna Stanley, remembered similar battles for money and space. “I would sit and comb the internet for grants,” she said. When she found money to buy DNA analysis equipment, recalled Stanley, now retired and living in Colorado, “It sat in the hallway boxed for two years because we had no place to put it.”
It wasn’t until September 2004 that the lab, located in a new forensic facility on Springdale Road in East Austin, finally began producing analyses. It was accredited a year later. Maurice Padilla, who worked in the DNA unit from 2004 to 2006, said the scientific work he observed then was sound.
Yet a 2008 outside evaluation of the Police Department noted the forensic division’s work was “hampered by a shortage of resources.” It also found more than half of the DNA analysts Austin had hired were still underqualified: “In 2006, only two of the unit’s five analysts were qualified to process DNA samples because the three analysts did not have the required experience and certifications.”
By 2013, the Police Department hadn’t hired any new scientists in nearly a decade while the number of sworn officers had grown more than 20 percent and demand for forensic testing soared. By mid-2016, 1,300 DNA samples were awaiting testing.
There were other hitches. Several audits noted Austin forensic employees lacked the recommended education. In 2010, the DNA lab was rocked by a complaint filed by one of its own workers. In a lengthy memo, DNA analyst Cecily Hamilton described how she had observed poor work and management dysfunction in the lab.
Hamilton singled out analyst Diana Morales as incompetent. In one instance, she wrote, she saw a supervisor help Morales cheat through a proficiency exam. Another DNA examiner reviewing Morales’ work “was shocked by some of the errors” as well as her inability to understand them, Hamilton reported. (A police spokeswoman said Morales was unavailable for an interview.)
The allegations were investigated by the Texas Rangers, who didn’t find the problems described by Hamilton. “The DNA analyst in question was found to be well-trained and competent,” the Rangers concluded.
This May, however, Travis County prosecutors in the middle of a sexual assault case raised familiar-sounding complaints about Morales. According to a memo, they found her unprepared to testify about her DNA analysis, giving contradictory answers to their questions and unable to explain her basic analysis methods and conclusions.
In December, the DPS, which had agreed to retrain the Austin lab’s staff, concluded only two of the city’s DNA analysts were worth keeping. Morales was one of those identified as untrainable.
Given the depth of problems in its DNA unit, Thompson, the Houston law professor, said the Austin Police Department’s top-to-bottom review of its forensic lab is crucial and overdue. She added that she wouldn’t be surprised if it turned up other areas in need of reform, as well.
“If your DNA lab is that bad,” she said, “the rest of the lab has problems, too.”
A Bumpy History: Austin police DNA lab timeline
October 1996: Texas Department of Public Safety tells Austin police forensic officials it will begin limiting DNA testing it can do for the department; the city begins planning for Austin’s own DNA analysis facility.
September 2004: After years of delays and false starts, the DNA lab starts performing analyses.
August 2005: The forensic lab is accredited by the American Society of Crime Laboratory Directors.
January 2010: Scientific Working Group on DNA Analysis Methods posts new standards about testing genetic material and starts promoting them. Austin doesn’t comply.
February 2010: DNA analyst Cecily Hamilton complains that an examiner is incompetent and that a manager helped her cheat on a skills test, among other concerns.
March 2010: An internal investigation of Hamilton’s complaints notes the lab had passed numerous audits and says, “There have been no significant issues identified within the DNA section.”
September 2010: The lab passes another accreditation review.
September 2010: Investigation by the Texas Rangers dismisses Hamilton’s complaints, expressing “complete confidence” in the DNA unit.
May 2015: The FBI notifies labs of errors in population data used to determine whether DNA found at a crime scene matches a particular person.
September 2015: The lab passes another accreditation review. It noted the lab’s “methods were fit for their intended use” and its analysts passed a proficiency test that “meets or exceeds” standards.
December 2015: Texas Forensic Science Commission surveys public forensic labs to see how they’re handling the new FBI recommendations. Austin’s responses are alarming.
February 2016: The Major Cities Chiefs Association visits the lab and concludes, “It was readily apparent that the laboratory and its staff upheld the high standards of accreditation.”
May 2016: Texas Forensic Science Commission visits the DNA lab and uncovers serious deficiencies.
June 2016: Police announce they’re closing the DNA lab temporarily while the problems are corrected.
July 2016: Final forensic commission report details deep, fundamental problems in the DNA unit.
December 2016: DPS says it can’t retrain most of Austin’s DNA analysts. Police say the unit could be closed more than two years and cost up to $14 million to remediate.