The Washington Post has a report today:
The Justice Department and FBI have formally acknowledged that nearly every examiner in an elite FBI forensic unit gave flawed testimony in almost all trials in which they offered evidence against criminal defendants over more than a two-decade period before 2000.
Of 28 examiners with the FBI Laboratory’s microscopic hair comparison unit, 26 overstated forensic matches in ways that favored prosecutors in more than 95 percent of the 268 trials reviewed so far, according to the National Association of Criminal Defense Lawyers (NACDL) and the Innocence Project, which are assisting the government with the country’s largest post-conviction review of questioned forensic evidence.
If the vaunted FBI examiners can make errors, then surely the examiners at USACIL, DCFL, and similar “labs” can have problems.
One of the underlying issues may be that examinations for law enforcement purposes are not “blind.” What do I mean by that? Go back and look at a few examinations you’ve seen for your case recently. When the samples are submitted for testing the field agent normally submits a statement or outline of the case. Essentially, they are telling examiners the result they want–the evidence matches the suspect. See comments here. I know I talk about this frequently–the confirmation bias phenomenum–but here is my evidence Prof’s thoughts on how the manner in which samples are submitted, and tested, can lead to the potential for a flawed result.
Commentators have identified bias as a serious problem in the forensic setting. As one commentator noted: “To the extent that we are aware of our vulnerability to bias, we may be able to control it. In fact, a feature of good scientific practice is the institution of processes—such as blind testing, the use of precise measurements, standardized procedures, statistical analysis—that control for bias.” A 1996 National Academy of Sciences report on DNA testing recommended that laboratory procedures “be designed with safeguards to detect bias and to identify cases of true ambiguity. Potential ambiguities should be documented.
Keep in mind that internal quality assurance reviews do not solve this potential problem. The incentives for the QA examiner are no different than the initial examiner. Some of you may remember from quite some years ago the Brooks AF drug lab scandal. This involved a blind negative sample submitted by AFIP, which was reported by the laboratory as positive for a controlled substance. I remember a case where the female client’s sample was reported positive for a controlled substance. It was not until the case reached an adsep board and we got “discovery” that we noticed that the testing data showed the presence of male DNA in the sample. The government’s expert testified that the result was correct, but that it must have been contaminated somewhere in the collection and testing process. Despite this the command persisted in going forward to a finding of no misconduct.
Here is a piece from Marc Green, about how these human factors mentioned above can affect the reliability of a forensic test result. Whereas the FBI describes the handwriting methodology. They have this interesting statement.
The FBI Laboratory has not established a specific number of “points” or characteristics needed to identify a questioned writing as having been prepared by a particular individual. In order for a forensic document examiner to identify an individual as having prepared a questioned writing, agreement must exist between significant characteristics in the questioned and known writing with no significant differences. The examiner must explain any exceptions.
Yes, there is a wide degree of subjectivity. See here for a discussion of the points issue. Prof. Jennifer Mnoonkin writes here on fingerprints.
The point here is that you yourself should not engage in your own blind confirmation bias–which is to accept without examination the findings of a forensic examination. As several writers point out, there may be quite a bit of other evidence in your case which goes to prove the accused’s guilt, and that the forensic examination is merely icing on the cake. But in close cases it’s worth a read to peel the onion on forensic tests and reports. Certainly the research and questions about the reliability of forensic testing may well help lay a foundation for expert assistance. There may be support for what it is the expert is going to do for the defense.