4/23/2009 6:28 PM
DAUBERT V. MERRELL DOW
PHARMACEUTICALS AND THE LOCAL
CONSTRUCTION OF RELIABILITY
Robert Robinson *
ABSTRACT
Scholars considering how expert testimony will fare under
Daubert often apply the four dicta referenced by Justice
Blackmun (testing, peer-review, error rate, and general
acceptance) to determine whether such testimony will be
admissible. In this article I critique this approach, contending
that admissibility decisions cannot be adequately predicted by
Daubert itself. Daubert has no clear legal rule for judges to
apply, has no cognizable position on the degree of scrutiny expert
testimony should face, and has no clear stance—even given the
dicta—on what constitutes “good science.” When combined with
the relative autonomy trial judges possess in making
admissibility decisions, Daubert’s essential ambiguity leads to
what I call “local constructions of reliability,” disparate and often
competing conceptions of what constitutes reliable expert
evidence. What is considered reliable in one area of expert
testimony, such as medical causation, will be quite different from
what is required for another, such as handwriting expertise.
If Daubert leads to such variation among different spheres of
expert testimony, how can we generalize or predict judicial
decision-making? I argue that admissibility decisions can and
should be modeled empirically. Viewing judges as goal-oriented
actors, I transform likely goals for judges making Daubert
decisions into relevant variables for empirical modeling. These
goals include attention to the quality of expert testimony (if not
*
Assistant Professor of Political Science, University of Alabama at Birmingham; B.A.
Rhodes College 1997; M.A. University of Wisconsin, 1998; Ph.D. University of Wisconsin,
2004.
39
4/23/2009 6:28 PM
40
ALB. L.J. SCI. & TECH.
[Vol. 19.1
always under the Daubert dicta), maintaining institutional
stability and judicial autonomy when faced with controversial
scientific claims, and advancing judicial policy preferences. I
intend this discussion to serve as a template for further empirical
work on Daubert.
TABLE OF CONTENTS
I. INTRODUCTION ........................................................................... 41
II. ADMISSIBILITY “REGIMES”: A THEORETICAL FRAMEWORK ....... 45
A. Epistemology and Judicial Capacity in Assessing
Expert Testimony.......................................................... 45
B. The Federal Standard: Daubert v. Merrell Dow
Pharmaceuticals ............................................................ 49
C. Daubert and the Theoretical Framework: Realist,
Constructivist, or Neither? ........................................... 57
III. THE EMPTY VESSEL: DAUBERT AND THE LOCAL
CONSTRUCTION OF RELIABILITY........................................... 59
A. Law, Legal Realism, and Judicial Decision-Making ...... 59
B. The Institutional Context of Daubert Decisions ............ 63
C. Daubert’s Fractured Stance on Admissibility ................ 64
1. Admissibility in the Daubert Opinion ...................... 64
2. Contrasting Admissibility Standards in United
States v. Crisp .......................................................... 69
D. Daubert’s Fractured Stance on Epistemology ................ 73
IV. EMPIRICALLY MODELING DAUBERT DECISIONS ....................... 76
A. Judges as Goal-Oriented Actors ..................................... 76
B. Legal Goals: Assessing Methodological Quality ............. 77
C. Institutional Goals: Weighing the Effects of
Admissibility Decisions on Courts ................................ 80
D. Attitudinal Goals: Advancing Judicial Policy
Preferences .................................................................... 84
E. External Variables .......................................................... 86
V. CONCLUSION: NOTES ON GETTING DAUBERT DATA ................... 88
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
41
I. INTRODUCTION
Over the next few decades, it is conceivable that scientists will
be able to “see” the genesis of a thought. The development of
functional magnetic resonance imaging, or fMRI for short,
permits a visual approximation of the brain at work, enabling
scientists to determine more exactly the physiological starting
point of rational thought, emotions, and deception. While much
of the attention given to the “new neuroscience” will focus on ageold problems of free will and determinism, 1 fMRI’s potential to
detect lies may be of more immediate and practical importance to
the justice system. Given the centrality of credibility and truthtelling to court proceedings, it is unsurprising that neurologists
are already examining whether fMRI scans can provide valid and
reliable evidence that a particular individual is lying. 2
Expert testimony linking brain scans to deception will only
reach federal juries if it meets the evidentiary standard of
reliability set forth in Daubert v. Merrell Dow Pharmaceuticals.3
Compare Joshua Greene & Jonathan Cohen, For the Law, Neuroscience
Changes Nothing and Everything, 359 PHIL. TRANSACTIONS ROYAL SOC'Y
LONDON, SER. B, BIOLOGICAL SCI. 1775, 1775–76 (2004), with Stephen J. Morse,
Brain Overclaim Syndrome and Criminal Responsibility: A Diagnostic Note, 3
OHIO ST. J. CRIM. L. 397, 398, 402 (2006).
2 Donald H. Marks et al., Determination of Truth from Deception Using
Functional MRI and Cognitive Engrams, 5 INTERNET J. RADIOLOGY (2006),
http://www.ispub.com/ostia/index.php?xmlFilePath=journals/ijra/vol5n1/engram
.xml; Daniel D. Langleben et al., Brain Activity during Simulated Deception: An
Event-Related Functional Magnetic Resonance Study, 15 NEUROIMAGE 727, 727
(2002); Daniel D. Langleben et al., Telling Truth from Lie in Individual
Subjects with Fast Event-Related fMRI, 26 HUM. BRAIN MAPPING 262, 263
(2005), available at http://repository.upenn.edu/neuroethics_pubs/6; F. Andrew
Kozel et al., A Pilot Study of Functional Magnetic Resonance Imaging Brain
Correlates of Deception in Healthy Young Men, 16 J. NEUROPSYCHIATRY &
CLINICAL NEUROSCIENCES 295, 296–97 (2004), available at http://neuro.
psychiatryonline.org/cgi/content/abstract/16/3/295; Tatia M.C. Lee et al., Lie
Detection by Functional Magnetic Resonance Imaging, 15 HUM. BRAIN MAPPING
157, 157–58 (2002); Jennifer Marie Nunez et al., Intentional False Responding
Shares Neural Substrates with Response Conflict and Cognitive Control, 25
NEUROIMAGE 267 (2005); G. Ganis et al., Neural Correlates of Different Types of
Deception: An fMRI Investigation, 13 CEREBRAL CORTEX 830, 830 (2003),
available at http://cercor.oxfordjournals.org/cgi/content/full/13/8/830.
3 See Daubert v. Merrell Dow Pharm., Inc., 509 U.S. 579, 591–95 (1993). The
reaction of courts to new technologies has been decidedly mixed. In the forensic
context, the criminal justice system has eagerly and perhaps uncritically
accepted technological advances that individuate criminal suspects, such as
fingerprinting or DNA typing. Jennifer L. Mnookin, Fingerprint Evidence in an
Age of DNA Profiling, 67 BROOK. L. REV. 13, 16–17, 20 (2001). Advances that
attempt to assess the credibility of individual statements, however, have often
been received with suspicion and even disdain. Ric Simmons, Conquering the
1
4/23/2009 6:28 PM
42
ALB. L.J. SCI. & TECH.
[Vol. 19.1
A practitioner or legal scholar interested in the use of fMRI scans
in federal courts (and state courts that apply a Daubert-like test)
must therefore consider how current research meets this
standard. At present, at least three such analyses have been
After providing a layman’s
published by legal scholars. 4
summary of the fMRI technique and the research regarding its
potential as a lie detector, the articles examine whether fMRI
research meets the testing, peer-review, error rate, and general
acceptance dicta that often comprise a de facto test for applying
Daubert. The authors generally agree that while the detection of
deception using fMRI may someday be admissible, it currently
fails to pass muster under these criteria.
Given that federal judges sometimes structure their
admissibility opinions along similar lines, 5 it seems reasonable
for the authors of these articles to use the Daubert dicta to
predict judicial behavior. That said, I argue in this article that
such an approach is largely mistaken, and that better descriptive
understandings of how Daubert is applied, as well as how better
predictions of how Daubert will be applied in the future, require
empirical modeling. The problem lies not in the authors’ lack of
knowledge regarding the underlying science (which they
understand well), or their particular applications of the Daubert
dicta (which are reasonable). The problem lies instead in the
assumption that the Daubert precedent provides a global
definition of reliability that can be extrapolated to specific areas
of expert testimony.
I state simply that it does not. Law is most effective in guiding
judicial behavior when the law has a relatively clear rule, a
relatively clear substantive meaning, or where judges face
meaningful appellate oversight. Daubert decisions fit none of
these criteria. As a decision rule, Daubert does not dictate how
an admissibility determination should be carried out. The
Province of the Jury: Expert Testimony and the Professionalization of FactFinding, 74 U. CIN. L. REV. 1013, 1022, 1027–28 (2006).
4 Archie Alexander, Functional Magnetic Resonance Imaging Lie Detection: Is
a "Brainstorm" Heading Toward the "Gatekeeper"?, 7 HOUS. J. HEALTH L. &
POL'Y 1, 28–29, 56 (2006); Charles N. W. Keckler, Cross-Examining the Brain: A
Legal Analysis of Neural Imaging for Credibility Impeachment, 57 HASTINGS
L.J. 509, 511, 554–56 (2006); Michael S. Pardo, Neuroscience Evidence, Legal
Culture, and Criminal Procedure, 33 AM. J. CRIM. L. 301, 319 (2006).
5 See, e.g., United States v. Llera Plaza, 188 F. Supp. 2d 549, 551, 563 (E.D.
Pa. 2002); United States v. Lowe, 954 F. Supp. 401, 410–11 (D. Mass. 1996);
United States v. Starzecpyzel, 880 F. Supp. 1027, 1040–41 (S.D.N.Y. 1995).
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
43
opinion easily supports contrasting views on the appropriate
stringency of the courts towards admitting borderline expert
testimony; it also supports contrasting views on the epistemology
of science that judges should adopt. Even the Daubert dicta,
which suggest that expert testimony should be tested, peerreviewed, generally accepted, and present error rates, lacks a
consensus understanding of these terms. As precedent, Daubert
is much like the inscription on the Mirror of Erised from Harry
Potter and the Sorcerer’s Stone: “I show not your face but your
heart’s desire.” 6
A fractured precedent with no central meaning, Daubert leads
to what I call “local constructions of reliability;” what constitutes
reliable expert testimony in one sphere of science may be quite
different from what passes Daubert in another. For example,
Daubert’s application during a summary judgment motion in a
“toxic tort” may be quite different from a motion where criminal
defendants seek to exclude the state’s forensic testimony. The
growing Daubert literature finds this split to be real, finding
some evidence that admissibility standards have tightened in
civil cases 7 while remaining unchanged in criminal ones. 8 My
6 J.K. ROWLING, HARRY POTTER AND THE SORCERER'S STONE 207 (Scholastic
Press 1998). The quote in the original text must be read backwards to
understand its meaning. Bathrobe’s Harry Potter in Chinese, Japanese &
Vietnamese Translation, Treatment of Puns and Word Play in Translating
Harry Potter (Chinese, Japanese, and Vietnamese): The Mirror of Erised, http://
www.cjvlang.com/Hpotter/wordplay/erised.html (last visited Mar. 11, 2009).
7 See Margaret A. Berger, What Has a Decade of Daubert Wrought?, 95 AM. J.
PUB. HEALTH S59, S64 (2005), available at http://www.ajph.org/cgi/content/full/
95/S1/S59; Christina L. Studebaker et al., Changes in the Standards for
Admitting Expert Evidence in Federal Civil Cases Since the Daubert Decision, 8
PSYCHOL. PUB. POL'Y. & L. 251, 251, 301 (2002); Ronald L. Melnick, A Daubert
Motion: A Legal Strategy to Exclude Essential Scientific Evidence in Toxic Tort
Litigation, 95 AM. J. PUB. HEALTH S30, S30, S31, S34 (2005), available at
http://www.ajph.org/cgi/content/full/95/S1/S30. I leave for others arguments
over whether the impact in civil cases is the result of legal constraints imposed
by Daubert, the increased attention to expert testimony that Daubert may have
provided, or external changes, such as an increasingly conservative federal
judiciary, that took place around the same time-frame. See Jeremy Buchman,
The Effects of Ideology on Federal Trial Judges' Decisions to Admit Scientific
Expert Testimony, 35 AM. POL. RES. 671, 671, 673–74 (2007); Edward K. Cheng
& Albert H. Yoon, Does Frye or Daubert Matter? A Study of Scientific
Admissibility Standards, 91 VA. L. REV. 471, 503–05 (2005).
8 Simon A. Cole, Grandfathering Evidence: Fingerprint Admissibility Rulings
From Jennings to Llera Plaza and Back Again, 41 AM. CRIM. L. REV. 1189,
1274–75 (2004); Robert Epstein, Fingerprints Meet Daubert: The Myth of
Fingerprint "Science" is Revealed, 75 S. CAL. L. REV. 605, 647–52 (2002); Paul C.
Giannelli, The Supreme Court's "Criminal" Daubert Cases, 33 SETON HALL L.
4/23/2009 6:28 PM
44
ALB. L.J. SCI. & TECH.
[Vol. 19.1
point is a broader one: since Daubert neither constrains nor
guides judicial behavior, we must look beyond doctrinal analysis
to understand what takes place when expert testimony is on
trial.
If I am correct, Daubert scholarship would greatly benefit
empirical models of judicial behavior. While Daubert does not
lack for scholarly attention, 9 empirical research on Daubert,
whether descriptive or predictive, has been quite limited in
relation to the whole. In a recent survey of work on Daubert,
Saks and Faigman 10 found only seven empirical studies of
admissibility patterns and the factors that might influence
judicial decision-making. 11 Much of the legal literature regarding
admissibility is geared towards normative discussions about the
philosophy of science, institutional arguments about the courts’
ability to assess expert testimony, and the abstract merits and
deficiencies of particular admissibility regimes.
Though
undoubtedly important, this array of arguments would be well
complemented by widespread empirical analysis of Daubert
decisions at the trial court level, as well as by studies which
attempt to assess the impact and significance of particular
variables—beyond the dicta—that might influence the outcomes
of admissibility decisions.
The outline of this argument is as follows. In Part II, I briefly
examine the general problems of epistemology and judicial
capacity any admissibility regime must face, and trace how
Daubert skirts taking a clear stance on either issue. In Part III, I
develop my argument that Daubert leads to local constructions of
reliability rather than establishing a global one.
Lacking
effective institutional oversight, a formal decision rule, or a
REV. 1071, 1111 (2003); Peter J. Neufeld, The (Near) Irrelevance of Daubert to
Criminal Justice and Some Suggestions for Reform, 95 AM. J. PUB. HEALTH
S107, S109 (2005), available at http://www.ajph.org/cgi/content/full/95/S1/S107;
D. Michael Risinger, Navigating Expert Reliability: Are Criminal Standards of
Certainty Being Left on the Dock?, 64 ALB. L. REV. 99, 143, 149 (2000).
9 According to the author’s research, a simple keyword search of “Daubert”
among the law review articles in LEXIS NEXIS returns over three thousand
hits.
10 David L. Faigman et al., Check Your Crystal Ball at the Courthouse Door,
Please: Exploring the Past, Understanding the Present, and Worrying About the
Future of Scientific Evidence, 15 CARDOZO L. REV. 1799 (1994) (Faigman is a
professor of law at the University of California, Hastings College of Law, and
Saks is a professor of law at the University of Iowa College of Law).
11 Michael J. Saks & David L. Faigman, Expert Evidence After Daubert, 1
ANN. REV. L. & SOC. SCI. 105, 120–25 (2005).
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
45
shared understanding of either the stringency of the
admissibility standard or the proper epistemology behind it,
Daubert decisions take place in a position of considerable judicial
autonomy.
Accepting that judges may be guided or even
constrained by local, on-point precedent for specific areas of
expert testimony, we still need explanations for how local
constructions of reliability develop in the first place. When
assessing the reliability of new types of expertise such as using
fMRI to detect lies, these explanations become vital.
In Part IV, I survey the political science, empirical legal
studies, and sociology of science literatures to glean potential
variables for a decision model of admissibility decisions. Judges
undoubtedly care about the reliability of expert testimony, but
their modes of analyses are as likely to be intuitive as
deliberate, 12 assessing reliability through heuristics such as the
identity of the litigant or the reaction of professional and
academic interest groups. I also consider how judges might fear
that particular Daubert outcomes would negatively affect court
institutions, such as increasing caseload, or how some forms of
expert testimony may face special burdens to admission because
they appear to challenge the empirical foundations of the courts
themselves. For example, in appellate-level Daubert decisions
regarding the defendants’ attempts to introduce eyewitness
expert testimony, some judges worried that such testimony might
threaten the right of the jury to assess witness credibility. 13
Finally, Part V concludes the article by briefly noting an
important roadblock for future empirical research on Daubert,
namely the collection of representative data.
II. ADMISSIBILITY “REGIMES”: A THEORETICAL FRAMEWORK
A. Epistemology and Judicial Capacity in Assessing Expert
Testimony
American legal jurisdictions are paternalistic when it comes to
the admission of evidence, expert or otherwise. 14 Neither truth
12 Chris Guthrie et al., Blinking on the Bench: How Judges Decide Cases, 93
CORNELL L. REV. 1, 43 (2007).
13 United States v. Hall, 165 F.3d 1095, 1107 (7th Cir. 1999); United States v.
Amador-Galvan, No. 96-10137, 1997 U.S. App. LEXIS 5320, at *4–5 (9th Cir.
Mar. 14, 1997); United States v. Brien, 59 F.3d 274, 277 (1st Cir. 1995).
14 Joseph Sanders, Kumho and How We Know, 64 LAW & CONTEMP. PROBS.
373, 406–07 (2001).
4/23/2009 6:28 PM
46
ALB. L.J. SCI. & TECH.
[Vol. 19.1
nor fairness are best served, our legal system holds, by a laissezfaire presentation of any evidence various litigants might wish to
submit. Judges instead employ a ponderous set of filters and
rules in hopes of shielding decision-makers from their own
cognitive biases. 15
This gate-keeping process becomes dramatically more difficult
when expert testimony is assessed. The justification for judicial
gate-keeping of expert testimony is similar to the general one:
juries lack the training and awareness of their own perceptual
and cognitive errors to separate bad science from good. Expert
testimony, however, disrupts the standard assumption of special
judicial competence, as judges may be no better equipped than
juries in determining methodological quality. This well-known
dictum, often attributed to Learned Hand, appears insoluble: if
the legal system could adequately discern the validity of expert
opinions, it should not need that testimony in the first place. 16
Given this paradox, any legal regime designed to assess expert
testimony will be imperfect. Absent the wholesale admission of
all proffered expertise, however, some test must be adopted.
The choice of potential admissibility regimes from alternatives
such as the “market test,” 17 the general acceptance test from Frye
v. United States, 18 or Daubert depend in large part on two
criteria: the nature of “science” the legal system envisions and
the capability of judges to determine whether the evidence meets
the chosen criteria. 19 Selecting an epistemology to ground an
15 This central assumption regarding the degree of juror biases—at least
relative to the biases of judges—has come under question. See, e.g., Luke M.
Froeb & Bruce H. Kobayashi, Naive, Biased, Yet Bayesian: Can Juries Interpret
Selectively Produced Evidence?, 12 J.L. ECON. & ORG. 257, 257 (1996).
16 See,
e.g., Learned Hand, Historical and Practical Considerations
Regarding Expert Testimony, 15 HARV. L. REV. 40, 54 (1902). Hand’s concern
dealt primarily with juries, but the problem would apply to any trier of fact not
versed in the substantive field of expertise at hand.
17 This regime permitted expert opinions to be admitted in court if the
opinion had been validated by the market in some way, whether because others
had paid that expert in private endeavors, or because that expert claim
engendered a product or service consumers had purchased. Faigman et al.,
supra note 10, at 1803–04.
18 Frye v. United States, 293 F. 1013, 1014 (D.C. Cir. 1923).
19 See, e.g., David S. Caudill & Richard E. Redding, Junk Philosophy of
Science?: The Paradox of Expertise and Interdisciplinarity in Federal Courts, 57
WASH. & LEE L. REV. 685, 689, 703, 764–65 (2000); CARL F. CRANOR, REGULATING
TOXIC SUBSTANCES: A PHILOSOPHY OF SCIENCE AND THE LAW 55–56 (Oxford
University Press 1993); Susan Haack, Trial and Error: The Supreme Court's
Philosophy of Science, 95 AM. J. PUB. HEALTH S66, S70–S71 (2005); IAN
HACKING, THE SOCIAL CONSTRUCTION OF WHAT? 32–33, 61–62 (Harvard
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
47
admissibility regime is a difficult task, exacerbated by the sheer
range of argument regarding whether scientific findings are
“real.” At one end of this spectrum, philosophers such as Comte
have proffered an almost naïve faith in scientists to deduce the
nature of the universe from methods that transcend all problems
At the other end,
of culture, ideology, and practicality. 20
epistemological anarchists offer the proposition that science
bears little if any connection to reality, its findings instead
wholly constructed by society. 21 Between these poles of naïve
positivism and anarchism lie two more feasible epistemologies for
courts to adopt: realism and constructivism. 22 Realists believe
that scientists do reveal an underlying reality through the
scientific method, at least in probabilistic terms. Constructivists,
by contrast, contend that scientific findings are substantially
dependent on the culture and institutions that produce them. 23
For constructivists, idealized notions of scientific production
ignore the obstacles posed by cultural assumptions, religious
teachings, interest group politics, disciplinary disputes,
ideological beliefs, personal interactions, conflicts of self-interest,
the desire for fame, and the hanging sword of tenure. 24 Realizing
the impact of these factors, constructivists say, should lead one to
displace the reification of scientific methodology with the
realization that some, even most research findings result from
social factors. 25 To paraphrase Shelia Jasanoff, legal actors
should abandon the notion that there are clear definitions of
“good science” by which judges may separate inadequately
“scientific” opinions from reliable expert testimony. 26
University Press 1999); PETER W. HUBER, GALILEO'S REVENGE: JUNK SCIENCE IN
THE COURTROOM 14–17 (BasicBooks 1991).
20 See AUGUSTE COMTE, THE POSITIVE PHILOSOPHY OF AUGUSTE COMTE 25–26
(Harriet Martineau trans., Calvin Blanchard 1855).
21 See PAUL FEYERABEND, AGAINST METHOD: OUTLINE OF AN ANARCHISTIC
THEORY OF KNOWLEDGE 17, 180–81, 295 (Humanities Press 1975).
22 See Caudill & Redding, supra note 19, at 688, 752, 756; Margaret G.
Farrell, Daubert v. Merrell Dow Pharmaceuticals, Inc.: Epistemology and Legal
Process, 15 CARDOZO L. REV. 2183, 2185, 2193, 2198, 2206–07 (1994).
23 SHEILA JASANOFF, SCIENCE AT THE BAR: LAW, SCIENCE, AND TECHNOLOGY IN
AMERICA 22, 224–26 (Harvard University Press 1995).
24 Id. at 206–09; Sheila Jasanoff, What Judges Should Know About the
Sociology of Science, 32 JURIMETRICS J. 345, 346–47 (1992).
25 JASANOFF, supra note 23, at 52.
26 Id. at XIII-XIV. Though this remains sound advice, it is unclear exactly how
naïve judges are in practice about such claims. Though a simplistic reliance on
tests such as those in the Daubert dicta might raise this critique, historical
studies of judges and expert testimony suggest judges might not be so
4/23/2009 6:28 PM
48
ALB. L.J. SCI. & TECH.
[Vol. 19.1
There is little evidence to suggest that either federal judges or
members of Congress explicitly considered epistemology during
the choice or creation of our admissibility regimes. The Daubert
court addresses the issue only in a brief and haphazard manner,
as I will argue below. 27 Regardless, evidentiary regimes may still
be properly analyzed from an epistemological standpoint, since
their tests and guidelines often adopt a particular vision of
science whether or not that was their authors’ intent.
Apart from epistemology, support for a particular admissibility
regime also depends on whether courts have the institutional
capacity to implement a particular epistemological vision. A
realist might believe that the scientific method haltingly
uncovers fundamental reality and yet argue that judges have
little or no ability to assess this process. If judges—or the court
system in general—lack the expertise or will to incorporate
mainstream scientific analysis into admissibility decisions, a
deferential evidentiary standard might be the only alternative. 28
simplistic. See TAL GOLAN, LAWS OF MEN AND LAWS OF NATURE: THE HISTORY OF
SCIENTIFIC EXPERT TESTIMONY IN ENGLAND AND AMERICA 259–60, 263–64
(Harvard University Press 2004) (discussing the complexities involved in a
judge's gate-keeper role).
27 Daubert makes Popperian falsification the central marker of “good
science.” See discussion infra Part II. Though important, and particularly
useful, for rejecting tautologies, modern philosophers of science no longer see
falsification as indispensible to the scientific method. See, e.g., Caudill &
Redding, supra note 19, at 738–39; Haack, supra note 19, at S67–S69; Gary
Edmond & David Mercer, Keeping Junk History, Philosophy and Sociology of
Science Out of the Courtroom: Problems with the Reception of Daubert v. Merrell
Dow Pharmaceuticals Inc, 20 U. NEW S. WALES L.J. 48, 51, 54–55 (1997).
28 Katherine M. Atikian, Nasty Medicine: Daubert v. Merrell Dow
Pharmaceuticals, Inc. Applied to a Hypothetical Medical Malpractice Case, 27
LOY. L.A. L. REV. 1513, 1556 (1994); Paul S. Milich, Controversial Science in the
Courtroom: Daubert and the Law's Hubris, 43 EMORY L.J. 913, 917–19, 926
(1994); Craig Lee Montz, Trial Judges As Scientific Gatekeepers After Daubert,
Joiner, Kumho Tire, and Amended Rule 702: Is Anyone Still Seriously Buying
This?, 33 UWLA L. REV. 87, 104–07, 114–15 (2001). On a similar note, if the
jury does not suffer from cognitive errors, or at least suffers no worse than the
judge who would be gate-keeper, perhaps the evidentiary test should be relaxed.
See, e.g., Sanja Kutnjak Ivkovi & Valerie P. Hans, Jurors' Evaluations of Expert
Testimony: Judging the Messenger and the Message, 28 LAW & SOC. INQUIRY
441, 441–42, 445–46 (2003); NEIL VIDMAR, MEDICAL MALPRACTICE AND THE
AMERICAN JURY: CONFRONTING THE MYTHS ABOUT JURY INCOMPETENCE, DEEP
POCKETS, AND OUTRAGEOUS DAMAGE AWARDS 178, 181–82 (The University of
Michigan Press 1995). Though there are fine studies of how jurors process
expert testimony, such as for eyewitness expert testimony, see, e.g., BRIAN L.
CUTLER & STEVEN D. PENROD, MISTAKEN IDENTIFICATION: THE EYEWITNESS,
PSYCHOLOGY, AND THE LAW 216–218, 224 (Cambridge University Press 1995);
ELIZABETH F. LOFTUS, EYEWITNESS TESTIMONY 191, 200–01 (Harvard University
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
49
One of the best known statements of concern over active
gatekeeping roles comes from Justice Rehnquist’s opinion
(concurring in part and dissenting in part) in Daubert:
I defer to no one in my confidence in federal judges; but I am at a
loss to know what is meant when it is said that the scientific status
of a theory depends on its “falsifiability,” and I suspect some of
them will be, too. I do not doubt that Rule 702 confides to the
judge some gatekeeping responsibility in deciding questions of the
admissibility of proffered expert testimony. But I do not think it
imposes on them either the obligation or the authority to become
amateur scientists in order to perform that role. 29
Rehnquist may sell judges short, particularly if the choice is
framed not as judges or nothing, but instead as judges or juries.
Judges are “repeat players” 30 who will encounter similar claims
of expertise over multiple occasions, and will almost certainly
have more experience than a jury in evaluating the claims of
expert witnesses. Moreover, as repeat players, judges have
educational and developmental resources available to them that
juries do not, as a judge who wishes to understand the PCR
method for DNA typing may turn to the relevant article from the
excellent Reference Manual on Scientific Evidence, 31 or attend a
workshop on the same.
Daubert, I believe, neither provides a clear picture of
epistemology nor realistically weighs the burdens it imposes on
judicial institutions.
B. The Federal Standard: Daubert v. Merrell Dow
Pharmaceuticals
At present, most American legal jurisdictions have adopted one
of two admissibility regimes: either some version of the general
acceptance test (the “Frye” regime), or some version of Daubert. 32
This leads to the tendency to label jurisdictions as having “Frye”
Press 1979), and for DNA evidence, see, e.g., Jason Schklar & Shari Seidman
Diamond, Juror Reactions to DNA Evidence: Errors and Expectancies, 23 LAW &
HUM. BEHAV. 159, 159 (1999), far more research is needed.
29 Daubert, 509 U.S. at 600–01.
30 See generally, Marc Galanter, Why the "Haves" Come Out Ahead:
Speculations on the Limits of Legal Change, 9 LAW & SOC'Y REV. 95, 97 (1974).
31 DAVID H. KAYE & GEORGE F. SENSABAUGH, JR., Reference Guide on DNA
Evidence, in REFERENCE MANUAL ON SCIENTIFIC EVIDENCE 485, 497 (Federal
Judicial Center ed., 2d ed. 2000).
32 See David E. Bernstein & Jeffrey D. Jackson, The Daubert Trilogy in the
States, 44 JURIMETRICS 351, 351 (2004) (noting the diversity existing in
jurisdictions adopting either the Frye or Daubert test).
4/23/2009 6:28 PM
50
ALB. L.J. SCI. & TECH.
[Vol. 19.1
or “Daubert” regimes, though this binary categorization
admittedly conflates important differences. 33 The Frye regime,
taking its name from a 1923 federal criminal case in which the
United States challenged the defendant’s proffer of a polygraph
test as evidence of his innocence, simply asks whether the expert
testimony at hand is “generally accepted” by the scientific
community at hand. 34 At that time, courts often judged the
admissibility of expert testimony according to its utility in the
marketplace; a technique, idea, or concept that was reliable
enough to be consumed, used, or purchased was reliable enough
for courts. 35 Aside from the rather heroic assumptions such a
test makes about consumers, 36 its criteria had little relevance for
forensic techniques, whose only “consumers” were law
enforcement and the court system. Faced with a defendant who
claimed a systolic blood pressure test could demonstrate his
innocence, the court ruled that this forerunner of the polygraph
had not been generally accepted in the scientific community, and
thus was not fit for admission. 37 Though the Frye case was
Id. at 365. As he finds, a state might accept a Daubert-like admissibility
test but refuse to accept Daubert’s progeny, Joiner and Kumho. A state might
also exempt certain kinds of evidence from an admissibility regime, or adopt
different standards of admissibility for criminal and civil courts. Finally, a
state might remain a “Frye” state but find its actual use of the general
acceptance test altered by Daubert’s influence. Bernstein gives a thorough
breakdown of these various categories. Id.
34 Frye, 293 F. at 1013–14.
35 Faigman et al., supra note 10, at 1804; DAVID L. FAIGMAN ET AL., MODERN
SCIENTIFIC EVIDENCE: THE LAW AND SCIENCE OF EXPERT TESTIMONY § 1:3, at 7
(2005-06 ed., Thomson West 2006); Michael J. Saks, Merlin and Solomon:
Lessons from the Law's Formative Encounters with Forensic Identification
Science, 49 HASTINGS L.J. 1069, 1073–74 (1998). A Rhode Island chief justice
from this time period made the case for this regime when he remarked,
“[i]ndeed, knowledge of any kind, gained for and in the course of one's business
as pertaining thereto, is precisely that which entitles one to be considered an
expert . . . .” Buffum v. Harris, 5 R.I. 243, 251 (R.I. 1858).
36 As Faigman, Porter, and Saks note:
But though it might be practical and easily administered, the test of
commercial value is a poor one. Its major weaknesses are perhaps more
obvious today than they were a century or two ago. The market not only
selects for validity, it selects also for entertainment, desire, wishful
thinking, hope, sometimes even desperation. These are not without their
value, but they are not good proxies for what courts are looking for in
expert testimony. If the marketplace approves, as it does, of astrologers,
sellers of phony cancer cures, and guides to new age vortexes, are those
therefore good enough to provide guidance in a courtroom?
The
marketplace test is incapable of distinguishing astrophysics from astrology.
Faigman et al., supra note 10, at 1805.
37 Frye, 293 F. at 1014.
33
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
51
largely ignored for decades, it became a leading candidate for the
national admissibility regime in the 1970s, perhaps in reaction to
the increasing debate over the use of scientific evidence in the
courtroom. 38
The Frye regime transferred judicial deference from the
marketplace to the scientific community. On the question of
epistemology, Frye takes a simplistic realist position, putting
faith in scientific communities to separate good science from
bad. 39 There is no recognition of the social and cultural factors
that may affect scientific claims, perhaps in line with the greater
influence positivism held in the academy at the time Frye was
written. As for the question of institutional capacity, the regime
largely absolves judges from gate-keeping; their only task is to
ascertain the extent of the relevant scientific community and the
degree of consensus within that community about the reliability
of the expert testimony before the court. Frye is properly viewed
as a “conservative” admissibility regime, both because it will take
time for new expert testimony to be generally accepted by at least
a plurality of the relevant expert community, 40 and because once
expert testimony has gathered a significant degree of consensus
it will become hard to exclude. 41
Frye’s virtue is its simplicity and the modest demands it makes
on judicial capacity. Its great vice, unsurprisingly, is that same
simplicity. A general acceptance test tells us very little about the
proper scope of the scientific community that constitutes that
group whose acceptance is needed. All things being equal, a
smaller group of scientists who are heavily invested in a
technique or opinion will be quicker to support more
controversial expert claims than a broader community. One can
thus manipulate Frye by broadening the scope of the relevant
community to the point where general acceptance is unlikely, or
narrowing it to the point where the chances of agreement
Sanders, supra note 14, at 376; Saks, supra note 35, at 1076.
See Faigman et al., supra note 10, at 1805 (noting that the marketplace
test was incapable of distinguishing the expert from the knowledge).
40 E. Donald Elliott, Toward Incentive Based Procedure: Three Approaches for
Regulating Scientific Evidence, 69 B.U. L. REV. 487, 495 (1989). At least one
scholar has argued that Frye is so restrictive that its use might more quickly
lead to evidentiary reform in the statutory sphere. In other words, she would
gamble that Frye would keep out so much useful and reliable expert testimony
that Congress or state legislatures would be moved to institute a more liberal
admissibility regime.
41 Saks & Faigman, supra note 11, at 119.
38
39
4/23/2009 6:28 PM
52
ALB. L.J. SCI. & TECH.
[Vol. 19.1
increase.
The problem of scope worsens when addressing
forensic testimony, where the expert community is likely limited
to the practitioners themselves. For example, fingerprint experts
will be the most capable group of individuals for ascertaining
what fingerprint techniques are generally accepted. However,
those same experts are unlikely to entertain claims that their
enterprise lacks scientific rigor or validity. 42
The stronger critique of Frye is that deference to expert
agreement is no substitute for an actual analysis of evidentiary
worth. The history of science is replete with expert communities
who favored theories that scientists now agree lack validity. 43 If
one accepts that scientific results are at least in part socially
constructed, that whether for cultural, religious, financial, or
institutional reasons expert communities do not always simply
work towards truth, then an admissibility regime based on
deference will inevitably admit unworthy expert testimony.
Indeed, without some degree of independent gate-keeping, the
Frye test can become an absurdity. If a litigant tendered
astrological evidence in support of a factual claim, would the
community of astrologers be consulted for general acceptance?
Or would the judge simply shift his analysis to astronomers or
scientists in general to ensure the expertise is excluded? Either
way, the example demonstrates the problem with equating
expert agreement with validity, particularly for small or
specialized expert communities. The problem exists even in more
legitimate debates—if doctors and epidemiologists disagree about
the proper methods of inferring causation for medical problems,
whose opinions should constitute the Frye analysis?
Apart from its theoretical problems, Frye faced a more concrete
difficulty following the 1975 promulgation of the Federal Rules of
Evidence.
Frye, viewed as a narrow and conservative
admissibility regime, seemingly conflicted with the Rules’ liberal
42 See Cole, supra note 8, at 1202 (showing that the difference between
eyewitness identification and fingerprint identification is that fingerprint
identification is completed by experts); Epstein, supra note 8, at 651; D. H.
Kaye, The Nonscience of Fingerprinting: United States v. Llera-Plaza, 21
QUINNIPIAC L. REV. 1073, 1087 (2003).
43 For example, one might cite phrenology (the study of how head shape
indicates character and intelligence), vile theories of racial difference, and
arguments that one’s body measurements serve as indicators of criminal
tendencies. That said, a defender of Frye might reasonably ask whether an
active judicial gate-keeper, even within the crucible of an idealized adversarial
process, would have been any more successful in exposing problems with these
theories.
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
53
attitude towards the admission of evidence. 44 Despite this
apparent conflict, the precise requirements of the new rules of
evidence were in doubt. 45 The initial version of Federal Rule of
Evidence 702, which dealt most directly with expert testimony,
stated that, “[i]f scientific, technical, or other specialized
knowledge will assist the trier of fact to understand the evidence
or to determine a fact in issue, a witness qualified as an expert
by knowledge, skill, experience, training, or education, may
testify thereto in the form of an opinion or otherwise.” 46 702
clearly required that expert testimony be relevant and helpful,
and that the expert who tendered it be qualified. What was not
clear, however, was whether the rule’s phrasing inferred a
“reliability” requirement. On the one hand, its plain meaning did
not present a reliability requirement. 47 On the other hand, the
rule’s use of the term “scientific knowledge” may reasonably be
read to require some baseline of “scientific” quality in order to
earn the name. 48 The federal circuits reflected this split of
opinion, as some rejected Frye as too restrictive in light of the
Federal Rules, 49 some read Frye into Rule 702, aiming for a more
Sanders, supra note 14, at 376–77.
See Daubert, 509 U.S. at 579 (illustrating the clash between the Federal
Rules of Evidence and the Frye test).
46 FED. R. EVID. 702 (1975) (amended 2000). In 2000, Federal Rule 702 was
updated, probably to bring it into accordance with the Daubert decision. The
new text reads:
If scientific, technical, or other specialized knowledge will assist the trier of
fact to understand the evidence or to determine a fact in issue, a witness
qualified as an expert by knowledge, skill, experience, training, or
education, may testify thereto in the form of an opinion or otherwise, if (1)
the testimony is based upon sufficient facts or data, (2) the testimony is the
product of reliable principles and methods, and (3) the witness has applied
the principles and methods reliably to the facts of the case.
FED. R. EVID. 702. Some scholars argue that the amended 702 creates a more
stringent standard for expert testimony than Daubert itself, though there is
little evidence that federal judges agree on this proposition. See, e.g., David
Bernstein, Courts Refusing to Apply Federal Rule of Evidence 702, THE
VOLOKH CONSPIRACY, May 6, 2006, http://volokh.com/archives/archive_
2006_04_30-2006_05_06.shtml (discussing how federal district courts are
ignoring Rule 702 and following previous court precedent decided before 2000).
47 Brief for Association of Trial Lawyers of America as Amicus Curiae
Supporting Petitioner at 8, Daubert v. Merrell Dow Pharm., Inc., 509 U.S. 579
(1992) (No. 92-102).
48 Brief for American Insurance Association as Amicus Curiae Supporting
Respondent at 2, Daubert v. Merrell Dow Pharm., Inc., 509 U.S. 579 (1993) (No.
92-102).
49 See, e.g., United States v. Baller, 519 F.2d 463, 466 (4th Cir. 1975)
(discussing how there must be an objective procedure for determining
44
45
4/23/2009 6:28 PM
54
ALB. L.J. SCI. & TECH.
[Vol. 19.1
restrictive standard, 50 and at least one circuit, foreshadowing
Daubert, developed its own standards for reliability. 51
In order to resolve this circuit conflict, the Supreme Court
chose to hear Daubert v. Merrill Dow Pharmaceuticals, a Ninth
Circuit appeal in which the Daubert family had sued Merrill
(later purchased by Dow), alleging its anti-nausea drug
Bendectin had caused serious birth defects in their unborn
child. 52 The Daubert case came at the tail end of a long series of
Bendectin torts in both state and federal court, whose progress
came alongside considerable legal, scientific, and media
attention. 53 While Bendectin plaintiffs had several early jury
successes (Merrell made for a particularly unfavorable
defendant, given its reticence to divulge its own research and its
prior involvement with other unsafe drugs such as thalidomide
and MER/29), ensuing epidemiological studies slowly but surely
built a body of scholarship that found no demonstrable
correlation between the use of Bendectin and birth defects.54
Both the trial court and the Ninth Circuit relied on the general
admissibility of scientific opinion); United States v. Williams, 583 F.2d 1194,
1197–98 (2d Cir. 1978) (discussing why Frye is better than Daubert to
determine the admissibility of evidence). Notably, these early cases rejected a
more stringent admissibility test in the context of criminal cases, where
admissibility rules for the prosecution remain relatively lax.
50 See, e.g., Dang Vang v. Vang Xiong X. Toyed, 944 F.2d 476, 481–82 (9th
Cir. 1991) (illustrating how the general acceptance requirement of Frye is read
into Rule 702); United States v. Kozminski, 821 F.2d 1186, 1210 (6th Cir. 1987)
(illustrating how courts read the Frye test into Rule 702).
51 United States v. Downing, 753 F.2d 1224, 1226 (3d Cir. 1985).
52 Daubert v. Merrell Dow Pharm., Inc., 951 F.2d 1128, 1129 (9th Cir. 1991),
rev'd 509 U.S. 579 (1993).
53 See JOSEPH SANDERS, BENDECTIN ON TRIAL: A STUDY OF MASS TORT
LITIGATION ix, 23 (The University of Michigan Press 1998) (discussing how
Bendectin litigation produced a long series of trials and court opinions).
54 Id. at 40. Forty such epidemiological studies were completed between 1963
and 1994. None of these studies found a significant correlation between the
drug and the defects, though six found that Bendectin might be a teratogen but
lacked sufficient evidence to draw a conclusion. While some of the early studies
lacked adequate statistical power to be truly persuasive, the body of work
became more sophisticated and convincing over time. Bendectin plaintiffs were
increasingly forced to rely on animal studies and structural analyses to make
their case for causation. Animal studies are dubious evidence of causation
because of the difficulties of extrapolating animal results to humans, as well as
because the animals in question often received atypical dosage of the substance
in question (reactions to toxic chemicals are not always linear). Structural
analyses compare the structure of known tetraogens to those of the drug in
question, but as minute differences in structure can lead to very different
physiological reactions, their use in establishing causation is also limited. Id. at
60–62, 64.
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
55
acceptance test from Frye in their analysis; the Ninth Circuit also
read a peer review requirement into Rule 702. 55 Finding the
Dauberts’ expert testimony inadequate, the trial court ruled to
exclude it, leaving the plaintiffs subject to summary judgment.56
The Ninth Circuit affirmed. 57 The Dauberts appealed to the
Supreme Court, arguing that the Ninth Circuit’s interpretation
of Frye contradicted the liberal emphasis of the Federal Rules of
Evidence, and that as statutory law the Rules superseded the
common-law Frye. 58
The Supreme Court unanimously agreed with the Dauberts
that Federal Rule 702 supplanted Frye as the standard for expert
admissibility, arguing that the more rigid general acceptance test
did not square with the “liberal thrust” of the Federal Rules. 59
However, the Court also agreed with the amicae who supported
Merrell Dow that Rule 702 imposed a reliability requirement. 60
While the Ninth Circuit’s construal of reliability was too narrow,
Blackmun agreed that FRE 702 bestowed a “gatekeeping”
responsibility on the trial judge to ensure that all expert
testimony was reliable, relevant, helpful, and “fit” the issue at
hand. 61
Daubert, 509 U.S. at 583–84.
Daubert, 951 F. 2d at 112–31. The Dauberts’ expert witnesses had either
reanalyzed previously unsupportive epidemiological data or relied upon animal
studies and case reports as the basis for their testimony. The trial judge ruled
that their methods were not generally accepted by epidemiologists for assessing
causation (though they were acceptable for risk assessment by a body such as
the EPA), and that their opinions had not been peer-reviewed.
57 Id. at 1131.
58 Daubert, 509 U.S. at 587.
59 Id. at 588 (quoting Beech Aircraft Corp. v. Rainey, 488 U.S. 153, 169
(1988)).
60 Brief of the American Insurance Association as Amici Curiae in Support of
Respondent, at *17–18, *22, Daubert v. Merrell Dow Pharm., Inc., 509 U.S. 579
(1993) (No. 92-102); Brief of the Pharmaceutical Manufacturers Association as
Amicus Curiae in Support of Respondent, at *30–31, Daubert v. Merrell Dow
Pharm., Inc., 509 U.S. 579 (1993) (No. 92-102); Brief of the New England
Journal of Medicine, Journal of the American Medical Association, and Annals
of Internal Medicine as Amici Curiae in Support of Respondent, at *20–21,
Daubert v. Merrell Dow Pharm., Inc., 509 U.S. 579 (1993) (No. 92-102); Brief for
the American Association for the Advancement of Science and the National
Academy of Sciences as Amici Curiae in Support of Respondent, at * 5, *6, *32,
Daubert v. Merrell Dow Pharm., Inc., 509 U.S. 579 (1993) (No. 92-102); Brief for
the United States as Amicus Curiae Supporting Respondent, at *19, 20,
Daubert v. Merrell Dow Pharm., Inc., 509 U.S. 579 (1993) (No. 92-102).
61 Daubert, 509 U.S at 591, 597. The last criterion is somewhat ambiguous,
but has generally been held to mean that the expert testimony must not only be
relevant but properly applied to the facts at hand. In Blackmun’s words:
55
56
4/23/2009 6:28 PM
56
ALB. L.J. SCI. & TECH.
[Vol. 19.1
The most important task for federal judges, uneasy with their
new responsibilities, was parsing Blackmun’s definition of
“reliability.” 62 Rightly contending that methodological quality
defied a straightforward definition, Blackmun said that the
judicial assessment of reliability should be flexible, and refrained
from creating tests or rules that would constrain how reliability
should be judged. 63 However, perhaps recognizing that many
federal judges would need some sort of guidance, he suggested
four criteria as dicta to guide lower courts in their assessment.
These “suggestions” were as follows:
Can the opinion be falsified?
Has the opinion appeared in a peer-reviewed journal?
Does the opinion present an error rate?
Is the opinion generally accepted among the scientific
community? 64
A brief examination of these factors shows the influence of the
philosopher Karl Popper, 65 the importance of peer review despite
its rejection as a formal requirement, 66 and the return of Frye,
The study of the phases of the moon, for example, may provide valid
scientific “knowledge” about whether a certain night was dark, and if
darkness is a fact in issue, the knowledge will assist the trier of fact.
However (absent creditable grounds supporting such a link), evidence that
the moon was full on a certain night will not assist the trier of fact in
determining whether an individual was unusually likely to have behaved
irrationally on that night.
Id. at 591.
Quoting Judge Becker from In re Paoli, 35 F.3d 717, 743 (3d Cir. 1994),
Professor Sanders helpfully points to Judge Becker’s opinion from Paoli:
“[A]nimal studies may be methodologically acceptable to show that chemical X
increases the risk of cancer in animals, but they may not be methodologically
acceptable to show that chemical X increases the risk of cancer in humans.”
Sanders, supra note 14, at 378 n.46.
62 Daubert v. Merrell Dow Pharm., Inc., 43 F.3d 1311, 1316 (9th Cir. 1995).
The law unfortunately conflates scientific validity and scientific reliability, as
well as general methodological quality, under the term “reliability.”
63 Daubert, 509 U.S. at 593, 594.
64 Id. at 593–94. When one compares the text of the Solicitor General’s brief
with the Court’s opinion, the influence of the former is hard to miss. See Brief
for the United States as Amicus Curiae Supporting Respondent, at *21–22,
Daubert v. Merrell Dow Pharm., Inc., 509 U.S. 579 (1993) (No. 92-102)
(discussing criteria for assessing admissibility of scientific evidence).
65 For a summary of what some philosophers of science see as Daubert’s overreliance on falsification, see Edmond & Mercer, supra note 27, at 81–89, 93.
66 For a summary of the problems with using peer review as a quality control
for methodology, see SHEILA JASANOFF, THE FIFTH BRANCH: SCIENCE ADVISERS AS
POLICYMAKERS 61–83 (Harvard University Press 1990).
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
57
though in a supporting role rather than a leading one. 67 Despite
the admonition that Daubert analyses should remain flexible,
some judges understandably rely on these dicta as a four-part
test to be applied in the face of unfamiliar expert testimony. 68
C. Daubert and the Theoretical Framework: Realist,
Constructivist, or Neither?
How does the Daubert regime treat the foundational concerns
of epistemology and judicial capacity? On epistemology, Daubert
is simply ambiguous. On one hand, Blackmun’s four informal
indicators of “good science” expressly ratify the realist position
that scientific methodology is the gateway to reliable evidence
and accurate decisions. On the other hand, several phrases in
the opinion suggest a more constructivist viewpoint. 69 For
example, the Court noted imperfections in the very indicia it
would later promulgate, such as citing Jasanoff’s work on the
failures of peer-review, 70 and praised cross-examination as a tool
for exposing inconsistencies that scientists—presumably bound
by a particular paradigm or their own institutional and scientific
interests—might not catch. 71 The adoption of judicial gatekeeping in Daubert is itself a nod towards the constructivist
position, since it proceeds on the expectation that the litigants’
67 The Daubert paradigm was clarified by two further Supreme Court
decisions: Gen. Elec. Co. v. Joiner, 522 U.S. 136, 143, 146–47 (1997), which
explained the appropriate standard of review for Daubert hearings, and Kumho
Tire Co. v. Carmichael, 526 U.S. 137, 141 (1999), which answered whether
Daubert was intended to apply to only “scientific” evidence, or to all expert
testimony. Together with Daubert, the three cases are sometimes referred to as
the “Daubert Trilogy.” Margaret A. Berger, The Supreme Court's Trilogy on the
Admissibility of Expert Testimony, in REFERENCE MANUAL ON SCIENTIFIC
EVIDENCE 9, 9–10 (LEXIS Publishing, 2d ed. 2000).
68 Daubert, 509 U.S. at 593–94. But see Lloyd Dixon & Brian Gill, Changes
in the Standards for Admitting Expert Evidence in Federal Civil Cases Since the
Daubert Decision, 8 PSYCHOL. PUB. POL'Y & L. 251, 282, 288, 299 (2002)
(showing additional reliability factors). In this study, slightly less than half of
Daubert decisions used the dicta to measure reliability. However, this finding
was not the result of judges using more flexible assessments of reliability, but
rather because many judges still relied on more traditional admissibility
requirements, such as prejudice or relevance, or because they ignored reliability
altogether. See generally Jennifer L. Groscup et. al., The Effects of Daubert on
the Admissibility of Expert Testimony in State and Federal Criminal Courts, 8
PSYCHOL. PUB. POL'Y & L. 339 (2002).
69 Caudill & Redding, supra note 19, at 737–42; Farrell, supra note 22, at
2198–208.
70 JASANOFF, supra note 66, at 61.
71 Daubert, 509 U.S. at 596–97.
4/23/2009 6:28 PM
58
ALB. L.J. SCI. & TECH.
[Vol. 19.1
experts cannot readily separate their scientific claims from their
own financial interests. 72
Daubert’s epistemology thus appears to be a mix of realism and
constructivism, though more as the result of “muddling through”
than a deliberate choice by the majority. 73 Certainly, a trial
judge who wished to find support for a realist or a constructivist
epistemology within Daubert could do so. This ambiguity has
been criticized as harmfully incoherent, 74 as well as accepted (if
not praised) on the grounds that judges have neither the time nor
the ability to resolve debates about the nature of science when
the philosophy of science itself lacks consensus. 75
On the question of judicial capacity, Daubert may require more
of federal judges than they have to give, raising problems the
Frye regime sidestepped. Daubert tells judges, to quote Michael
Beyond
Saks, to “figure out the science [for] yourself.” 76
Rehnquist’s concern that judges will misunderstand concepts
such as falsification, Learned Hand’s dictum remains in full
effect. 77 Discerning potential methodological shortfalls in the
aforementioned Bendectin litigation would require that the
deciding judge have a working knowledge of relative risk, in-vitro
studies,
epidemiology,
structural
analysis,
statistical
significance, the fallacies of temporal association, and the risks of
extrapolating from animal studies to human beings. And that
covers merely one portion of the substantive expertise in which a
generalist trial judge, who likely has had no formal scientific
72 Daubert, 43 F.3d at 1317.
Later refinements of Daubert increased the
formal recognition of constructivist factors. After the Court remanded the
Daubert case for reconsideration under the new standard, the trial judge again
excluded the plaintiff’s experts under Daubert. On appeal, the Ninth Circuit
upheld the exclusion, and added to the Daubert dicta a further consideration:
expert opinions that arose naturally from a scientist’s work should be viewed as
more reliable than those which arose during the trial process.
The
constructivist insight here is straightforward—while experts should be capable
of following the scientific method and doing good research when their financial
incentives depend on particular conclusions, we are rightly skeptical of their
ability to do so. Id. at 1317–19.
73 Sanders, supra note 14, at 390 (quoting Jasanoff, supra note 23, at 63);
Brian Leiter, The Epistemology of Admissibility: Why Even Good Philosophy of
Science Would Not Make for Good Philosophy of Evidence, 1997 BYU L. REV.
803, 810–12 (1997); David S. Caudill, Law and Science: An Essay On Links and
Socio-Natural Hybrids, 51 SYRACUSE L. REV. 841, 847 (2001).
74 Farrell, supra note 22, at 2198–207.
75 Caudill & Redding, supra note 19, at 762–66.
76 Saks, supra note 35, at 1139.
77 See supra notes 16 and 29 and accompanying text.
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
59
training, must assess the reliability of expertise. Needless to say,
one does not need to question the intelligence of federal judges to
wonder if they are up to the task. 78 Questions of institutional
capacity may be mitigated by repeated exposure to similar
Daubert motions. Federal judges also have access to training
seminars and resources such as the excellent Reference Manual
on Scientific Evidence, 79 though it remains an open question
whether they adequately take advantage of such opportunities. 80
My larger point is simply that as read, Daubert’s text does not
choose sides in either the ongoing debate between realism and
constructivism or in the argument as to whether admissibility
should be a liberal or restrictive standard. In the following
section, I will explain the likely results of this ambiguity, namely
the enhanced autonomy judges gain when applying a precedent
lacking both a rule-like structure and a clear message.
III. THE EMPTY VESSEL: DAUBERT AND THE LOCAL CONSTRUCTION
OF RELIABILITY
A. Law, Legal Realism, and Judicial Decision-Making
Since the early 20th century, political and legal scholars have
hotly debated the degree to which the law actually constrains,
guides, or shapes legal decision-making. 81 As with arguments
regarding epistemology, the range of debate is book-ended by a
pair of straw-man arguments.
On one end, judges
mechanistically interpret statutes and apply legal rules, finding
the law without making it. On the other, judges are no more
than legislators in black robes, free-wheeling creators of policy
who use the law as a cloak to blur their naked use of power.
Again moving from the poles of the continuum to the more
reasonable middle, one finds two basic schools of thought. Some
Atikian, supra note 28, at 1514–15; Paul S. Milich, Scientific and
Technological Evidence: Controversial Science in the Courtroom: Daubert and
the Law's Hubris, 43 EMORY L.J. 913, 918–19 (1994).
79 REFERENCE MANUAL ON SCIENTIFIC EVIDENCE, supra note 31.
80 A survey of Texas judges, for example, found that seventy percent of the
responding judges took “no continuing education or practical business
experience in the use and analysis of the reliability of scientific methodology.”
See Montz, supra note 28, at 111–12.
81 Steven L. Winter, Foreword: Foreword: On Building Houses, 69 TEX. L.
REV. 1595, 1595–96 (1991); Scott Altman, Beyond Candor, 89 MICH. L. REV. 296,
303–09 (1990).
78
4/23/2009 6:28 PM
60
ALB. L.J. SCI. & TECH.
[Vol. 19.1
distance from the latter extreme, one finds the proponents of the
“attitudinal model,” 82 inheritors of the legal realist school who
argue policy preferences and ideology are the single most
important factors in judicial decision-making at the Supreme
Court level, and an important factor for lower courts. 83 Their
theory is a simple one: laws, constitutional clauses, and
precedent are sufficiently indeterminate to allow judges to decide
cases according to extra-legal factors, particularly political
ideology. 84 This autonomy, it should be noted, requires a lack of
meaningful appellate oversight. The attitudinal model works
best in courts where institutional constraints are minimized,
such as the Supreme Court, where justices with life tenure, no
likely career ambitions beyond their current position, and no
superiors have little to fear from making any particular
decision. 85
Though the notion that judicial decision-making has a strong
ideological component is a consensus position among empirical
legal scholars (and perhaps by any serious observer of courts),
critics of attitudinalists argue the realists have overreached,
often reducing judicial decision-making to a single dimension. 86
This critique has led other scholars to examine alternative
models of decision-making, such as examining how judges make
strategic decisions to overcome collective action problems, or how
court institutions exert an independent effect on decision82 JEFFREY A. SEGAL & HAROLD J. SPAETH, THE SUPREME COURT AND THE
ATTITUDINAL MODEL 64–65 (Cambridge University Press 1993). The book can
be summed thus: “Rehnquist votes the way he does because he is extremely
conservative; Marshall voted the way he did because he is extremely liberal.”
Id. at 65. It should be noted that the initial claims of attitudinalists—that
political ideology overwhelmingly explains judicial behavior—have been dialed
down in more recent years. Ideology is likely the most influential factor in
Supreme Court decisions—particularly in constitutional and civil liberties
cases—but by itself only explains less than one-third of such decisions. Sean
Wilson, The Attitudinal Model, Political Science, Ecological Fallacy and
Exaggeration 2–6 (Aug. 3, 2006) (unpublished manuscript, Pennsylvania State
University), available at http://ssrn.com/abstract=922183.
83 See, e.g., BENJAMIN N. CARDOZO, THE NATURE OF THE JUDICIAL PROCESS
127–28 (Yale University Press 1921); JEROME FRANK, COURTS ON TRIAL: MYTH
AND REALITY IN AMERICAN JUSTICE 401–03 (Princeton University Press 1949);
JEROME FRANK, LAW AND THE MODERN MIND 42, 46, 54–55, 284, 293 (Tudor Pub.
Co. 1936).
84 Wilson, supra note 82, at 2–3.
85 Segal & Spaeth, supra note 82, at 69–72.
86 Richard A. Brisbin, Jr., Slaying the Dragon: Segal, Spaeth, and the
Function of Law in Supreme Court Decision Making, 40 AM. J. POL. SCI. 1004,
1007, 1012 (1996).
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
61
making. 87 One promising approach which focuses on precedent
as a type of institution views landmark rulings not as rules that
constrain behavior, but rather as “legal regimes” 88 that constitute
The
the frameworks and boundaries of decision-making. 89
authors of this approach have convincingly argued that the
Supreme Court voted differently on core speech cases following
the creation of the “two-track” “content-neutral[ity]” and
“content-based” regime in Police Dept. of Chicago v. Mosley and
Grayned v. Rockford. 90 In other words, while the legal regime of
content discrimination does not dictate the result of particular
cases or remove the ideological component from such decisions, it
has changed the manner in which the Justices approach and
consider core speech cases, in turn affecting the general pattern
of outcomes. 91 In a similar (albeit qualitative) vein, Howard
Gillman persuasively contends that legal ideology regarding the
proper use of state police powers (here seen as distinct from the
standard
liberal-conservative
continuum
along
which
th
attitudinalists situate their models) heavily influenced 14
amendment jurisprudence between the Civil War and the New
Deal. 92 Either a landmark precedent or a persuasive legal
ideology, therefore, may temper the standard liberal and
87 LEE EPSTEIN & JACK KNIGHT, THE CHOICES JUSTICES MAKE 57 (CQ Press
1998); Howard Gillman & Cornell W. Clayton, Beyond Judicial Attitudes:
Institutional Approaches to Supreme Court Decision-Making, in SUPREME COURT
DECISION MAKING 1, 2–3 (Cornell W. Clayton & Howard Gillman eds.,
University of Chicago Press 1999).
88 “Jurisprudential regimes,” for the purposes of this article as elsewhere, are
referred to as “legal regimes.” Mark J. Richards & Herbert M. Kritzer,
Jurisprudential Regime in Supreme Court Decision Making, 96 AM. POL. SCI.
REV. 305, 307 n.1 (2002).
89 Id. at 305–07, 315–16.
90 Id. at 310–15. Compare Police Dep’t of Chicago v. Mosley, 408 U.S. 92, 96–
102 (1972) (using the content-based approach) with Grayned v. Rockford, 408
U.S. 102, 113–21 (1972) (using the content-neutral approach).
91 Richards & Kritzer, supra note 88, at 315–16. Richards and Kritzer have
successfully replicated this approach for search and seizure Supreme Court
decisions centered on Mapp v. Ohio, 367 U.S. 643 (1961), as well as
establishment clauses Supreme Court decisions centered on Lemon v.
Kurtzman, 403 U.S. 602 (1971). Herbert M. Kritzer & Mark J. Richards, The
Influence of Law in the Supreme Court's Search-and-Seizure Jurisprudence, 33
AM. POL. RES. 33, 35–37, 39, 51 (2005); Herbert M. Kritzer & Mark J. Richards,
Jurisprudential Regimes and Supreme Court Decision Making: The Lemon
Regime and Establishment Clause Cases, 37 L. & SOC'Y REV. 827 passim (2003).
92 See HOWARD GILLMAN, THE CONSTITUTION BESIEGED: THE RISE AND DEMISE
OF LOCHNER ERA POLICE POWERS JURISPRUDENCE 14, 62–73 (Duke University
Press 1993). Howard Gillman is a faculty member at University of California, a
nationally recognized scholar, and an expert on political and judicial affairs.
4/23/2009 6:28 PM
62
ALB. L.J. SCI. & TECH.
[Vol. 19.1
conservative outlooks attitudinalists prescribe to judges.
How likely is Daubert to affect judicial decision-making, given
what we know about judicial decision-making? From a greatly
truncated description of a vast literature, let me draw the
following three observations. First, institutional context matters.
An appellate court judge, for example, will feel more pressure to
adhere to precedent than a Supreme Court justice. Second,
important constitutive precedents such as Mosley, or to borrow
from Kritzer and Richards, “legal regimes,” 93 may have greater
influence on lower court decisions when they are formalistic. 94 If
precedents and prior interpretation do constitute the grounds for
debate and decree the mode of analysis, it seems reasonable to
assume that the grounds for debate—and thus the degree of
autonomy for judges—will be broader under a balancing or a
totality of the evidence analysis than under a more formal
method, such as the three-part test found in Brandenburg v.
Ohio. 95
Third, textual or doctrinal ambiguity may be offset by shared
visions of legal ideology or legal goals, goals distinct from (if often
overlapping with) liberal or conservative policy agendas. In other
words, law does not consist simply of doctrine and tests, but also
a shared “state of mind” among legal practitioners. 96 To draw an
example from constitutional law, the chance of a racially
discriminatory law surviving strict scrutiny is low not only
because of the difficulty in crafting a law that is sufficiently
narrowly tailored to address a compelling interest, but also
because of the shared understanding of post-Warren Court
judges that such laws are generally illegitimate. 97 Likewise,
th
interpretations of the 10 Amendment have depended greatly on
competing understandings of whether the Court should
demarcate and enforce the boundaries of federalism, or whether
those boundaries should be set by voters. 98 Once rule-based
Richards & Kritzer, supra note 88, at 307 n.1.
See Guthrie et al., supra note 12, at 40–43; ADRIAN VERMEULE, JUDGING
UNDER UNCERTAINTY: AN INSTITUTIONAL THEORY OF LEGAL INTERPRETATION 4–5
(Harvard University Press 2006).
95 Brandenburg v. Ohio, 395 U.S. 444, 447–48 (1969).
96 Howard Gillman, What's Law Got to Do With It? Judicial Behavioralists
Test the "Legal Model" of Judicial Decision Making, 26 LAW & SOC. INQUIRY 465,
486 (2001).
97 See, e.g., Hunter v. Underwood, 471 U.S. 222, 228–33 (1985) (striking down
a voting exclusion statute with a discriminatory motivation).
98 See, e.g., Printz v. United States, 521 U.S. 898, 920–35 (1997) (holding that
the federal government may not commandeer the states into administering or
93
94
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
63
federalism or political federalism gained currency among a
majority of the Court, that philosophy animated future doctrine,
further clarifying what the Court expects from lower courts. A
legal regime that creates such a shared understanding should
constrain or guide judicial decision-making more than a
precedent whose meaning is contested, or worse, unclear,
regardless of the rule, test, or analysis involved.
B. The Institutional Context of Daubert Decisions
How do these considerations apply to admissibility decisions
under the Daubert regime? As far as institutional context goes,
federal trial court judges making admissibility decisions operate
largely free from the fear of reversal. If Daubert decisions do
reach an appellate court, the legal standard for appellate review
is whether the judge’s decision was an “abuse of discretion,” a
deferential rule. 99 The appellate judge clearly has some wiggle
room in deciding what constitutes such an abuse, but the
Supreme Court has made it clear in Joiner that it expects
significant deference from its appellate courts. 100 Moreover,
admissibility decisions may lack the salience that would lead to
active policing of Daubert decisions by busy circuit judges,
especially given that appellate judges may correctly determine
that trial judges are “closer to the facts” than they are. One
recent study of Daubert decisions in federal torts supports this
theory, finding no evidence that judges act strategically to avoid
being overturned by appellate courts. 101
A quick tabulation of the affirmance rate of federal trial-level
evidentiary decisions also supports this hypothesis. Between
2002 and 2005, for example, the affirmance rate of appellate
courts for Daubert decisions at the trial level sat at just under
ninety percent. 102 Of course, this high affirmance rate does not
enforcing federal statutes); Garcia v. San Antonio Metro. Transit Auth., 469
U.S. 528, 554 (1985) (holding that the state transit authority was not immune
from minimum wage and overtime requirements of the Fair Labor Standards
Act); Cornell W. Clayton & J. Mitchell Pickerill, Guess What Happened on the
Way to Revolution? Precursors to the Supreme Court's Federalism Revolution, 34
PUBLIUS 85, 85–89 (2004).
99 Gen. Elec. Co., 522 U.S. at 139–47.
100 See id. at 142–43.
101 Buchman, supra note 7, at 688.
102 Peter J. Neufeld, Temporal Trends (?) in Appellate Review of Daubert
Decisions, BLOG 702, Feb. 6, 2005, http://www.daubertontheweb.com/2005/02/
temporal-trends-in-appellate-review-of.html.
There is a slight chance—as
4/23/2009 6:28 PM
64
ALB. L.J. SCI. & TECH.
[Vol. 19.1
count the numerous decisions that are never appealed. While
judges may not be aware of this exact rate, they are surely aware
of the magnitude of its effect, making them secure that their
admissibility determination is unlikely to be successfully
challenged. In short, the institutional context for admissibility
decisions is one of considerable autonomy. Judicial autonomy, in
turn, permits wide variation among judges in interpreting and
implementing Daubert in different contexts.
C. Daubert’s Fractured Stance on Admissibility
1. Admissibility in the Daubert Opinion
How does Daubert fare in providing doctrinal guidance for trial
judges? Here again, the nature of the Daubert regime increases
the probability of fragmentation, for as doctrine, Daubert is
severely underdetermined.
The decision not only eschews
formalism; it explicitly denies that its gate-keeping responsibility
can be formulated in any sort of rule-like language. Such
avoidance seems reasonable from a philosophic standpoint, as
asking a legal test to duplicate the “scientific method” when there
is little agreement on exactly what that entails is a doomed
enterprise. Nevertheless, as empty doctrine, Daubert essentially
makes each judge the master of his own fiefdom, absent on-point
precedent from the relevant circuit. The lack of doctrinal clarity
should also decrease the chance of reversal (again absent onpoint precedent), as finding an abuse of discretion may be more
difficult to justify when the standards are hopelessly vague.
Daubert observers might contend that Blackmun’s dicta—the
“considerations” of testing, peer-review, error rate, and general
acceptance—serve as a rough formal test that mitigates these
effects. A harried judge might simply ask whether the expert’s
testimony is testable, is peer-reviewed, presents an error rate, is
generally accepted, and be done. Two objections prevent this
claim from gaining traction. First, while use of the Daubert dicta
remains more common than the use of other indicia of arguments
about reliability, they appear in less than half of all such
Neufeld concedes—that his study may omit those few appellate decisions whose
opinion lacked the term “Daubert.” However, even if these omissions raise the
possibility of selection bias, slanting the win rate in a particular direction,
Neufeld argues that as probable trite affirmations, their inclusion would
increase, not decrease, the win rate.
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
65
admissibility decisions. 103
Frequency of use aside, even if one treats the dicta as doctrine,
the Daubert opinion provides little aid to their interpretation.
Judges may not even understand concepts such as
falsifiability, 104 especially if comprehension requires not only
adequately defining the term but understanding the philosophic
debates regarding its application. 105 The difference between one
reasonable interpretation and another might mean the difference
in admitting or excluding the evidence before the court. For
example, do testing and falsification require that the expert’s
opinion have been actually tested? That it is capable of being
tested? Or does a history of prior admission in a federal court
serve as “adversarial testing” that satisfies the criterion? 106
Similarly, does peer review require publication within “an
unbiased and financially disinterested community of
practitioners?” 107 Or may peer review literally require only that
expert opinions be reviewed by one’s physical peer, as has been
the case for some fingerprint technicians? 108
Moving to error rates, does the error rate prong require a
formal probability, as with DNA cases? 109 Or can it be satisfied
by the expert’s (laughable) claim that the error rate of a
103 See Christina L. Studebaker & Jane Goodman-Delahunty, Expert
Testimony in the Courts: The Influence of the Daubert, Joiner, and Kumho
Decisions, 8 PSYCHOL. PUB. POL'Y & L. 251, 228 (2002).
104 See Montz, supra note 28, at 107–15.
Montz discusses a survey of a
number of Texas state judges, finding their understanding of falsification, for
example, was limited at best.
D. H. Kaye, On “Falsification” and
“Falsifiability”: The First Daubert Factor and the Philosophy of Science, 45
JURIMETRICS J. 473, 475–81 (2005). It is possible that federal judges have a
better understanding of such terms than their state counterparts.
105 See Haack, supra note 19, at S66–68; Edmond & Mercer, supra note 27, at
93–95; David Ozonoff, Epistemology in the Courtroom: A Little "Knowledge" Is a
Dangerous Thing, 95 AM. J. PUB. HEALTH S13, S13–14 (2005).
106 See, e.g., United States v. Havvard, 117 F. Supp. 2d 848, 854 (S.D. Ind.
2000) (holding that latent finger print identifications are considered reliable
evidence under the Daubert test).
107 See, e.g., United States v. Brewer, No. 01 CR 892, 2002 U.S. Dist. LEXIS
6689, at *24 (N.D. Ill. Apr. 12, 2002) (holding that the government failed to
show the reliability of the handwriting comparison testimony under the
Daubert test).
108 See, e.g., Havvard, 117 F. Supp. 2d at 854 (discussing the application of
Daubert to latent fingerprint identification).
109 See, e.g., United States v. Ewell, 252 F. Supp. 2d 104, 113–14 (D.N.J.
2003) (holding that PCR/STR technology used for DNA identification has been
subjected to peer review and has been generally accepted in satisfaction of
Daubert).
4/23/2009 6:28 PM
66
ALB. L.J. SCI. & TECH.
[Vol. 19.1
particular technique is zero? 110 Finally, must general acceptance
analysis extend beyond the agreement of a small expert
community whose financial and employment interests are bound
up in a technique’s acceptance? 111 Or is general acceptance
intended to examine what technique that particular community
Judges may offer significantly different
supports? 112
interpretations of these dicta, with little guidance from the
Daubert decision or the Supreme Court as to which
interpretations are more appropriate. Such differences make it
unlikely that use of the dicta as a rough and ready doctrinal test
could elucidate an analysis the Court has chosen to make opaque.
Daubert’s vacuous doctrine could still be limited by signals
from the Supreme Court that it desired a general position on the
admission of experts, whether liberal or restrictive. Blackmun’s
recently released papers suggest the Court did intend to use
Daubert to clarify the proper stance on admissibility. 113 Despite
such intent, his opinion raises more questions than it answers.
Daubert supports both a liberal and a restrictive judicial stance
on admissibility, depending on what parts of the opinion one
cites. In his attempts to mollify both the vociferous critics of
“junk science,” 114 as well as those who worried that a reliability
standard would “sanction a stifling and repressive scientific
orthodoxy,” 115 Blackmun equivocated on whether the Daubert
standard was a liberal or a more restrictive one.
Support for a liberal standard, for example, can be found in
Blackmun’s assurance that removing the Ninth Circuit’s “general
acceptance” and peer-review requirements as the formal
110 See, e.g., United States v. Plaza, 179 F. Supp. 2d 492, 515–16 (E.D. Pa.
2002) (holding that expert fingerprint testimony would be allowed, but not
expert testimony that a particular latent print matches); Havvard, 117 F. Supp.
2d at 854–55 (holding that despite the absence of a single quantifiable standard
for measuring the sufficiency of any latent print for purposes of identification,
latent print identification easily satisfies the standards of reliability in Daubert
and Kumho). Note that in Plaza I, Judge Pollack rejected the expert’s claim,
though he would later reconsider his motion to limit fingerprint expert
testimony because he came to believe that the FBI, at least, had sufficiently
reliable procedures. United States v. Plaza, 188 F. Supp. 2d 549, 566, 571, 576
(E.D. Pa. 2002).
111 Cole, supra note 8, at 1216.
112 Id. at 1246; Epstein, supra note 8, at 646.
113 E-mail from Michael Saks, Professor of Law, University of Iowa College of
Law, to Robert Robinson, Assistant Professor of Political Science, University of
Alabama-Birmingham (July 1, 2007, 10:25 PM) (on file with author).
114 Huber, supra note 19, at 2–6.
115 Daubert, 509 U.S. at 596.
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
67
standard of admissibility would not lead to “a ‘free-for-all’ in
which befuddled juries are confounded by absurd and irrational
pseudoscientific assertions.” 116 Blackmun instead counseled that
the traditional tools of the adversarial system, such as crossexamination and jury instructions, would suffice to supplement
Daubert analysis. 117 Blackmun’s support for the jury system and
traditional legal safeguards against unreliable evidence, as well
as his position that a liberal standard of admissibility was most
consistent with the spirit of the Federal Rules of Evidence,
convinced some legal scholars that Daubert ultimately took a
“liberal” stance on admissibility. 118 Similarly, in executing their
gate-keeping responsibilities in the years immediately following
Daubert, several district courts specifically noted the “liberal”
nature of the Federal Rules and the Daubert decision in making
their decision. 119
However, Blackmun’s opinion also supports a more
conservative or restrictive interpretation of the gate-keeping
responsibility. Unlike scientific arguments, legal disputes must
be resolved “finally and quickly,” Blackmun says, and hypotheses
that are “probably wrong” will be “of little use” to courts. 120
Moreover, Blackmun accepted Merrell Dow’s position that the
word “science” in Rule 702 obligates judges to apply some
reliability analysis, one presumably based on traditional realist
criteria. 121 Given this apparent elevation of judges to gatekeepers of reliability—a view itself strengthened by Rehnquist’s
Id. at 595–96.
Id. at 596.
118 See Brian Stuart Koukoutchos, Solomon Meets Galileo (And Isn't Quite
Sure What to Do With Him), 15 CARDOZO L. REV. 2237, 2246 (1994); see also
Anthony Z. Roisman, Conflict Resolution in the Courts: The Role of Science, 15
CARDOZO L. REV. 1945, 1951 (1994) ("A categorical refusal even to examine and
consider scientific evidence … is a recipe for error in any forum.").
119 See, e.g., Mitchell v. Gencorp, Inc., 968 F. Supp. 592, 594 (D. Kan. 1997);
Higgins v. Diversey Corp., 998 F. Supp. 598, 601 n.6 (D. Md. 1997); Pick v. Am.
Med. Sys., Inc., 958 F. Supp. 1151, 1155, 1162 (E.D. La. 1997).
120 Daubert, 509 U.S. at 597.
While Blackmun is correct that legal and
scientific practice differ in their methods for ascertaining truth, he greatly overidealizes scientific practice in this statement. Funding, tenure, and scholarly
competition, for example, put rather discrete time limits on scientific
exploration in the short-term, just as path dependency, cultural paradigms, and
scientific organizations do so in the long run. See, e.g., Caudill & Redding,
supra note 19, at 729, 744–45; Jasanoff, supra note 24, at 347, 349, 358; BRUNO
LATOUR & STEVE WOOLGAR, LABORATORY LIFE: THE SOCIAL CONSTRUCTION OF
SCIENTIFIC FACTS 27, 31–32, 36–37 (Sage Publications 1979); PETER L GALISON,
HOW EXPERIMENTS END 277 (University of Chicago Press 1987).
121 Daubert, 509 U.S. at 589–91.
116
117
4/23/2009 6:28 PM
68
ALB. L.J. SCI. & TECH.
[Vol. 19.1
explicit concern over the same—other legal scholars viewed
Daubert as establishing a more conservative stance than had
been true before it. 122 Such authors either celebrated that federal
courts would now take a harder look at “junk” testimony
plaguing civil and criminal cases 123 or worried that this stance
would raise the costs of bringing a case to trial, aiding corporate
defendants more than anything else. 124 Support for a more
restrictive position on admissibility can also be found in the
second and third cases of the “Daubert trio:” G.E. v. Joiner and
Kumho Tire v. Carmichael. 125 While Joiner and Kumho are
better known for their clarifications of Daubert standards than
their factual situations, the Court’s own application of its
standards to the cases before them better fit a conservative
stance than a liberal one. 126
The split regarding Daubert’s position on admissibility has,
after almost fifteen years, given way to more nuanced arguments
about Daubert’s admissibility standards being liberal or
conservative in particular contexts. For example, one emerging
scholarly consensus views federal courts as taking a relatively
liberal position in criminal cases (particularly where the state’s
evidence is concerned) while being much more restrictive in civil
122 George Lakoff, A Cognitive Scientist Looks At Daubert, July 29, 2005,
http://www.rockridgeinstitute.org/research/lakoff/daubert/.
123 David E. Bernstein, The Admissibility of Scientific Evidence After Daubert
v. Merrell Dow Pharmaceuticals, Inc., 15 CARDOZO L. REV. 2139, 2139 (1994);
see also Randolph Jonakait, The Meaning of Daubert and What That Means for
Forensic Science, 15 CARDOZO L. REV. 2103, 2110 (1994) (stating that trial courts
must now determine whether "scientific testimony is indeed scientific").
124 Roisman, supra note 118, at 1951.
125 Gen. Elec. Co., 522 U.S. at 136; Kumho Tire Co., 526 U.S. at 153 (finding
that despite an expert's qualifications, his methodology was unreliable, and
thus the Court concurred with the District Court's exclusion of his testimony);
Berger, supra note 67, at 38.
126 Rehnquist’s opinion in Joiner required that each piece of the plaintiff’s
expert testimony be reliable in its own right, rather than reliable as a whole.
See generally Joiner, 522 U.S. at 142–47 (agreeing with the District Court "that
the studies upon which the experts relied were not sufficient … to support their
conclusions …."). Breyer’s opinion in Kumho found the plaintiff’s expert to be
qualified, and his method of visually inspecting tires to look for defects a
reasonable one, but rejects his argument that because no other causes for the
blowout could be found, a manufacturing defect must have been the cause.
Kumho, 526 U.S. at 153–58. These two types of arguments—requiring that
each piece of evidence be adequate evidence of causation on its own, and
requiring that an expert “rule in” a factor as a cause by substantive evidence
rather than simply “ruling out” other potential causes—have frequently been
used to exclude plaintiff expert testimony under Daubert. See generally Grimes
v. Hoffmann-LaRoche, Inc., 907 F. Supp. 33, 37–38 (D.N.H. 1995).
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
69
cases. 127
Another possibility is that federal courts
“grandfathered” previously admitted expert testimony by
providing it with a liberal standard, while requiring genuinely
new expert testimony to meet a more restrictive standard. 128 In
any case, the possibility of Daubert creating a globally shared
understanding of admissibility seems remote.
As with
institutional context, the Supreme Court’s own doctrinal
ambiguity, as well as division among influential commentators,
contributes to fragmentation and the local construction of
reliability.
2. Contrasting Admissibility Standards in United States v. Crisp
As an example of the degree to which divergent positions on
expert admissibility may be taken from Daubert, I will turn to
United States v. Crisp, a Fourth Circuit appeal challenging the
admission of the government’s fingerprint and handwriting
comparison experts. 129 Crisp had been convicted of bank robbery,
largely on the strength of accomplice testimony, latent prints
drawn from palm prints, and handwriting comparisons of a note
Crisp allegedly wrote to another accomplice. 130 The fingerprint
and handwriting expertise was central to the verdict. As the
majority concedes, “[t]he credibility of both Mitchell and Torain
[the co-conspirators] was substantially impeached,” and the
government itself conceded that the expert testimony “was
necessary to prove Crisp’s guilt beyond a reasonable doubt.” 131
Chief Judge Wilkins joined Judge King in affirming the district
court’s decision to admit the government’s experts, while Judge
Michael dissented. 132 More striking than the split decision,
however, was the manner in which each opinion envisioned the
Daubert gate-keeping responsibility. King began by citing some
recent precedent in which fingerprint expertise had been
admitted under Daubert, as well as placing significant
importance on the fact that fingerprint evidence had been
admissible for decades. 133 Moving to the Daubert dicta, King
127 See Berger, supra note 7, at S63–64; see also Studebaker, supra note 7, at
256, 263, 300–01; Giannelli, supra note 8, at 1073; Neufeld, supra note 8, at
S109.
128 Cole, supra note 8, at 1272.
129 United States v. Crisp, 324 F.3d 261, 265 (4th Cir. 2003).
130 Id. at 263–65.
131 Id. at 264 n.1, 282.
132 Id. at 263.
133 Id. at 266.
4/23/2009 6:28 PM
70
ALB. L.J. SCI. & TECH.
[Vol. 19.1
applied these criteria to the state’s evidence in an extremely
deferential fashion. 134 Referencing a similar opinion in United
States v. Havvard, 135 the majority argued that despite its lack of
empirical testing, fingerprint evidence and handwriting evidence
had been sufficiently “tested in the adversarial system.” 136 In
other words, if there were methodological problems with such
evidence, or if their premises were false, surely several decades
worth of cases would have caught them.
King then glided over the peer review requirement, only citing
Havvard’s statement that an individual technician’s work was
“reviewed” by one of his peers. 137 When addressing the error rate
requirement, King’s opinion simply accepted at face value the
risible claim by the government that fingerprinting has an error
rate of zero. 138 Given that even DNA identification presents nonzero error rates, no judge should accept such a claim. 139 Finally,
addressing the general acceptance prong, the majority noted
that, “[w]hile the principles underlying fingerprint identification
have not attained the status of scientific law, they nonetheless
bear the imprimatur of a strong general acceptance, not only in
the expert community, but in the courts as well.” 140 Expert
communities whose interests and incentives depend entirely on
the acceptance of a particular technique do not provide a good
measure of its reliability or validity. Worse, the position that
general acceptance can rest on acceptance by courts in effect
“grandfathers” certain techniques, exempting them from actual
gate-keeping. 141
The most arresting fact about the majority opinion, however, is
its assumption that the defendant was required to demonstrate
Id. at 265–70.
United States v. Havvard, 260 F.3d 597, 600–01 (7th Cir. 2001).
136 Crisp, 324 F.3d at 266; Havvard, 260 F.3d at 601.
137 Crisp, 324 F.3d at 266, 269, 270–71.
138 Id. at 269.
139 Id. at 278. One response to this claim is that while the technique itself
produces no errors, individual examiners may, of course, err. However, since
the abandonment of the “points” system and the movement towards the more
subjective ACE-V technique pioneered by Ashbaugh, the examiner essentially is
the technique. DAVID R. ASHBAUGH, QUANTITATIVE-QUALITATIVE FRICTION RIDGE
ANALYSIS: AN INTRODUCTION TO BASIC AND ADVANCED RIDGEOLOGY 196–97 (CRC
Press 1999). Much like a harbor pilot, one cannot readily separate the
examiner from the technique. The same is true for handwriting expertise,
where even the majority opinion in Crisp implicitly concedes that there is no
uniform method or set of standards.
140 Crisp, 324 F.3d at 268.
141 Cole, supra note 8, at 1244–45, 1274.
134
135
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
71
the prosecution’s experts were not reliable, rather than the
converse. Confronted with the accusation that the foundations
and methods of fingerprint comparison lack adequate empirical
testing, Judge King responded that “Crisp cites no studies
demonstrating the unreliability of fingerprinting analysis . . . ” 142
Wrapping up the Court’s discussion of the Daubert dicta, King
wrote that, “[w]hile Crisp may be correct that further research,
more searching scholarly review, and the development of even
more consistent professional standards is desirable, he has
offered us no reason to reject outright a form of evidence that has
so ably withstood the test of time.” 143 King then framed Daubert
as a liberal standard, citing Blackmun’s dictum that “[v]igorous
cross-examination, presentation of contrary evidence, and careful
instruction on the burden of proof are the traditional and
appropriate means of attacking shaky but admissible
evidence.” 144
Why is this opinion so deferential? King showed his hand
when he accused that “Crisp today advocates the wholesale
exclusion of a long-accepted form of expert evidence. Such a
drastic step is not required of us under Daubert, however, and we
decline to take it.” 145 Later, he similarly wrote that “to postpone
present in-court utilization of this ‘bedrock forensic identifier’
pending such research would be to make the best the enemy of
the good.” 146 King clearly feared the institutional consequences
of excluding forensic evidence regularly utilized by the state; the
degree to which this fear influenced his decision is, of course,
unknown.
By contrast, the dissent put teeth into the Daubert dicta and
found the governments’ expertise wanting. Judge Michael
argued that viewing the test prong as adversarial testing, or the
view of general acceptance as acceptance in court misconstrues
the judiciary’s gate-keeping responsibilities: “[o]ur adversarial
system has much to commend it, but it is not a general substitute
for the specific Daubert inquiry.” 147 If, on the other hand, testing
Crisp, 324 F.3d at 267.
Id. at 269.
144 Id. at 269–70 (quoting Daubert, 509 U.S. at 596).
145 Crisp, 324 F.3d at 268.
146 Id. at 270 (quoting Llera Plaza, 188 F. Supp. 2d at 572).
147 Crisp, 324 F.3d at 273 (Michael, J., dissenting). Michael notes that at its
inception, fingerprint testimony was admitted largely without challenge,
allowing it to build a reputation of reliability that no lawyer—particularly no
lawyer representing an indigent like Crisp—would directly challenge (or even
142
143
4/23/2009 6:28 PM
72
ALB. L.J. SCI. & TECH.
[Vol. 19.1
requires actual empirical testing, peer review requires
publication in peer-reviewed journals, and general acceptance
requires acceptance beyond an expert community whose
livelihood would be threatened by exclusion, Michael said, the
prosecution’s experts fail on all counts. 148 The foundations of
these two techniques—such as the probability that a latent print
might match more than one full set of exemplars, or whether
one’s handwriting is fixed, unique, and capable of individuation—
have not been tested, nor have their propositions been put
forward for review in scientific (rather than trade) journals.
Some semblance of an error rate might be found in the discussion
of proficiency tests, Michael noted, but the particular tests cited
either lacked external validity (i.e., they were too easy), or only
marginally established expert proficiency. 149 Concluding that the
government had failed to satisfy Daubert, Michael voted to
exclude both sets of testimony. “The government has had ten
years to comply with Daubert. It should not be given a pass in
this case.” 150
It would be tempting to conclude that the dissent has applied
the “true” meaning of Daubert and the majority has misread it;
tempting, but mistaken. The majority is certainly correct on one
count: its approach to Daubert squares with the manner in which
most federal courts have treated most attacks on the
How could the
government’s experts in criminal cases. 151
majority federal decisions overwhelmingly advance the “wrong”
interpretation of Daubert and suffer no consequences? While I
would argue the majority’s interpretation of terms such as
“testing” and “general acceptance” is laughably weak, King’s
insistence that the Daubert court did not intend a radical
restructuring of expert testimony in criminal cases may be
correct. In any case, the point is not that one side or another
applied the “correct” interpretation of Daubert, but instead that
know to challenge). SIMON A. COLE, SUSPECT IDENTITIES: A HISTORY OF
FINGERPRINTING AND CRIMINAL IDENTIFICATION 280 (Harvard University Press
2001) (explaining how in more than forty criminal cases fingerprint evidence
went unchallenged despite significant discrepancies).
148 Crisp, 324 F.3d at 274–76 (Michael, J. dissenting).
149 Id. at 273–75. See Saks, supra note 35, at 1073–77; D. Michael Risinger
et al., The Daubert/Kumho Implications of Observer Effects in Forensic Science:
Hidden Problems of Expectation and Suggestion, 90 CAL. L. REV. 1, 32–33 (2002)
(arguing that forensic proficiency tests are too easy).
150 Crisp, 324 F.3d at 272.
151 See Neufeld, supra note 8, at 109; Risinger, supra note 8, at 99.
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
73
Daubert has no correct interpretation. Rather than create a clear
and consistent stance on the standard by which trial courts
should admit expert testimony, the Daubert regime allows
individual courts or circuits to develop localized understandings
of admissibility.
D. Daubert’s Fractured Stance on Epistemology
A final factor that might unify Daubert’s admissibility regime
is the Supreme Court’s stance on what constitutes good science.
As with its doctrine and position on admissibility, however,
Daubert’s stance on epistemology can justify multiple
interpretations. In one scholar’s terms, Daubert encourages a
“pragmatic constructivism” about epistemological issues,
sidestepping direct consideration of epistemology and inviting
lower courts to do the same. 152 Even were one to read into
Daubert a particular philosophy of science, however, I argue here
that epistemology is likely to fade before other, more salient
th
judicial goals. Unlike Gillman’s depiction of early 20 century
federal judges, many of whom were sincerely convinced that the
well-being of the Republic depended on constraining the use of
the police power to the public good, 153 it is hard to see judges
spending a great deal of capital or placing particular emphasis on
championing a particular epistemology.
As argued in Section II, Daubert does not rest on a single point
along the realist-constructivist continuum. The language of the
Daubert opinion gives judges considerable freedom to employ
either realist or constructivist factors as they see fit. Why would
the Court endorse such ambiguity, given that its apparent goal
was to resolve circuit conflict over the appropriate standard?
First, frankly put, engaging the philosophy of science is a difficult
task not only because it is complex, but because there is no
consensus position on what constitutes good science. When
constructivists criticize courts for naively relying on realist
indicia of reliability, they contend there is no toolbox of methods
that will readily allow judges to demarcate the wheat from the
chaff. 154 In their view, deference to realist criteria not only
Caudill & Redding, supra note 19, at 756.
GILLMAN, supra note 92, at 147–48.
154 See, e.g., Farrell, supra note 22, at 2199–2207; Sheila Jasanoff, What
Judges Should Know About the Sociology of Science, 32 JURIMETRICS J. 345,
358–59 (1992); Shana M. Solomon & Edward J. Hackett, Setting Boundaries
Between Science and Law: Lessons From Daubert v. Merrell Dow
152
153
4/23/2009 6:28 PM
74
ALB. L.J. SCI. & TECH.
[Vol. 19.1
creates an unrealistic and unattainable vision of scientific
certainty, but also leads courts to miss their own part in its social
construction.
In due turn, however, such critics overlook that the conclusions
one might draw from their own work are equally varied and
conflicting. 155 How would a generalist judge, unlikely to have
any training in epistemology or the sociology of science,
discriminate between the views of Bruno Latour, Peter
Feyerbrand, Peter Galison, and Thomas Kuhn in appropriating
the “right” lessons from the sociology of science? 156 Asking judges
to take a “constructivist” stance towards science is itself a
request with little internal consistency. If anything, such
constructivist arguments are likely to serve as tools for
pragmatic judges wishing to attack specific realist claims in
particular situations. 157 The judicial capacity for evaluating and
discriminating higher-order epistemological arguments is
arguably no greater than the judicial capacity for evaluating
high-level scientific disputes. It is difficult to blame the Supreme
Court for not jumping headfirst into such murky debates.
Second, even were a judge to take a relatively principled
“realist” or “constructivist” stance on scientific epistemology from
Daubert, choosing their favorite philosophers of science to
emulate, such a stance might not correlate with any expected
position on the admissibility of expert testimony. Constructivist
and realist positions may justify either a liberal or a restrictive
approach to the admission of expert testimony. Ultimately, the
decision to admit challenged expert testimony is a decision about
which party should bear the burden of uncertainty and risk. 158
For example, a judge inclined towards a constructivist view of
Pharmaceuticals, Inc., 21 SCI. TECH. & HUM. VALUES 131, 131–32 (1996).
155 See Caudill & Redding, supra note 19, at 743–44.
156 FEYERABEND, supra note 21, at 165–67; BRUNO LATOUR & STEVE WOOLGAR,
LABORATORY LIFE: THE SOCIAL CONSTRUCTION OF SCIENTIFIC FACTS 17–18 (Sage
1979); GALISON, supra note 120, at 258; 2 THOMAS S. KUHN, THE STRUCTURE OF
SCIENTIFIC REVOLUTIONS 1–3 (2d ed. University of Chicago Press 1970).
157 Edmond and Mercer, who surveyed judges’ use of various important
philosophers of science, found, unsurprisingly, that such use demonstrated little
understanding of that field’s development over the last few decades. See
Edmond & Mercer, supra note 27, at 97–100. Specifically, the authors track the
heavy influence of Karl Popper, whose work is largely responsible for the
falsification prong of the Daubert “test,” but whose importance in the actual
philosophy of science has faded. By contrast, recent leading lights of the field
were ignored. The authors conclude that the sociology of science has at best
failed to permeate the law, and at worst has been misappropriated. Id.
158 JASANOFF, supra note 23, at 44.
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
75
science might agree that there are multiple valid scientific
positions on contested issues, rather than right and wrong
answers. As such, she may be reluctant to exclude expert
testimony.
However, this heightened sense of scientific
uncertainty might just as well lead a different constructivistminded judge to become less liberal in his admissibility stance, as
greater uncertainty justifies the admission of only extremely
well-settled opinions. The decision of how one views science does
not in itself answer the question of who should bear the risk of
scientific uncertainty.
Third and most importantly, even if judges recognized,
understood, and purported to adopt a particular epistemology of
science, its impact on judicial decisions would still be limited.
Current research on judicial behavior suggests that judges work
to advance a variety of goals, 159 such as enacting their ideological
policy preferences, 160 engaging in strategic maneuvering to reach
those policy preferences, 161 advancing specific legal ideologies, 162
navigating specific institutional concerns, 163 and following
Though scholars hotly debate the relative
precedent. 164
importance of these factors, 165 it seems safe to say that
maintaining a consistent epistemological theory would sit at the
low end. I do not mean to argue that judges do not care about the
quality of expert testimony or its impact on the trials in their
courtroom; I simply imply that they will do so on a case by case
basis, rather than working from a grand theory. Whether this
occurs because judges are unaware of the theoretical basis for
their decisions, because they deliberately avoid epistemological
debates, or because they rationalize the use of a particular
epistemology to achieve particular goals, pragmatism is the
result.
Given these considerations, I see little chance for the
159 LAWRENCE BAUM, THE PUZZLE OF JUDICIAL BEHAVIOR 15–16 (University of
Michigan Press 1997).
160 See SEGAL & SPAETH, supra note 82, at 65.
161 See EPSTEIN & KNIGHT, supra note 87, at 11–12.
162 See GILLMAN, supra note 92, at 1.
163 See THE SUPREME COURT IN AMERICAN POLITICS: NEW INSTITUTIONALIST
INTERPRETATIONS 2 (Howard Gillman & Cornell Clayton eds., University Press
of Kansas 1999).
164 See Richards & Kritzer, supra note 91, at 827–28; THOMAS G. HANSFORD &
JAMES F. SPRIGGS II, THE POLITICS OF PRECEDENT ON THE U.S. SUPREME COURT 5
(Princeton University Press 2006).
165 THE PIONEERS OF JUDICIAL BEHAVIOR 90 (Nancy Maveety ed., University of
Michigan Press 2003).
4/23/2009 6:28 PM
76
ALB. L.J. SCI. & TECH.
[Vol. 19.1
emergence of a consistent epistemological understanding of
Daubert.
Different areas of testimony will raise different
arguments, bring forth different types of litigants, activate
different sorts of judicial goals, and lead judges to be more
receptive to specific realist or constructivist claims. Judges may
rely on both realist indicators and constructivist concerns,
depending upon the judicial goals in question and the type of
expertise before them. Local constructions of reliability may
indeed create shared understandings about how Daubert should
be carried out, in spheres as disparate as civil suits involving the
drug Paradol, discussion over the validity of hedonic damages, or
the reliability of handwriting expertise.
However, such
understandings will be constructed by judges sui generis.
Lacking clear doctrine, a clear stance on admissibility, and a
clear stance on epistemology, Daubert should allow judges to
construct the reliability of particular types of evidence in
accordance with a number of different variables, leading to what
are in effect different admissibility standards for different types
of evidence. This relative autonomy is further enhanced by the
institutional context of Daubert decisions, where even when they
occur, successful appeals are unlikely. As a result, we lack a
universal conception of reliability, instead having local ones.
IV. EMPIRICALLY MODELING DAUBERT DECISIONS
A. Judges as Goal-Oriented Actors
If doctrinal analysis cannot predict admissibility decisions,
must we simply compile these different constructions of
reliability and eschew general theorizing? Perhaps. However, a
more fruitful option would model Daubert decisions using the
insights of the judicial behavior literature. That the application
of Daubert itself cannot explain the variation among
admissibility cases does not mean that the variation cannot be
explained. Daubert analysis must join the manifold areas of law
to which empirical study has been applied if we are to
understand what drives the construction of reliability.
In this section, I suggest variables for building such a model. I
ground the model in the well-regarded theory that views judges
as goal-oriented actors in a particular institutional context. 166
Judges are neither cogs in the mechanistic application of law nor
166
See BAUM, supra note 159, at 13–14.
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
77
unfettered policy-makers; like any political actor, they have goals
that shape and are shaped by legal institutions (including the
law). I suggest three categories of goals in admissibility decisions
that might help identify relevant variables: (1) regulating the
“reliability” of expert testimony, (2) maintaining institutional
stability and autonomy, and (3) advancing one’s “policy
preferences.” Beyond these three categories, I add the potential
effect of on-point precedent, potential funding disparities among
various classes of litigants, and specific circuit rules or practices
that might shape admissibility decisions.
B. Legal Goals: Assessing Methodological Quality
First, I argue judges care about the quality, validity, or
reliability of the expert evidence before them.
Some
commentators state that the first goal of courts is justice, not
truth, and that an overly restrictive standard for expert
testimony would harm the courts’ ability to resolve social
disputes fairly. 167 Oddly, this position dismisses the connection
between finding truth and the perception of justice, that justice
rests not only on fairness but also on the belief that “correct”
decisions have been reached.
Downplaying truth in legal
decisions could have ugly consequences: punishing an
unsympathetic corporation on “social fairness” grounds could
lead to actual harm if its product, was, in fact, safe and helpful.
It may be true, of course, that in particular cases the uncertainty
regarding the accurate distribution of blame cannot be reduced,
and we will be stuck, as Jasanoff has argued, deciding on fairness
grounds who should assume the burden of risk. 168 Conceding
this, I still assume most judges would agree that “getting it right”
is an important part of their gate-keeping responsibilities, and
that there will be some concern about the reliability and validity
of the evidence before them.
In line with Section III, I argue that the realist indicia from the
Daubert dicta are too fluid to serve as independent variables. As
the Crisp case illustrates, there are no fixed meanings of terms
such as falsifiability, peer-review, or general acceptance. 169 If a
consensus on their meaning did arise within a particular circuit,
Farrell, supra note 22, at 2204–07.
JASANOFF, supra note 23, at 114–15.
169 Crisp, 324 F.3d at 266, 273–76 (“But ‘[t]he Daubert court did not suggest
that acceptance by a legal, rather than a scientific community, would suffice.’”
(quoting Starzecpyzel, 880 F. Supp. at 1038.
167
168
4/23/2009 6:28 PM
78
ALB. L.J. SCI. & TECH.
[Vol. 19.1
or on a particular issue, they might serve as reasonably reliable
variables, but such a consensus does not exist on a global level. I
aim here for a more parsimonious theory of judicial decisionmaking, and case-specific definitions of the dicta do not suffice.
Moreover, as epistemological pragmatists, judges will not restrict
their reliability analysis to the realist demarcations of good
science found in Daubert, but may also consider “constructivist”
insights—even if the judge is unlikely to understand them as
such (or care about the difference). If the model is to include
variables that measure judicial concern over reliability,
something beyond the dicta must be included.
What factors might a judge utilize as general proxies for
reliability? Without exhausting the possibilities, I suggest two
variables here.
First, faced with complex methodological
disagreements, a judge might trust his instincts (or prejudices)
and assume that particular classes of litigants are more likely to
proffer expert testimony of dubious quality. This assumption fits
with recent research suggesting that judges, like most people,
often rely on intuition for their assessments rather than
deliberation. 170 I assume that judges would be less likely to trust
civil plaintiffs—whose experts they might see as nothing more
than tools to advance the plaintiff’s chances of winning in the
“big casino” of litigation—than civil defendants. 171 This view of
civil plaintiffs has been advanced at least since the 1990s by
polemicists such as Peter Huber, who launched a war on “junk
science” in the civil court system. 172 Conversely, in criminal
trials, judges might find defendants the more likely party to
proffer unreliable expertise or baselessly attack experts proffered
by the state, as their freedom depends on it. 173
See Guthrie, supra note 12, at 2.
See ROY GRUTMAN & BILL THOMAS, LAWYERS AND THIEVES 57 (Simon &
Schuster 1990) (showing how the goal of a lawyer is to make his or her client a
beneficiary); HUBER, supra note 19, at 42–43, 55–56 (discussing the change in
judicial attitudes toward civil plaintiffs).
172 HUBER, supra note 19, at 182–83.
173 In other words, judges generally assume criminal defendants are guilty.
See JAMES EISENSTEIN & HERBERT JACOB, FELONY JUSTICE: AN ORGANIZATIONAL
ANALYSIS OF CRIMINAL COURTS 5–6 (Little, Brown & Company 1977) (describing
how judges' opinions "vary" with "their attitudes toward crime and criminals").
That said, even if defense experts can be discounted because of their clients’
desperation, government experts are also susceptible to pressures to help the
police and the prosecution secure convictions. See M. J. Saks et al., Context
Effects in Forensic Science: A Review and Application of the Science of Science to
Crime Laboratory Practice in the United States, 43 SCI. & JUST. 77, 83–85
(2003).
170
171
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
79
Evidence that litigant identity impacts judicial decisions can be
justified by more than just broad generalizations. Research by
Clermont and Eisenberg, for example, demonstrates that federal
appellate courts disparately favor civil defendants on appeal
because they (inaccurately) view trial courts as biased for
plaintiffs. 174 The nascent empirical literature on Daubert also
suggests admissibility decisions privilege civil defendants and
criminal prosecutors, although not for the same reasons. Civil
cases are overwhelmingly levied by civil defendants against civil
plaintiffs, with the latter losing the majority of these
challenges. 175 In the criminal sphere, by contrast, defendant
experts face the most challenges, even though prosecutors utilize
Like their civil plaintiff
the vast majority of experts. 176
counterparts, criminal defendants are also far, far more likely to
lose their challenges to plaintiff experts as well as have their own
experts excluded than the reverse. 177 Even accounting for the
reasonable objection that such disparities rest on a studied
consideration of the quality of the proffered expertise rather than
a heuristic distrust of particular litigant classes, litigant identity
may serve as a testable “constructivist” measure of reliability.
A second proxy for reliability might come from positive or
negative signals by professional groups or academic
organizations. General acceptance remains part of the Daubert
regime, and such signals might anchor a judge’s considerations of
whether an expert claim is sufficiently valid. Sociologists refer to
attempts by academic or professional fields to demarcate the line
between science and bad science or non-science as “boundary
work.” 178 The absence of a particular psychological theory from
the DSM-IV, 179 for example, or a negative appraisal about a
theory of medical causation from the American Medical Society,
174 Kevin M. Clermont & Theodore Eisenberg, Anti-Plaintiff Bias in the
Federal Appellate Courts, 84 JUDICATURE 128, 128–31 (2000) (suggesting that
plaintiffs appeal more often and get fewer reversals than defendants).
175 See Risinger, supra note 8, at 108, 110.
176 Saks & Faigman, supra note 11, at 120–21.
177 See Risinger, supra note 8, at 110.
178 Thomas F. Gieryn, Boundary-Work and the Demarcation of Science from
Non-science: Strains and Interests in Professional Ideologies of Scientists, 48
AM. SOC. REV. 781, 781 (1983); see also JASANOFF, supra note 23, at xiii-xvi
(discussing the "difficulties faced by judges and juries in recognizing 'good
science' and 'legitimate expertise' … "and the inconsistent measures courts use
to interpret and understand science).
179 AMERICAN PSYCHIATRIC ASSOCIATION, DIAGNOSTIC AND STATISTICAL MANUAL
OF MENTAL DISORDERS (4th ed., text rev. 2000).
4/23/2009 6:28 PM
80
ALB. L.J. SCI. & TECH.
[Vol. 19.1
might shape the way in which a judge evaluates a particular
expert. To cite a particular example, a number of toxic tort
plaintiffs have brought forward experts to claim that a company’s
product has led to them to suffer from Multiple Chemical
Sensitivity (MCS), a syndrome which one commentator calls a
“multi-symptomatic disorder affecting multiple organ systems
resulting from exposure to a multitude of chemicals at levels
To the
tolerated by the majority of the population.” 180
mainstream medical world, MCS is fatally lacking in objective
diagnostic critera injury. 181 Courts have been quick to cite major
medical organizations when they lambast the adherents of MCS
(usually called clinical ecologists) as promulgating unscientific
theories, sometimes explicitly relying on such statements to
exclude the plaintiff’s experts. 182
C. Institutional Goals: Weighing the Effects of Admissibility
Decisions on Courts
While reliability may be the primary consideration in
admissibility decisions, it is certainly not the only one. As
“repeat players,” judges should see their decisions as iterations in
a larger process, rather than a series of unconnected outcomes. 183
Decisions which may impact the operation of court institutions,
then, should garner special attention. The Supreme Court’s
recent decision in Blakely v. Washington, for example, which held
the Sixth Amendment to require all facts introduced at
sentencing to have been heard by a jury, was criticized by Justice
O’Connor as much on institutional grounds as constitutional
ones. 184 O’Connor worried that the decision would impose a
“constitutional tax” on jurisdictions that wished to use facts at
sentencing that had not been brought forward during the guilt
phase, requring them to employ costly bifurcated trials for crimes
Carl H. Johnson, When Science is Too Daunting: Multiple Chemical
Sensitivity, Federal Courts, and the Struggling Spirit of Daubert, 11 VILL.
ENVTL. L.J. 273, 276–78 (2000).
181 See William J. Waddell, The Science of Toxicology and its Relevance to
MCS, 18 REG. TOXICOLOGY & PHARMACOLOGY 13, 13 (1993).
182 See, e.g., Frank v. New York, 972 F. Supp. 130, 133–34 (N.D.N.Y. 1997);
Treadwell v. Dow-United Tech., 970 F. Supp. 974, 981 (M.D. Ala. 1997); Coffin
v. Orkin Exterminating Co., Inc., 20 F. Supp. 2d 107, 111 n.5 (D. Me. 1998).
183 Galanter, supra note 30, at 96–97, 122.
184 Blakely v. Washington, 542 U.S. 296, 298, 305 (2004); Id. at 322–26
(O'Connor, J., dissenting).
180
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
81
short of murder. 185 Similarly, some constitutional concerns, such
as standing or justiciability, are openly influenced by concerns
over the courts’ institutional capabilities.
In Daubert decisions, judges might consider how their
decisions to admit or exclude certain types of expert testimony
might harm the courts. Admitting certain types of plaintiff
expert testimony in the civil sphere might signal other litigants
to file similar suits, increasing caseload pressures or greatly
complicating previously routine decisions. Conversely, excluding
prosecutorial expert testimony might encourage criminal
defendants to escalate their own use of Daubert. The Crisp
majority, for example, argued against the “wholesale exclusion of
a long-accepted form of expert evidence,” a phrase that implies a
fear of institutional consequences for granting the Daubert
motion to exclude. 186 Other admissibility decisions have been
more direct in their consideration of institutional impact. 187 In
response to a Daubert motion to exclude fingerprint expertise on
grounds similar to those made by Crisp, the judge worried that:
This Court is unpersuaded by such reasoning, for it hearkens to an
imprudently stringent understanding of scientific objectivity.
Indeed, such reasoning could function to render numerous
categories of expert evidence, such as psychiatric or medical
forensic evidence which rest in some part upon that individual’s
skill and experience in analyzing data, unreliable. Daubert and its
progeny simply do not mandate such a conclusion. 188
A similarly situated judge in United States v. Hilerdiu Alteme
makes an even more direct reference to institutional concerns:
The difficulty with the argument advanced by the defenses is that
it proves too much. If Dr. Story’s criticism of the process of
fingerprint examination were sufficient to preclude the testimony
of other experts, large categories of scientific and technical
testimony would be inadmissible. At a minimum, it would be
necessary to eliminate the defense of insanity, since virtually all
psychiatric opinions are subjective, in whole or in part. 189
Note that the latter opinion essentially bases its application of
Daubert upon the institutional consequences of exclusion, rather
Id. at 318–320 (O’Connor, J. dissenting).
Crisp, 324 F.3d at 268.
187 See, e.g., United States v. Salim, 189 F. Supp. 2d 93, 101 (S.D.N.Y. 2002);
United States v. Hilerdieu Alteme, No. 99-8131-CR, at *7–8 (S.D. Fla. Apr. 7,
2000), available at http://onin.com/fp/altemedaubert.pdf.
188 Salim, 189 F. Supp. 2d at 101.
189 Hilerdieu Alteme, No. 99-8131-CR, at *7–8.
185
186
4/23/2009 6:28 PM
82
ALB. L.J. SCI. & TECH.
[Vol. 19.1
than addressing reliability per se. Given these examples,
consideration of how the admissibility decision might impact
caseload or bring new complexity to cases should be included in
the behavioral model. 190
Institutional variables should not be restricted to caseload. A
more nuanced way in which categories of expert testimony might
“threaten” courts is when particular expert theories challenge the
empirical assumptions on which courts operate. For example,
the act of witnessing—the presentation of evidence by laymen
regarding things they have seen—has been a pillar of our trial
system for centuries. 191 Unfortunately, psychologists who study
the ways in which individuals process, store, and recall memories
convincingly contend that eyewitness testimony is all too
fallible. 192 The Innocence Project, famous for freeing wrongfully
convicted defendants using DNA evidence, estimates on its
website that mistaken eyewitness identification played at least a
contributing role “in more than 75% of convictions overturned
through DNA testing.” 193 Unsurprisingly, when the case against
a criminal defendant is based wholly or in large part upon
eyewitness testimony, some defendants have attempted to
present expert testimony regarding these problems. 194
190 Such a variable is also needed to help tease out the true meaning of the
litigant identity variable. For example, do civil plaintiffs and criminal
defendants generally lose more from Daubert decisions because judges
perceived their experts to be of lower quality than other litigants, or because
deciding in their favor might substantially increase judicial workloads? Or,
perhaps, do these variables interact?
191 See, e.g., Andrew Elliot Carpenter, Chambers v. Mississippi: The Hearsay
Rule and Racial Evaluations of Credibility, 8 WASH. & LEE RACE & ETHNIC
ANCESTRY L.J. 15, 15 (2002); Catherine M. Polizzi, A New View into the Truth:
Impact of a Reliable Deception Detection Technology on the Legal System, 21
RUTGERS COMPUTER & TECH. L.J. 395, 409 (1995); Sonja R. West, The Story of
Me: The Underprotection of Autobiographical Speech, 84 WASH. U. L. REV. 905,
947 (2006).
192 See CUTLER & PENROD, supra note 28, at 7, 13–14; LOFTUS, supra note 28,
at 6–7.
193 The
Innocence
Project,
Understand
the
Causes:
Eyewitness
Misidentification,
http://www.innocenceproject.org/understand/EyewitnessMisidentification.php (last visited Mar. 16, 2009). This estimation is largely
consistent with earlier studies of the role mistaken eyewitness testimony played
in wrongful convictions. See C. Ronald Huff et al., Guilty Until Proved
Innocent: Wrongful Conviction and Public Policy, 32 CRIME & DELINQ. 518, 524
(1986); Arye Rattner, Convicted But Innocent: Wrongful Conviction and the
Criminal Justice System, 12 L. & HUM. BEHAV. 283, 289 (1988).
194 See CUTLER & PENROD, supra note 28, at 14–15; see also United States v.
Burton, No. 3:97-cr-154,1998 U.S. Dist. LEXIS 18730, at *11–13 (E.D. Tenn.
Oct. 8, 1998). These limits include how an individual’s ability to remember a
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
83
Under either a liberal or a more stringent admissibility
standard, expert testimony on eyewitnesses should be deemed
reliable. 195 The field’s array of claims has been tested and peerreviewed multiple times, and there even exists concrete data
regarding the degree of general acceptance of particular claims
among experts. 196 Despite these bona fides, however, eyewitness
experts have often met with a frosty reception from federal
judges during admissibility hearings. 197 Appeals regarding the
exclusion of eyewitness testimony, for example, have found
appellate judges offering vague or unusual arguments when
finding no abuse of discretion in the decision of the trial court. 198
The majority of published admissibility decisions regarding
eyewitness experts, however, avoid reliability concerns altogether
and focus on the institutional problems that eyewitness experts
may create. 199 Despite a 1984 amendment to Federal Rule of
face accurately declines rapidly after long periods of time, the weakened ability
to remember a face when a weapon is present, or the additional difficulties in
eyewitness recall that occur when the witness is of a different race than the
person he witnesses. See CUTLER & PENROD, supra note 28, at 70, 101–02, 104–
06.
195 Some critics, such as Dr. Ebbe Ebbesen, have argued that because many
of these claims have been tested in mock “crimes” on university campuses, they
lack sufficient external validity. See, e.g., Burton, 1998 U.S. Dist. LEXIS at
*35–39.
196 See Saul M. Kassin et al., On the 'General Acceptance' of Eyewitness
Testimony Research: A New Survey of the Experts, 56 AM. PSYCHOLOGIST 405,
405 (2001) (reporting an approximately 80% "agreement rate" for "30
eyewitness phenomena" among surveyed psychologists).
197 Robert J. Hallisey, Experts on Eyewitness Testimony in Court-A Short
Historical Perspective, 39 HOW. L.J. 237, 282–85 (1995).
198 Amador-Galvan, 1997 U.S. App. LEXIS at *4–5. The judge in AmadorGalvan upheld the exclusion of an eyewitness expert on the grounds that “the
testimony … ‘is based upon suspect, scientific analysis in that the alleged
scientific analysis is abstract, incomplete, and disregards many known
variables impacting upon the receipt of evidence and the decision making
process of juries.’” Id. What these other variables are, and why the evidence is
incomplete, is unclear from the opinion, though in fairness, an inadequate
Daubert proffer may have been to blame. The judge in United States v. Brien,
by contrast, upheld the district court’s exclusion by noting that the expert’s
testimony “does not concern a single long-established scientific principle such as
whether radar can measure speed.” Brien, 59 F.3d at 277. How such principles
are “established,” and by whom is unclear, but to rule that only “longestablished” principles are admissible as expert testimony is an interpretation
of Daubert that most commentators would reject.
199 See, e.g., United States v. Smithers, 212 F.3d 306, 314 (6th Cir. 2000);
United States v. Kime, 99 F.3d 870, 884 (8th Cir. 1996) (quoting Bachman v.
Leapley, 953 F.2d 440, 441 (8th Cir. 1992); United States v. Dorsey, 45 F.3d
809, 816 (4th Cir. 1995)); United States v. Larkin, 978 F.2d 964, 971 (7th Cir.
1992).
4/23/2009 6:28 PM
84
ALB. L.J. SCI. & TECH.
[Vol. 19.1
Evidence 704 that states, “testimony in the form of an opinion or
inference otherwise admissible is not objectionable because it
embraces an ultimate issue to be decided by the trier of fact,” 200
some judges openly worry that eyewitness experts usurp the role
of the jury by “deciding” witness credibility. 201 In his concurring
opinion in United States v. Hall, Judge Easterbrook suggests the
basis for this fear:
Consider, for example, the proposition—fundamental to any
system that relies on lay adjudicators—that jurors understand and
follow the instructions given by the court. It may be that jurors
don’t understand legalese, or if they understand the instructions
don’t follow them . . . Or consider cross-examination. Jurors may
believe that witnesses who hesitate, perspire, or fidget during
cross-examination are hiding the truth. This is the view that
underlies polygraph examinations, but without the precision of
measurement . . . Because trials rest on so many contestable
empirical propositions, including those about eyewitness
recollection, it always would be possible to offer expert evidence
along these and related lines. 202
“[A] trial about the process of trials,” he later contends, only
distracts the jury from the issue of the criminal charge at
hand. 203 In other words, the admission of expert eyewitness
testimony or polygraph experts into court might have long-term
deleterious effects on trials, 204 particularly if their admission
encourages defendants to use social science research to challenge
other foundations of the trial process. 205 The possibility that
courts will be particularly unreceptive to such testimony merits
some attention in empirical work on admissibility decisions.
D. Attitudinal Goals: Advancing Judicial Policy Preferences
Judicial goals also may include policy preferences, the desire to
reach particular outcomes or patterns of outcomes regardless of
the law. Judicial attitudes will be connected to the larger
political framework by the process of appointment; judges
FED. R. EVID. 704(a).
See Hall, 165 F.3d at 1119 (Easterbrook, J. concurring) (discussing the
effect on jurors of an expert’s testimony).
202 Id. at 1119.
203 Id.
204 Simmons, supra note 3, at 1046–47.
205 In one such (pre-Daubert) example noted by Easterbrook, Gacy v.
Welborn, 994 F.2d 305, 306 (7th Cir. 1993), the defendant appealed his death
sentence using social science research to attack the clarity of the court’s jury
instructions on mitigating factors.
200
201
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
85
appointed by conservative presidents may have different desired
legal outcomes than those appointed by liberal executives. While
judges are not legislators, they do retain considerable freedom to
shape decision outcomes, and often do so along a standard
liberal-conservative continuum. 206 As discussed in Section III,
such discretion varies among different judicial contexts—a
Supreme Court justice ruling on an issue of constitutional law
has far more discretion than a trial court judge implementing a
matter of routine procedure. Daubert decisions, I have noted, are
contexts where trial court judges are relatively unconstrained,
and there is little reason to expect that liberal and conservative
positions on torts and criminal prosecution will be absent from
admissibility decisions. A more conservative judge, for example,
might tend to side with business litigants in tort cases, while a
liberal judge might be more apt to question the government’s use
of expert testimony in criminal cases. 207
Some measure of judicial ideology must also be included as a
control for previously discussed factors. Do all judges view civil
plaintiffs as more likely to offer unreliable testimony, or are only
conservative judges more likely to do so? Would a liberal judge
be more willing to tempt caseload pressures by limiting or
excluding prosecutorial testimony? Such findings might show
that a construction of reliability was heavily mediated by prior
political positions, rather than a more widespread, institutional
suspicion of evidence offered by particular litigant classes. While
mapping standard liberal-conservative attributes onto federal
judges is not an easy task, political scientists who study judicial
behavior have developed reasonably valid measurements for
doing so. 208
206 See SEGAL & SPAETH, supra note 82, at 1, 65 (arguing that ideology greatly
influences justices' decisions).
207 See Buchman, supra note 7, at 682–88 (finding that ideology influences
judges decisions to admit evidence).
208 The best such measurements for federal appellate and district judges, in
my view, are Giles scores. Giles begins by relying on Poole’s DW-Nominate
scores (scores for each Senator and President, which vary from-1 to 1 on a
liberal-conservative scale). Federal judges receive the appointing President’s
ideology score, unless their home-state senators are the same party as the
President. In this case, the judge is either given either the home-state senator’s
Poole score, or if both home-state senators are of the president’s party, the
average of the two. This method has two advantages over simply coding federal
judges as appointed by Republicans or Democrats. First, it recognizes that
political party and attitudes are related but distinct; G. W. Bush appointees are
more likely to be conservative than Ford or Eisenhower appointees. Secondly, it
accounts for the potential moderating influence of home-state senators; Nixon,
4/23/2009 6:28 PM
86
ALB. L.J. SCI. & TECH.
[Vol. 19.1
E. External Variables
Finally, some variables in the admissibility process will be
independent of the deciding judge. I have already suggested that
despite their relative autonomy, district judges may constrain
themselves to on-point precedent, whether because of normative
concerns or to reduce workload. The proper normative role of
precedent in judicial decision-making, as well as its actual
empirical impact, is hotly debated by both scholars of judicial
behavior 209 and the justices of the Supreme Court. 210 Where
precedent is surely at its strongest, however, is when the facts of
that precedent and the facts in the current case essentially
mirror each other. 211 This might occur with some regularity in
Daubert cases, where individual litigants in different courts move
to exclude or introduce similar types of expert testimony. Given
the factors discussed above, there is likely to be some variation in
how different judges initially address a particular type of expert
testimony; as time goes on, however, a particular construction of
reliability may become dominant. Decisions that are unusually
thorough, thoughtful, or time-consuming should be more likely to
be cited as precedent, especially since a judge should be less
likely to duplicate an already existing Daubert analysis. The
greatest opportunity for the full play of reliability concerns,
institutional concerns, and policy preferences to shape a
particular construction of reliability may be in the first few
instances a particularly type of expert testimony faces
admissibility hearings. Afterwards, the costs of engaging in fullbore Daubert analyses may be too high, and variation will tend to
for example, may have appointed more liberal justices in Pennsylvania than in
Utah, because the Republican senators in Pennsylvania were more liberal than
he was. See generally Keith Poole, Description of NOMINATE Data, Jul. 13,
2004, http://voteview.com/page2a.htm (discussing how DW-Nominate data is
calculated); see also generally Michael W. Giles et al., Picking Federal Judges: A
Note on Policy and Partisan Selection Agendas, 54 POL. RES. Q. 623, 631 (2001)
(discussing how the authors relied on Poole’s scores to calculate their own data).
209 See, e.g., Brisbin, supra note 86, at 1005–06, 1008–09, 1014; Gillman,
supra note 96, at 479–80, 482; HANSFORD & SPRIGGS, supra note 164, at 18–23;
Jeffrey A. Segal & Harold J. Spaeth, The Influence of Stare Decisis on the Votes
of United States Supreme Court Justices, 40 AM. J. POL. SCI. 971, 971–73 (1996).
210 See Payne v. Tennessee, 501 U.S. 808, 827, 835 (1991) (quoting Burnet v.
Coronado Oil & Gas Co., 285 U.S. 393, 406 (1932); Smith v. Allwright, 321 U.S.
649, 665 (1944)).
211 See Richard L. Pacelle, Jr. et. al., Keepers of the Covenant or Platonic
Guardians? Decision Making on the U.S. Supreme Court, 35 AM. POL. RES. 694,
708–16 (2007) (finding that the Court most often follows precedent when cases
have similar facts).
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
87
be greater between different types of expert testimony rather
than within them.
Second, the model should consider the relative resources that
individual litigant classes bring to the table. A judge might
grant the prosecution’s motion to exclude a psychologist who
would testify on eyewitness testimony not because he assumes
the defendant is desperate or fears the effects of such witnesses
on the court system, but because the defense attorney has failed
to describe what that expert will do, why he satisfies Daubert, or
why his testimony will be reliable, relevant, and helpful. 212 The
success rate for civil defendants, for example, may in part be
explained by their superior resources in hiring high-quality
experts and retaining attorneys with considerable experience in
dealing with admissibility issues. An accused criminal, by
contrast, may rely on a public defender with little or no
experience in writing Daubert proffers, or have insufficient funds
to hire an expert with adequate credentials. 213 Litigant resources
should also be included in the model to better isolate the
possibility that judges use litigant identity as a proxy for
reliability, as discussed above, or simply dislike particular
litigants on policy grounds.
Next, society in general may hold assumptions about the
reliability of certain fields or types of testimony, or be
predisposed them to view these fields in a favorable or
unfavorable light.
Jennifer Mnookin, for example, has
th
convincingly argued that the legal system of the late 19 and
th
early 20 century had inculcated the belief that individuals
generally, and criminals specifically, possessed unique
physiological traits that permitted “comprehension of a deeper,
otherwise unattainable reality.” 214 As such, judges were more
212 My own sense of such briefs, though I have read only a few, is that they
vary wildly in quality; experience with Daubert motions and a substantive
understanding of the underlying science are the most likely causes of this
variation. I do not expect that these qualities will be distributed evenly
throughout all categories of attorneys.
213 Most jurisdictions do not extend Ake v. Oklahoma’s narrow holding
regarding the right to experts in death penalty cases to a more general right to
experts for indigents. See 470 U.S. 68, 83 (1985); Paul C. Giannelli, Ake v.
Oklahoma: The Right to Expert Assistance in a Post-Daubert, Post-DNA World,
89 CORNELL L. REV. 1305, 1310–11, 1365–69 (2004).
214 Jennifer L. Mnookin, Scripting Expertise: The History of Handwriting
Identification Evidence and the Judicial Construction of Reliability, 87 VA. L.
REV. 1723, 1808, passim (quoting Carlo Ginzburg, Clues: Roots of an Evidential
Paradigm, in CLUES, MYTHS, AND THE HISTORICAL METHOD 101, 105 (John &
4/23/2009 6:28 PM
88
ALB. L.J. SCI. & TECH.
[Vol. 19.1
inclined to accept various nascent forensic individuation
techniques as valid and reliable. Further exploration of current
social or intellectual paradigms might reveal similar tendencies
of courts and society alike to more readily view certain ideas as
scientific.
Finally, it is possible that circuits themselves serve as relevant
independent variables.
Circuits may differ in their
interpretation and application of Daubert, leading to differences
that do not draw the Supreme Court’s attention but may affect
outcomes nevertheless. 215 These rules and norms may constrain
judicial behavior in ways that Daubert itself does not do, and are
worth including in the model.
V. CONCLUSION: NOTES ON GETTING DAUBERT DATA
Building and testing such a model is beyond the scope of this
paper. I hope, however, I have made a convincing case that doing
so is useful and necessary. Empirical research on Daubert has
already begun: Risinger, for example, has systematically
examined differences in decision outcomes between civil and
criminal admissibility decisions, 216 while Buchman has found
that political ideology affects admissibility decisions in federal
civil cases at the trial court level. 217 The body of work is sparse,
however, and many questions remain. Do presumed reliability,
superior resources, or political attitudes lie behind the significant
advantages which accrue to criminal prosecutors and civil
defendants? When do institutional concerns affect Daubert
decisions, and how can one measure the effect of those concerns?
Are local constructions of reliability for new technologies heavily
path-dependent, making initial or particularly studious judicial
assessments of reliability the critical cases? I hope my article
has made the case that such questions are interesting and
necessary.
I conclude by noting one major obstacle to the empirical study
of Daubert.
A primary problem, for either qualitative or
quantitative studies, is ensuring that inferences are drawn from
Anne C. Tedeschi trans., 1989)).
215 See Ruth Saunders, The Circuit Courts' Application of Daubert v. Merrell
Dow Pharmaceuticals, Inc., 46 DRAKE L. REV. 407, 414, 422 (1997) (discussing
how the split in Circuit Courts' readings of Daubert lead to different treatment
of expert testimony).
216 Risinger, supra note 8, at 99, 102–104.
217 Buchman, supra note 7, at 688.
4/23/2009 6:28 PM
2009]
DAUBERT V. MERRELL DOW
89
representative data. However, obtaining a representative or
random sample of admissibility decisions from the universe of all
federal district court decisions will be quite difficult. Standard
database searches, such as Westlaw and Lexis, garner some
district court decisions, but because the available cases are a
small and possibly unrepresentative sample of the whole, relying
on them for generalizations risks making serious inferential
errors. If, for example, decisions to exclude expert testimony are
more likely to be published, one might overestimate the absolute
proportion of excluded decisions if only published cases are
Relying on appellate decisions for data mostly
studied. 218
overcomes this problem, as federal appellate decisions, at least
new ones, are reliably recorded. The gain is somewhat illusory,
though, as the benefit of easier sampling is offset by the skewed
picture of Daubert decisions appellate opinions create. A more
valid study of Daubert decisions should be conducted “on the
ground,” where the full range of issues are addressed, and the
judges do not employ Joiner in reviewing a previous
determination.
In other research agendas, scholars have relied upon case data
from the Administrative Office of the United States Courts to
create a representative data base. 219 The Federal Judiciary’s
PACER (Public Access to Court Electronic Records) system may
also grant researchers access to the closest approximation of the
actual universe of district court cases, at least for recent years. 220
However, these free or low-cost solutions suffer from a lack of
appropriate search capabilities. Though quite useful for studies
that test general propositions about federal decisions, these
databases do not permit the ready retrieval of Daubert decisions.
218 Peter Siegelman & John J. Donohue III, Studying the Iceberg from its Tip:
A Comparison of Published and Unpublished Employment Discrimination
Cases, 24 L. & SOC'Y REV. 1133, 1134–35, 1144–45 (1990). Another problem
with relying on judicial decisions is that Daubert’s primary impact may be on
persuading parties to settle, rather than on trials. I concede this probability; I
simply wish to study a different facet of the process. For a look at how Daubert
factors in the early stages of litigation, see Herbert M. Kritzer, Daubert in the
Law Office: Routinizing Procedural Change, 5 J. EMPIRICAL LEGAL STUD. 109,
110–11 (2008).
219 Clermont & Eisenberg, supra note 174, at 128–30; Kevin M. Clermont &
Theodore Eisenberg, Appeal from Jury or Judge Trial: Defendants' Advantage,
3 AM. L. & ECON. REV. 125, 127 (2001).
220 See PACER, http://www.pacer.psc.uscourts.gov (last visited Mar. 16,
2009). Most, but not all federal jurisdictions make their work product available
for PACER.
4/23/2009 6:28 PM
90
ALB. L.J. SCI. & TECH.
[Vol. 19.1
Emerging commercial solutions such as Daubert Tracker, 221
LexisNexis CourtLink, 222 or Westlaw Dockets Coverage 223 rely on
the same information as PACER but may provide the search
capabilities that make a systematic study of Daubert district
decisions feasible. 224 That said, such solutions may be too
expensive for the individual researcher to employ. Moreover,
broader searches such as these will require limiting the study to
a few districts or a narrow slice of time, barring prodigious
resources. Regardless of how the researcher ultimately selects
cases, she must understand the tradeoffs inherent in that choice
and adjust her interpretation of the data and her presentation of
the results accordingly.
None of this is to say that empirical research on Daubert must
be quantitative. Indeed, while quantitative research provides
more reliable generalizations for large numbers of cases and a
more precise estimate of uncertainty, qualitative explorations of
particular case areas may uncover new variables, explicate the
relationship between them, or give the researcher a better
understanding of the judge’s purported justification for the
decision. This is particularly true for the institutional concerns
listed in Part IV, which may not easily translate into
quantitative measures.
In either case, if scholars and
practitioners alike want a better understanding of admissibility
decisions, as well as a reasonable shot at accurately predicting
judicial reactions to new technology like fMRI scans, there is
much work to do.
221 See Daubert Tracker, http://www.dauberttracker.com (last visited Mar.
16, 2009).
222 See LexisNexis Courtlink, http://lexisnexis.com/courtlink/online/ (last
visited Mar. 16, 2009).
223 See
West, Westlaw Dockets Coverage, http://west.thomson.com/
productdetail/1-6676-5/RM166765/productdetail.aspx (last visited Mar. 16,
2009).
224 For more information on these options, see Linda Fields, Using Docket
and E-Filing Retrieval Systems for Legal Research, 33 COLO. LAW. 61, 61–62
(2004), available at http://www.aallnet.org/chapter/coall/Lrc/Lrc1104.pdf.