Professional Psychology: Research and Practice © 2018 American Psychological Association

2018, Vol. 49, No. 1, 22–30 0735-7028/18/$12.00 http://dx.doi.org/10.1037/pro0000172

How to Spot Hype in the Field of Psychotherapy: A 19-Item Checklist

Donald Meichenbaum Scott O. Lilienfeld

University of Waterloo and Melissa Institute for Violence Emory University and University of Melbourne
Prevention, Miami, Florida

How can consumers of psychotherapies, including practitioners, students, and clients, best appraise the
merits of therapies, especially those that are largely or entirely untested? We propose that clinicians,
patients, and other consumers should be especially skeptical of interventions that have been substantially
overhyped and overpromoted. To that end, we offer a provisional “Psychotherapy Hype Checklist,”
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

which consists of 19 warning signs suggesting that an intervention’s efficacy and effectiveness have been
This document is copyrighted by the American Psychological Association or one of its allied publishers.

substantially exaggerated. We hope that this checklist will foster a sense of healthy self-doubt in
practitioners and assist them to become more discerning consumers of the bewildering psychotherapy
marketplace. This checklist should also be useful in identifying the overhyping of well-established
treatments.

Public Significance Statement

Sizable pockets of the psychotherapy field are replete with exaggerated claims of efficacy and
effectiveness. We provide a 19-item checklist of warning signs designed to help practitioners and
others with the task of identifying psychotherapy hype. This provisional checklist should also help
to nurture critical thinking, healthy self-doubt, and intellectual humility in the selection and promo-
tion of psychotherapeutic interventions.

Keywords: psychotherapy, hype, fads, pseudoscience, science

The world of psychotherapy is bewildering. There are at least research evidence notwithstanding, scores of untested interven-
600 “brands” of psychotherapy, and this figure is almost certainly tions are extensively and enthusiastically promoted, often with
growing on a virtually monthly basis (Eisner, 2000; Lilienfeld, great fanfare and accompanied by expansive claims of efficacy and
Lynn, & Lohr, 2014). The substantial majority of these interven- effectiveness. Nevertheless, practitioners and graduate students in
tions have never been subjected to controlled clinical trials. Many training receive scant guidance for how to appraise such interven-
of these largely or entirely untested treatments may very well be tions in the absence of adequate research: Should they be partic-
effective, but some may be largely or entirely ineffective, and a ularly dubious of some of them, and, if so, which ones?
few may even be directly harmful (Lilienfeld, 2007). The lack of
The Dodo Bird Verdict
Some scholars might contend that consumers of the psychother-
apy literature need not be concerned by the challenges posed by
DONALD MEICHENBAUM received his PhD in psychology (clinical) from untested interventions. To support this view, they frequently in-
the University of Illinois, Champaign in 1966. He is Distinguished Pro-
voke the Dodo Bird verdict (Rosenzweig, 1936), which implies
fessor Emeritus at the University of Waterloo, Ontario, Canada, from
which he took early retirement 20 years ago. He is presently Research
that all psychological treatments work equally well (the name of
Director of the Melissa Institute for Violence Prevention in Miami, Florida this verdict derives from the Dodo Bird in Lewis Carroll’s Alice in
(www.melissainstitute.org). His major areas of professional interest are Wonderland, who declares after a race, “Everybody has won, and
ways to bolster resilience across the life span and ways to treat clients with all must have prizes”). Hence, this reasoning continues, we should
posttraumatic stress disorder and co-occurring psychiatric disorders. not be alarmed by the promotion and marketing of pseudoscientific
SCOTT O. LILIENFELD received his doctoral degree in psychology (clin- and otherwise questionable treatments, because these treatments
ical) from the University of Minnesota in 1990. He is Samuel Candler are likely to be as effective as well-established interventions. Nor
Dobbs Professor of Psychology at Emory University in Atlanta, where he should we be especially worried about the overhyping of unsub-
has been a faculty member since 1994. His research interests include stantiated treatments given that these treatments will probably turn
personality disorders, psychiatric classification, evidence-based practice,
out to work just about as well as others.
and the application of scientific thinking to psychology.
THE AUTHORS THANK Michael Hoyt and Scott Miller for their helpful
Comparative studies of psychotherapy impart a valuable lesson,
comments on a previous draft of this article. namely, that nonspecific factors (e.g., the therapeutic alliance)
CORRESPONDENCE CONCERNING THIS ARTICLE should be addressed to Scott account for sizable proportions of variance in treatment outcomes
O. Lilienfeld, Department of Psychology, Emory University, 36 Eagle (Wampold & Imel, 2015). In this respect, research on the Dodo
Row, Atlanta, Georgia 30322. E-mail: slilien@emory.edu Bird verdict reminds us not to advance expansive claims concern-

22
 PSYCHOTHERAPY HYPE 23

ing treatment specificity. There is also little doubt that for some By healthy self-doubt, we mean a propensity to engage in
psychological conditions, such as major depressive disorder, a thoughtful self-reflection regarding one’s biases and limitations, as
variety of different treatments are efficacious (Wampold et al., well as regarding one’s selection and interpretation of treatment
1997). and assessment techniques. Practitioners marked by healthy self-
Nevertheless, there are at least three reasons that findings con- doubt are not diffident. To the contrary, they are confident, but not
cerning approximate therapeutic equivalence should not be cause overconfident: Their confidence is properly calibrated to their
for complacency with respect to untested interventions. First, the level of knowledge and skills. Moreover, their confidence derives
Dodo Bird verdict as originally conceptualized referred only to a from an adequate appreciation of their shortcomings and of the
broad equivalence in efficacy across different schools of psycho- best means of compensating for them: “Forewarned is forearmed.”
therapy (e.g., behavioral, cognitive– behavioral, humanistic, psy- In the lingo of social cognition, therapists with a sense of healthy
chodynamic); it never implied that every intervention was equally self-doubt are characterized by a smaller bias blind spot (Pronin,
efficacious overall, let alone equally efficacious for every psycho- Lin, & Ross, 2002) compared with other therapists.
logical condition. Second, most data call into question the claim of Admittedly, virtually all of us are probably oblivious of our
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

exact equivalence of therapeutic effectiveness across all disorders biases to some degree, but we posit that therapists with a sense of
This document is copyrighted by the American Psychological Association or one of its allied publishers.

(Hunsley & Di Giulio, 2002; Lilienfeld, 2014; Tolin, 2014; but see healthy self-doubt are more cognizant of their propensity toward
Wampold et al., 2017, for an alternative view). To take merely one systematic error than are other therapists. In addition, we hypoth-
example, meta-analytic evidence suggests that critical incident esize that therapists with a sense of healthy self-doubt are inclined
stress (crisis) debriefing, a widely used prophylactic treatment for to rightly turn a doubtful eye to interventions that have been
trauma-exposed victims, is associated with negligible and perhaps substantially overhyped and overpromoted. As a consequence,
even negative effect sizes (Litz, Gray, Bryant, & Adler, 2002). The they may be less likely to fall prey to the seductive charm of
same conclusion holds for several popular “get-tough” interven- therapeutic fads and fallacies, as well as to psychological pseudo-
tions for antisocial adolescents, such as Scared Straight and boot science more broadly. Although excessive self-doubt may under-
camp treatments (Lilienfeld, 2007). Third, the conclusion of ap- mine the power of the expectancies that very likely drive some of
proximate equivalence of psychotherapies across all major condi- the success of psychotherapy (Frank & Frank, 1993), a modest
tions applies largely or entirely to “bona-fide” interventions, that dose of self-doubt, which cultivates a nondefensive acknowledg-
is, well-specified treatments grounded in a well-supported theoret- ment of the strengths and weaknesses of one’s preferred treatment
ical rationale and that have already been found to work reasonably approach, may foster confidence in patients.
well (Wampold et al., 1997). There are no compelling grounds for Much of what we have written in the preceding paragraph is
extending this verdict to psychological interventions that fall far conjectural. Nevertheless, correlational research raises the possi-
outside of the scientific mainstream. Furthermore, the onus of bility that psychotherapists’ self-doubt predicts better treatment
evidence falls on the proponents of novel interventions to demon- outcomes, at least among experienced therapists (Nissen-Lie,
strate that they are efficacious and effective, not on critics to Monsen, Ulleberg, & Rønnestad, 2013; Nissen-Lie et al., 2017; but
demonstrate otherwise. see Odyniec, Probst, Margraf, & Willutzki, 2017, for a replication
failure). In this research, endorsement of such items as “Lacking in
confidence that you might have a beneficial effect on a patient”
Healthy Self-Doubt
and “Unsure about how best to deal effectively with a patient” was
Rendering the evaluation of the psychotherapy outcome litera- tied to superior treatment outcomes, especially among therapists
ture more complicated, findings point to marked variability in with a positive self-concept. Aptly, the title of Nissen-Lie et al.’s
efficacy among psychotherapists themselves. At the risk of paint- (2017, p. 48) article was “Love yourself as a person, doubt yourself
ing with an overly broad brush, the most successful psychothera- as a therapist?” Similarly, in a small-sample (N ⫽ 16) study of
pists average 50% better outcomes and 50% fewer dropouts than psychodynamically oriented therapists, self-criticism significantly
do psychotherapists in general (Wampold, 2017). predicted superior patient outcomes. Perhaps counterintuitively,
We hypothesize that one largely unappreciated characteristic of more effective therapists rated their treatment sessions as having
successful psychotherapists is their penchant for maintaining a been less successful than did less effective therapists (Najavits &
skeptical attitude, both toward their own practice and toward Strupp, 1994), probably because they were more inclined to en-
psychological treatments in general. Although skepticism has ac- gage in self-scrutiny. It is unknown, however, whether therapist
quired a bad name in many quarters, it refers only to a propensity self-doubt is trainable, and if so, whether it is causally related to
to withhold judgment on assertions until adequate evidence is better client outcomes.
available (Shermer, 2002). In this respect, skepticism is merely a More broadly, overconfidence is linked to suboptimal decision-
broader term to describe what many scholars have referred to as making in medicine and allied health fields (Berner & Graber,
the scientific attitude (Sagan, 1995). In clinical psychology, such 2008; Croskerry & Norman, 2008), raising the possibility that
skepticism is well illustrated by Meehl’s (1973) classic chapter instilling a well-calibrated sense of self-confidence— one that bal-
“Why I Do Not Attend Case Conferences,” which in our view ances appropriate self-assurance with healthy self-doubt - will
should be required reading (and regular rereading!) for all mental enhance therapeutic outcomes. This goal is important for several
health professionals-in-training and current mental health profes- reasons, not the least of which is that many therapists, like most
sionals. We can also conceptualize skepticism in terms of several people in general (Kruger, 1999), appear to substantially overes-
closely allied concepts, such as epistemic (intellectual) humility timate their abilities (Miller, Hubble, Seidel, Chow, & Bargmann,
(Leary et al., 2017; Lilienfeld, Lynn, O’Donohue, & Latzman, 2014). For example, among 129 independent practice psychother-
2017) and the term we elect to emphasize here, healthy self-doubt. apists, the average clinician rated him- or herself at the 80th
 24 MEICHENBAUM AND LILIENFELD

percentile of all therapists in effectiveness and skills; 25% rated rafalo, 2006; Overholser, 2014; Thyer & Pignotti, 2015; Singer &
themselves at the 90th percentile. None rated themselves below Lalich, 1996; Witkowski, 2015).
average (Walfish, McAlister, O’Donnell, & Lambert, 2012). Fur- Several items on this checklist mirror commonly proposed in-
ther, data demonstrate that most therapists markedly overestimate the dicators (“warning signs”) of pseudoscience (e.g., Bunge, 1984;
percentage of their clients who are getting better and underestimate Hines, 2003; Lilienfeld et al., 2014). Nevertheless, our consider-
the percentage of their clients who are getting worse (Hannan et al., ably more extensive checklist goes well beyond previous lists of
2005). To minimize the risk of therapeutic error, psychotherapists pseudoscientific indicators in its focus on psychotherapeutic
need to steer clear of the hazards of overconfidence, both with claims in particular rather than scientific claims more broadly.
respect to their own therapeutic skills and with respect to their Moreover, our checklist applies not merely to the marketing of
enthusiasm for embracing unsubstantiated or overhyped interven- pseudoscientific or otherwise questionable interventions, but also
tions. to the overpromotion of claims concerning all psychological treat-
ments, even those underpinned by a solid evidentiary base (e.g.,
A Checklist of Psychotherapy Hype Warning Signs cognitive– behavioral therapy, acceptance and commitment ther-
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

apy, dialectical behavior therapy).

This document is copyrighted by the American Psychological Association or one of its allied publishers.

In the following section, we present an admittedly provisional

We offer this checklist primarily for mental health practitioners
checklist of 19 “Psychotherapy Hype Warning Signs” (see Table 1,
for a capsule summary). In the spirit of our own humility, we and practitioners-in-training who are attempting to navigate the
provide this list merely as a first approximation, and we welcome often-confusing maze of mental health treatments. This checklist is
suggestions and constructive criticisms from readers. We have intended to plant the seeds of healthy self-doubt in practitioners
drawn the items on this list from academic publications and and trainees, and to help to nurture in them a sense of humility in
presentations, trade books, claims advanced at continuing educa- treatment selection and delivery. In the long term, this checklist
tion workshops, inspection of printed and online advertisements of may also enhance treatment outcomes by dissuading practitioners
treatments, promotional emails, informal consultations with col- from embracing overhyped and pseudoscientific interventions, al-
leagues inside and outside of academia, and other sources. Some of though this conjecture awaits formal research corroboration. Ide-
these warning signs (especially 1–13) bear primarily on the pro- ally, nonclinician readers, especially (a) mental health consumers,
motion and marketing of treatments, whereas others (especially their friends, and loved ones, (b) psychology instructors, and (c)
14 –19) bear primarily on the quality of research ostensibly sup- science journalists should also find this checklist helpful as a field
porting them, although there is some overlap between these two guide to spotting overhyped and dubious interventions.
broad categories. Although we do not provide specific references We discourage readers from implementing this checklist in a
for each warning sign, we encourage interested readers to consult cookbook, DSM-style fashion. There is almost certainly no cate-
the following sources for examples of the overhyping of interven- gorical cut-off that demarcates largely pseudoscientific from
tions (Dawes, 1994; Eisner, 2000; Herbert et al., 2000; Jacobson, largely scientific therapies, so we are reluctant to suggest a specific
Foxx, & Mulick, 2005; Lilienfeld et al., 2014; Lilienfeld, Marshall, “number” of warning signs for a treatment to acquire “overhyped
Todd, & Shane, 2014; Mercer, 2015; Norcross, Koocher, & Ga- status.” Furthermore, even many well-established psychotherapies,

Table 1
Psychotherapy “Hype” Checklist

(1) Substantial exaggeration of claims of treatment effectiveness

(2) Conveying of powerful and unfounded expectancy effects
(3) Excessive appeal to authorities or “gurus”
(4) Heavy reliance on endorsements from presumed experts
(5) Use of a slick sales pitch and the use of extensive promotional efforts, including sale of paraphernalia
(6) Establishment of accreditation and credentialing procedures
(7) Tendency of treatment followers to insulate themselves from criticism
(8) Extensive use of “psychobabble”
(9) Extensive use of “neurobabble”
(10) Tendency of advocates to be defensive and dismissive of critics; selective reporting of contradictory
findings, such as the results of dismantling studies
(11) Extensive reliance on anecdotal evidence
(12) Claims that treatment “fits all”
(13) Claims that treatment is “evidence-based” on the basis of informal clinical observations
(14) Inadequate empirical support: Limited reports or omission of treatment outcome information, such as
patient selection criteria, drop-out rates, and follow-up data
(15) No proposed scientific basis for change mechanisms; proposed theoretical treatment mechanism lacks
“connectivity” with extant science
(16) Repeated use of implausible ad hoc maneuvers to explain away negative findings
(17) Comparison of treatment with weak and “intent to fail” treatment groups, or with only partial
(incomplete) treatment conditions
(18) Failure to consider or acknowledge potential allegiance and decline effects
(19) Failure to consider differential credibility checks across treatment groups; failure to consider the role of
non-specific factors, such as the therapeutic alliance
 PSYCHOTHERAPY HYPE 25

including some cognitive– behavioral and acceptance-based inter- mindfulness is markedly superior to extant interventions for de-
ventions, have at times been substantially overhyped (see Rosen & pression and other conditions (see Coyne, 2017, and Van Dam et
Lilienfeld, 2016). al., 2017, for discussions of the overpromotion of mindfulness
Nevertheless, it seems safe to conclude that the more warning techniques relative to the strength of the scientific evidence).
signs a given psychological treatment displays, the more alarm Nevertheless, meta-analyses offer at best mixed and largely neg-
bells should ring in therapists’ and other consumers’ minds. Such ative evidence for this claim (e.g., Khoury et al., 2013).
overpromotion can be misleading to both practitioners and pa- (2) Advocates inform patients that “If this treatment does not
tients, both of whom may come to expect dramatic or even help you, then nothing else will.” They strive to convey a powerful
miraculous cures. Patients in particular may become demoralized expectancy that reinforces treatment outcomes at the expense of
and disillusioned after receiving overhyped interventions that are sound scientific information that informs patients. This propensity
largely ineffective or substantially less effective than promised. may engender unrealistic hopes among patients. In addition, it may
Furthermore, because a presumably small minority of psycholog- undermine practitioners’ ethical obligations to describe interven-
ical interventions appear to be iatrogenic (Dimidjian & Hollon, tions accurately and provide patients with fully informed consent
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

2010; Lilienfeld, 2007), these warning signs may help to safeguard (see also Blease, Lilienfeld, & Kelley, 2016).
This document is copyrighted by the American Psychological Association or one of its allied publishers.

mental health consumers against psychological harm. (3) Advocates advance claims that one can— or needs to—learn
As Marcello Truzzi (1978) and later, Carl Sagan (1980), re- the technique from a “master,” a “leading expert,” “a renowned
minded us, extraordinary claims require extraordinary evidence. specialist,” and so on. In this regard, Meehl (1992) warned of the
Hence, proponents of interventions who advance remarkable guru omniscience fantasy, the temptation to believe that one glo-
claims of efficacy and effectiveness in the absence of equally rified expert can provide most or all of the answers to exceedingly
convincing data are opening themselves to justifiable criticism. complex psychological questions. As one example, Arthur Janov,
founder of primal therapy (colloquially called primal scream ther-
Psychotherapy Hype Warning Signs: apy), was widely viewed as a guru and virtual messiah by many of
A 19-Item Checklist his therapeutic acolytes, as well as by celebrities, such as ex-Beatle
John Lennon and his wife Yoko Ono (Fox, 2017). Nevertheless,
even recognized academicians can be elevated by their followers
Promotion and Marketing Red Flags to “guru” status. In some cases, the treatment developer may have
discovered the approach in a sudden personal epiphany, which
(1) Advocates of a therapeutic approach routinely advance
may contribute to the mystique of the approach.
greatly exaggerated and often unsubstantiated claims. They may
(4) Advocates rely heavily on the endorsements of presumed
assert that their treatment is “revolutionary,” “ground-breaking,”
leaders in the field, often without offering references to support
or that it is a “gold standard.” For example, the developer of
such endorsements. For example, many therapists in the trauma
psychodrama described his method as launching a third psychiatric
field cite Bessel van der Kolk as an advocate and endorser of their
revolution, the first two revolutions being initiated by Pinel and
approach. Although the endorsements of well-established experts
Freud (Moreno, 1964). More recently, the developer of Thought
can sometimes be informative for consumers, this practice should
Field Therapy (TFT), a prominent energy therapy, claimed to be able
to cure specific phobias in 5 min or less (Callahan, 1985), and several never substitute for systematic research evidence.
websites assert that hypnosis is 30 times more effective for weight (5) Advocates establish a coterie of trainers and perhaps an
loss compared with no treatment (e.g., see http://johnmongiovi.com/ international organization to promote the treatment. They often use
pages/weightloss). public media (TV, blogs, magazine articles) to oversell their treat-
Proponents may further assure clients and practitioners that their ment approach. In addition, they are “slick salespersons,” setting
“complete satisfaction” will be guaranteed. It is perhaps worth up clinics, training settings, workshops, and in-house conferences.
noting that there have been few or no changes in the overall effect Treatment proponents may also promote advanced, multilevel
sizes in psychotherapy outcome over the past three decades (Budd training, and sell paraphernalia and tapes that accompany their
& Hughes, 2009), suggesting that humility with respect to the treatment approaches. For example, some advocates of eye move-
prospect of treatment breakthroughs is in order. ment desensitization and reprocessing (EMDR) sell wands and
Other commonly used terms and phrases to beware of include “Megapulsars” to assist them with providing bilateral stimulation
“simple, but powerful treatment”; “breakthrough”; “remarkable (see https://www.colleenwest.com/for-therapists/what-equipment-
advance”; “paradigm shift”; “miracle cure”; “transformative,” do-i-use/). Proponents may require that trainees sign confidential-
“life-changing” or “uniquely effective” treatment; “dramatic” or ity statements that they will not share treatment protocols with
“remarkable” improvements; “unique and ultimate training”; “life- others.
changing benefits”; and “deep psychological healing.” One should (6) Advocates provide a certificate or diploma indicating that
also be wary of such terms as “proof” or “cure.” These two terms, one has taken the training and can now call oneself an X therapist.
although widely used, are suspect given that virtually all scientific They may offer to place clinicians’ names on a referral list of
claims are provisional and that few if any psychological treatments Certified X practitioners.
are associated with close to 100% symptom remission. (7) Followers of the treatment are insular. They create special-
In other cases, the hyped claims may be subtler, but arguably ized listservs and Facebook pages for advocates of the intervention
just as problematic. For example, some proponents of mindfulness to share their positive experiences and to criticize skeptics of their
interventions, a heterogeneous class of treatments that holds some perspectives, newsletters for treatment acolytes, and special inter-
promise for treating mood and anxiety disorders, have asserted that est groups at conventions.
 26 MEICHENBAUM AND LILIENFELD

(8) Advocates make frequent use of “psychobabble,” psycho- of those who have raised concerns regarding the efficacy or
logical verbiage that sounds scientific but in fact contains little or theoretical basis of their treatment approach. They may argue that
no content, to market their treatment approach (Rosen, 1977). “outsiders” are not qualified to evaluate their approach, because
Consumers should be especially dubious of advertisements or they have not administered the treatment themselves.
courses that make extensive and uncritical use of such terms as In addition, such advocates frequently neglect to discuss or even
“inner child,” “internal family systems,” “closure,” “codepen- acknowledge legitimate criticisms of their treatment approach.
dency,” “attachment wounds,” “sex addiction,” “holistic healing,” When they do mention criticisms, they frequently present them in
“synergy,” and so on, or that invoke concepts from quantum straw-person form that can be easily rebutted. Advocates fail to
mechanics to explain psychological change principles (see Hum- mention the results of dismantling studies that question the osten-
mler, 2017, for a critique of the use of quantum mechanisms to sible theoretical basis of their treatment approach, or the absence
explain everyday phenomena). of such studies.
(9) Advocates liberally use “neurobabble” and naïve biological (11) Advocates rely extensively on anecdotal evidence at the
reductionism (often accompanied by brightly colored functional expense of controlled outcome data (e.g., “Read these testimonials
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

imaging figures or diagrams of the brain) to promote their treat- from three people who claim that treatment X helped them”).
This document is copyrighted by the American Psychological Association or one of its allied publishers.

ment approach. Such neurobabble may involve the use of such Anecdotal evidence from multiple satisfied patients sometimes
terms as “neuro-networks,” “synaptic networks,” “hemispheric provides sufficient grounds for investigating a novel treatment in
synchronization,” “right brain attachment,” “sensorimotor integra- greater depth, but it rarely if ever provides sufficient grounds for
tion,” “memory integration,” “body memories,” “reptilian brain,” concluding that the treatment is effective (Davison & Lazarus,
or “neuroplasticity,” especially when they are detached from their 2007; Lilienfeld et al., 2014). Putting it somewhat differently,
original meanings. A further and largely unappreciated problem is anecdotal evidence can often be enormously helpful in the context
that many and arguably most “brain-based therapies” are not ready of discovery— hypothesis generation— but it is rarely informative
for application to patients given our present lack of understanding in the context of justification— hypothesis testing (see Reichen-
of how to bridge the vast gulf between the neural and psycholog- bach, 1938). As the old saw reminds us, “the plural of anecdote is
ical levels of analysis (Francken & Slors, 2017). In other cases, not evidence” (Ratzan, 2002, p. 169).
proponents may overinterpret weak or ambiguous brain imaging (12) The treatment claims are marked by an absence of clear
data in the service of making strong claims. For example, psychi- boundary conditions (Hines, 2003). Advocates may claim that the
atrist Daniel Amen (2001), who is a regular fixture on public TV, treatment approach can be applied successfully with patients who
has argued that the brains of a well-defined subset of individuals suffer from a wide variety of psychiatric and physical conditions,
with attention-deficit/hyperactivity disorder are marked by a “ring as well as across multiple age groups, without any supportive
of fire” characterized by pronounced overactivation in multiple clinical trial evidence. Some may even claim that their approach
brain regions. Nevertheless, the scientific evidence for the “ring of works for pets. Advocates may imply that their treatment “fits all”
fire” activation pattern is feeble (Hall, 2013). or “cures all” (“One size fits all”). For example, the developer of
Exacerbating this problem, proponents of brain-based treat- TFT insisted that this treatment is efficacious not only for adults
ments often resort to dubious neurological hypotheses to explain but for “horses, dogs, cats, infants, and very young children”
the apparent success of their approach. Such hypotheses are fre- (Callahan, 2001, p. 1255).
quently couched in neuroscientific terminology (see Schwartz, (13) Advocates maintain that their intervention is “evidence-
Lilienfeld, Meca, & Sauvigne, 2016). For example, consider the based,” “empirically supported,” or “empirically validated,” but
following passage from a scholar’s effort to offer a neurobiological they define “evidence” broadly and subjectively, referring largely
basis for the effectiveness of EMDR: or exclusively to their informal clinical observations (e.g., “I saw
it work with my clients, and that is my evidence”) or to informal
. . . the constant reorienting of attention demanded by the alternating, reports from clients rather than systematic sources of evidence
bilateral visual, auditory, or tactile stimuli of EMDR automatically obtained from well-controlled studies.
activates brain mechanisms which facilitate this reorienting. Activa-
tion of these systems simultaneously shifts the brain into a memory
processing mode similar to that of REM sleep. This REM-like state Research Evidence Red Flags
permits the integration of traumatic memories into associative cortical
(14) Advocates maintain that their treatment approach is
networks without interference from hippocampally mediated episodic
recall. . . . Once successfully integrated, corticohippocampal circuits
“evidence-based” because it has met a low criterion for evidence,
induce the weakening of the traumatic episodic memory and its such as two randomized controlled trials demonstrating significant
associated affect. (Stickgold, 2002, pp. 71–72) differences from no treatment. Nevertheless, advocates do not
discuss effect sizes, nor provide details about the exclusionary
Although this explanation may or may not be correct, it is criteria of the patients. They also do not report on drop-out rates or
premature in light of intense scientific controversy over whether follow-up data. Advocates may also advance vague claims without
the eye movements of EMDR are even relevant to its efficacy referencing them, such as “More than X number of studies have
(Devilly, Ono, & Lohr, 2014; Lee & Cuipers, 2014). In this regard, consistently demonstrated efficacy and superiority,” without citing
practitioners should bear in mind “Hyman’s maxim,” named after or critically evaluating them.
psychologist Ray Hyman: Before trying to explain how something (15) Advocates do not present a critical account of the scientific
works, one should first verify that it works (Hall, 2014). validity, or theoretical basis, for the effectiveness of the proposed
(10) Advocates are defensive and thin-skinned about their ap- treatment. They frequently offer little or no scientific basis for the
proach. They often question the motives, background, and training proposed change mechanisms for the treatment. Many energy
 PSYCHOTHERAPY HYPE 27

interventions, such as Emotional Freedom Techniques (EFT) and tend to drop off over time (Lehrer, 2010; Schooler, 2011). Initial
TFT, exemplify this problem. The intervention may “work” (in the positive effects for a given psychotherapy may sometimes be
weak sense of outperforming a no-treatment control group), but inflated because early studies were conducted by enthusiastic
this success probably has little or nothing to do with the proposed adherents of the intervention (“strike while the iron is hot”); these
treatment model. In particular, the intervention may perform better effect sizes may shrink when the intervention is later examined by
than no treatment or than weak control groups largely or entirely impartial investigators (see Johnsen & Friborg, 2015, for potential
because of nonspecific factors, such as placebo effects or the evidence of decline effects for cognitive– behavioral therapy; but
beneficial influence of therapeutic support (Frank & Frank, 1993). see Ljótsson, Hedman, Mattsson, & Andersson, 2017 and Walt-
In other cases, however, advocates do supply a theoretical man, Creed, & Beck, 2016; for alternative views). The same
rationale, but it conflicts overwhelmingly with known scientific principle holds in some domains of psychiatry, where an old adage
evidence. That is, the treatment rationale lacks “connectivity” with holds that one should “use the new drugs while they still work.”
well-established science (Stanovich, 2012). For example, propo- For example, the efficacy of antipsychotic medication appears to
nents of energy therapies claim that psychopathology is produced have decreased in recent decades (Leucht et al., 2009), although
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

by blockages in invisible, unmeasurable energy fields that violate some of this decline may also reflect more rigorous methodology
This document is copyrighted by the American Psychological Association or one of its allied publishers.

the known laws of physics. Proponents of hypnotic regression in more recent studies.
therapy claim that hypnosis can recover distinct and detailed (19) Advocates do not independently determine whether the
memories that date prior to the onset of infantile amnesia. Some treatment rationale offered to the alternative treatment and control
maintain that they can bring back memories from before birth, or groups was as credible as for the advocated treatment. This po-
even from past lives (Singer & Lalich, 1996). tential confound can lead to differences in expectancy effects
(16) Advocates routinely resort to multiple implausible “ad hoc across groups. Such advocates also do not acknowledge the po-
hypotheses” (after-the-fact excuses or loopholes) to explain away tential role of nonspecific treatment factors, such as the therapeutic
negative findings. This indiscriminate use of ad hoc explanations alliance, expectancy effects, and other placebo-related effects. For
for unsupportive findings renders the key treatment claims difficult example, their studies do not include measures of the ongoing
or impossible to falsify. As a consequence, the theoretical rationale quality of the therapeutic alliance, such as the Therapeutic Alliance
for the intervention becomes a “moving target.” For example, Scales, or the Quality of Relationship Measures, or session-by-
when advocates of EMDR were confronted with controlled re- session treatment-informed feedback (Prescott, Mareschalack, &
search evidence that their intervention did not outperform a fixed Miller, 2017).
eye movement condition, some responded that it did not discon-
firm the intervention’s theoretical rationale because the eyes
Conclusions
“wanted” to move (see Lilienfeld et al., 2014) As another example,
in response to a published study of EFT that demonstrated com- David Shakow (1969), one of the founders of modern clinical
parable effects on phobic fear from tapping on a doll as from psychology, wrote that “psychology is immodest” (p. 146). By
tapping on oneself (Waite & Holder, 2003), the creator of the this, he was referring largely to the habitual propensity of psychol-
method contended that because the fingertips themselves contain ogists to promise far more than they can deliver. Yet science,
energy meridians, this control condition was invalid (Craig, 2003). including clinical science, is fundamentally a prescription for
In other cases, advocates of a therapy may claim, without adequate intellectual humility, as it reminds us that we can all fool ourselves
justification, that unsuccessful replications of their positive treat- and be fooled by others (Lilienfeld et al., 2017; McFall, 1991;
ment results are attributable to failures to implement the treatment Tavris & Aronson, 2007). Such humility should extend to all
protocol with adequate fidelity (see DeBell & Jones, 1997 and domains of clinical practice, including the marketing, promotion,
Rosen, 1999, for critiques of such ad hoc reasoning by proponents evaluation, selection, and administration of treatments.
of EMDR). We expect this provisional 19-item checklist to evolve in re-
(17) Advocates compare their favored approach with “weak” sponse to constructive feedback. This checklist is itself a modest
comparison groups, that is, “intent- to-fail” conditions, which are step toward safeguarding practitioners and other consumers of
virtually guaranteed to yield null or weak effects (Westen & psychotherapy against exaggerated claims and ideally, toward
Bradley, 2005). They do not compare their treatment with “bona- instilling a sense of healthy self-doubt in clinicians. Although our
fide” conditions that are intended to be efficacious or effective (see checklist is designed primarily for professionals who are knowl-
Wampold et al., 1997). In other cases, advocates may compare edgeable regarding research design, many of the warning signs and
their proposed treatment with a diluted or weaker version of a red flags for identifying hype, especially the first 13, can be
comparative treatment. For an example, see Foa et al.’s (1991) profitably used by members of the general public and media
comparison of Prolonged Exposure versus Stress Inoculation resource outlets. More broadly, a number of the checklist items
Training (SIT), in which the third application phase of SIT was may also be helpful for spotting hype in (a) clinical assessment,
omitted (Meichenbaum, 2017). and (b) other domains of psychological science, such as social
(18) Advocates do not report on or acknowledge potential psychology, developmental psychology, and neuroscience (e.g.,
allegiance effects (see Luborsky et al., 1999), that is, positive see Ferguson, 2015, and Lilienfeld, Marshall, Aslinger, & Satel,
outcomes that depend on whether the primary investigator was 2017, for discussions).
favorably disposed to the intervention, or on who conducted the We encourage consumers of interventions, especially those that
outcome studies. Allegiance effects may help to account in part for are largely or entirely untested, to bear this checklist in mind when
another phenomenon, namely, the decline effect (“the law of initial appraising the scientific status of treatment claims. We also be-
results”), in which effect sizes from treatment studies in early trials lieve, however, that users will find this checklist helpful even
 28 MEICHENBAUM AND LILIENFELD

when evaluating claims concerning well-established therapies, in- Dawes, R. M. (1994). House of cards: Psychology and psychotherapy built
cluding those on lists of empirically supported treatments. Many on myth. New York, NY: Free Press.
proponents of such interventions have hardly been immune to DeBell, C., & Jones, R. D. (1997). As good as it seems? A review of
hype, and practitioners should not fall prey to the error of con- EMDR experimental research. Professional Psychology: Research and
Practice, 28, 153–163. http://dx.doi.org/10.1037/0735-7028.28.2.153
cluding that a treatment is a “gold standard” or is “highly effec-
Devilly, G. J., Ono, M., & Lohr, J. M. (2014). The use of meta-analytic
tive” merely because it is included on a list of empirically sup-
software to derive hypotheses for EMDR. Journal of Behavior Therapy
ported therapies. and Experimental Psychiatry, 45, 223–225. http://dx.doi.org/10.1016/j
We should be clear that we are not discouraging creativity. This .jbtep.2013.10.004
checklist does not preclude or diminish the importance of devel- Dimidjian, S., & Hollon, S. D. (2010). How would we know if psycho-
oping novel techniques, including those for which the evidence therapy were harmful? American Psychologist, 65, 21–33. http://dx.doi
base is presently minimal or nonexistent. Clinical innovation is an .org/10.1037/a0017299
essential driving force in the scientific progress of psychotherapy Eisner, D. A. (2000). The death of psychotherapy: From Freud to alien
(Lazarus & Davison, 1971; Simon & Ludman, 2009). Therapists abductions. New York, NY: Greenwood Publishing Group.
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

should not hesitate to invent or discuss new and largely untested Ferguson, C. J. (2015). “Everybody knows psychology is not a real
This document is copyrighted by the American Psychological Association or one of its allied publishers.

science”: Public perceptions of psychology and how we can improve our

interventions so long as they openly acknowledge the limitations
relationship with policymakers, the scientific community, and the gen-
of the evidence base (Blease et al., 2016).
eral public. American Psychologist, 70, 527–542. http://dx.doi.org/10
As noted earlier, an overriding objective of the checklist is to .1037/a0039405
cultivate an enduring habit of healthy self-doubt among clinicians. Foa, E. B., Rothbaum, B. O., Riggs, D. S., & Murdock, T. B. (1991). A
As Carl Sagan (1995) observed, we can think of science as a little comparison of exposure therapy, stress inoculation training and their
voice in our heads that incessantly intones, “You might be mis- combination for reducing PTSD in female assault victims. Journal of
taken. You’ve been wrong before” (p. 39). Once readers have Clinical and Consulting Psychology, 59, 715–723. http://dx.doi.org/10
perused the checklist, they may wish to ask themselves the fol- .1037/0022-006X.59.5.715
lowing question: Am I open to questioning and modifying any of Fox, M. (2017, October 2). Arthur Janov, 93, Dies; Psychologist Caught
my beliefs, claims, or clinical practices? World’s Attention With ‘Primal Scream. New York Times. https://www
.nytimes.com/2017/10/02/obituaries/arthur-janov-dead-developed-
primal-scream-therapy.html
References Francken, J. C., & Slors, M. (2017). Neuroscience and everyday life:
Facing the translation problem. Brain and Cognition. Advance online
Amen, D. (2001). Healing ADD. New York, NY: Penguin. publication. https://doi.org/10.1016/j.bandc.2017.09.004
Berner, E. S., & Graber, M. L. (2008). Overconfidence as a cause of Frank, J. D., & Frank, J. B. (1993). Persuasion and healing: A comparative
diagnostic error in medicine. The American Journal of Medicine, study of psychotherapy. Baltimore, MD: Johns Hopkins University
121(Suppl.), S2–S23. http://dx.doi.org/10.1016/j.amjmed.2008.01.001 Press.
Blease, C. R., Lilienfeld, S. O., & Kelley, J. M. (2016). Evidence-based Hall, H. (2013, March 19). Dr. Amen’s love affair with SPECT scans.
practice and psychological treatments: The imperatives of informed Science-Based Medicine. Retrieved from https://sciencebasedmedicine
consent. Frontiers in Psychology, 7, 1170. http://dx.doi.org/10.3389/ .org/dr-amens-love-affair-with-spect-scans/
fpsyg.2016.01170 Hall, H. (2014). On miracles. Skeptic, 19, 17–24.
Budd, R., & Hughes, I. (2009). The Dodo Bird Verdict—Controversial, Hannan, C., Lambert, M. J., Harmon, C., Nielsen, S. L., Smart, D. W.,
inevitable and important: A commentary on 30 years of meta-analyses. Shimokawa, K., & Sutton, S. W. (2005). A lab test and algorithms for
Clinical Psychology & Psychotherapy, 16, 510 –522. http://dx.doi.org/ identifying clients at risk for treatment failure. Journal of Clinical
10.1002/cpp.648 Psychology, 61, 155–163. http://dx.doi.org/10.1002/jclp.20108
Bunge, M. (1984). What is pseudoscience? The Skeptical Inquirer, 9, Herbert, J. D., Lilienfeld, S. O., Lohr, J. M., Montgomery, R. W. T.,
36 – 47. O’Donohue, W. T., Rosen, G. M., & Tolin, D. F. (2000). Science and
Callahan, R. J. (1985). Five Minute Phobia Cure: Dr. Callahan’s Treat- pseudoscience in the development of eye movement desensitization and
ment for Fears, Phobias, and Self Sabotage. Wilmington, DE: Enterprise reprocessing: Implications for clinical psychology. Clinical Psychology
Publishing. Review, 20, 945–971. http://dx.doi.org/10.1016/S0272-7358(99)00017-3
Callahan, R. J. (2001). Thought field therapy: Response to our critics and Hines, T. (2003). Pseudoscience and the paranormal. Amherst, NY: Pro-
a scrutiny of some old ideas of social science. Journal of Clinical metheus Books.
Psychology, 57, 1251–1260. http://dx.doi.org/10.1002/jclp.1093 Hümmler, H. G. (2017). Relativer quantumquark. Berlin, Germany:
Coyne, J. C. (2017). Mindfulness: A web-based mastercourse. https:// Springer. http://dx.doi.org/10.1007/978-3-662-53829-6
www.coyneoftherealm.com/blogs/news/mindfulness-a-web-based- Hunsley, J., & Di Giulio, G. (2002). Dodo bird, phoenix, or urban legend?
mastercourse The Scientific Review of Mental Health Practice, 1, 11–22.
Craig, G. (2003). Rebuttal: An open letter regarding the serious flaws in the Jacobson, J. W., Foxx, R. M., & Mulick, J. A. (Eds.). (2005). Controversial
Waite & Holder EFT study. EP Studies on anxiety. http://www Therapies for Developmental Disabilities: Fad, Fashion, and Science in
.energypsych.org/page/EP_for_Anxiety Professional Practice. London, England: Taylor & Francis.
Croskerry, P., & Norman, G. (2008). Overconfidence in clinical decision Johnsen, T. J., & Friborg, O. (2015). The effects of cognitive behavioral
making. The American Journal of Medicine, 121(Suppl.), S24 –S29. therapy as an anti-depressive treatment is falling: A meta-analysis.
http://dx.doi.org/10.1016/j.amjmed.2008.02.001 Psychological Bulletin, 141, 747–768. http://dx.doi.org/10.1037/
Davison, G. C., & Lazarus, A. A. (2007). Clinical case studies are impor- bul0000015
tant in the science and practice of psychotherapy. In S. O. Lilienfeld & Khoury, B., Lecomte, T., Fortin, G., Masse, M., Therien, P., Bouchard, V.,
W. T. O’Donohue (Eds.), The great ideas of clinical science: 17 prin- . . . Hofmann, S. G. (2013). Mindfulness-based therapy: A comprehen-
ciples that every mental health professional should understand (pp. sive meta-analysis. Clinical Psychology Review, 33, 763–771. http://dx
149 –162). New York, NY: Routledge. .doi.org/10.1016/j.cpr.2013.05.005
 PSYCHOTHERAPY HYPE 29

Kruger, J. (1999). Lake Wobegon be gone! The “below-average effect” and Meichenbaum, D. (2017). The evolution of cognitive behavior therapy: A
the egocentric nature of comparative ability judgments. Journal of personal and professional journey with Don Meichenbaum. New York,
Personality and Social Psychology, 77, 221–232. http://dx.doi.org/10 NY: Routledge.
.1037/0022-3514.77.2.221 Mercer, J. (2015). Controversial therapies. In R. L. Cautin & S. O.
Lazarus, A. A., & Davison, G. C. (1971). Clinical innovation in research Lilienfeld (Eds.), Encyclopedia of clinical psychology (pp. 755–763).
and practice. In A. E. Bergin & S. L. Garfield (Eds.), Handbook of New York, NY: Wiley. http://dx.doi.org/10.1002/9781118625392
psychotherapy and behavior change (pp. 196 –213). New York, NY: .wbecp515
Wiley. Miller, S. D., Hubble, M. A., Seidel, J. A., Chow, D., & Bargmann, S.
Leary, M. R., Diebels, K. J., Davisson, E. K., Jongman-Sereno, K. P., (2014, Summer). Feedback informed treatment (FIT): Improving clinical
Isherwood, J. C., Raimi, K. T., . . . Hoyle, R. H. (2017). Cognitive and practice one person at a time. In Practice, 50, 78 – 85.
interpersonal features of intellectual humility. Personality and Social Moreno, J. L. (1964). The third psychiatric revolution and the scope of
Psychology Bulletin, 43, 793– 813. http://dx.doi.org/10.1177/ psychodrama. Group Psychotherapy, 17, 149 –171.
0146167217697695 Najavits, L. M., & Strupp, H. H. (1994). Differences in the effectiveness of
Lee, C. W., & Cuijpers, P. (2014). What does the data say about the psychodynamic therapists: A process-outcome study. Psychotherapy:
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

importance of eye movement in EMDR? Journal of Behavior Therapy Theory, Research, Practice, Training, 31, 114 –123. http://dx.doi.org/10
This document is copyrighted by the American Psychological Association or one of its allied publishers.

and Experimental Psychiatry, 45, 226 –228. http://dx.doi.org/10.1016/j .1037/0033-3204.31.1.114

.jbtep.2013.10.002 Nissen-Lie, H. A., Monsen, J. T., Ulleberg, P., & Rønnestad, M. H. (2013).
Lehrer, J. (2010). The truth wears off: Is there something wrong with the Psychotherapists’ self-reports of their interpersonal functioning and dif-
scientific method? The New Yorker. Retrieved from https://www ficulties in practice as predictors of patient outcome. Psychotherapy
.newyorker/com/magazine/2010/12/13/the-truth-wears-off Research, 23, 86 –104. http://dx.doi.org/10.1080/10503307.2012
Leucht, S., Corves, C., Arbter, D., Engel, R. R., Li, C., & Davis, J. M. .735775
(2009). Second-generation versus first-generation antipsychotic drugs Nissen-Lie, H. A., Rønnestad, M. H., Høglend, P. A., Havik, O. E.,
for schizophrenia: A meta-analysis. Lancet, 373, 31– 41. http://dx.doi Solbakken, O. A., Stiles, T. C., & Monsen, J. T. (2017). Love yourself
.org/10.1016/S0140-6736(08)61764-X as a person, doubt yourself as a therapist? Clinical Psychology &
Lilienfeld, S. O. (2007). Psychological treatments that cause harm. Per-
Psychotherapy, 24, 48 – 60. http://dx.doi.org/10.1002/cpp.1977
spectives on Psychological Science, 2, 53–70. http://dx.doi.org/10.1111/
Norcross, J. C., Koocher, G. P., & Garofalo, A. (2006). Discredited
j.1745-6916.2007.00029.x
psychological treatments and tests: A Delphi poll. Professional Psychol-
Lilienfeld, S. O. (2014). The dodo bird verdict: Status in 2014. The
ogy: Research and Practice, 37, 515–522. http://dx.doi.org/10.1037/
Behavior Therapist, 37, 91–95.
0735-7028.37.5.515
Lilienfeld, S. O., Lynn, S. J., & Lohr, J. M. (Eds.). (2014). Science and
Odyniec, P., Probst, T., Margraf, J., & Willutzki, U. (2017). Psychother-
pseudoscience in clinical psychology. New York, NY: Guilford Press.
apist trainees’ professional self-doubt and negative personal reaction:
Lilienfeld, S. O., Lynn, S. J., O’Donohue, W. T., & Latzman, R. D. (2017).
Changes during cognitive behavioral therapy and association with pa-
Epistemic humility: An overarching educational philosophy for clinical
tient progress. Psychotherapy Research, 27, 1–16. http://dx.doi.org/10
psychology programs. Clinical Psychologist, 70, 6 –14.
.1080/10503307.2017.1315464
Lilienfeld, S. O., Marshall, J., Aslinger, E., & Satel, S. (2017). Neurohype:
Overholser, J. C. (2014). Chasing the latest fad: Confronting recent and
A field guide to exaggerated brain-based claims. In S. M. Johnson & S.
historical innovations in mental illness. Journal of Contemporary Psy-
Rommelfanger (Eds.), Routledge Handbook of Neuroethics (pp. 241–
chotherapy, 44, 53– 61. http://dx.doi.org/10.1007/s10879-013-9250-z
261). New York, NY: Routledge.
Prescott, H. D., Mareschalack, C. C., & Miller, S. D. (Eds.). (2017).
Lilienfeld, S. O., Marshall, J., Todd, J. T., & Shane, H. C. (2014). The
persistence of fad interventions in the face of negative scientific evi- Feedback informed treatment in clinical practice. Washington, DC:
dence: Facilitated communication for autism as a case example. American Psychological Association. http://dx.doi.org/10.1037/
Evidence-Based Communication Assessment and Intervention, 8, 62– 0000039-000
101. http://dx.doi.org/10.1080/17489539.2014.976332 Pronin, E., Lin, D. Y., & Ross, L. (2002). The bias blind spot: Perceptions
Litz, B. T., Gray, M. J., Bryant, R. A., & Adler, A. B. (2002). Early of bias in self versus others. Personality and Social Psychology Bulletin,
intervention for trauma: Current status and future directions. Clinical 28, 369 –381. http://dx.doi.org/10.1177/0146167202286008
Psychology: Science and Practice, 9, 112–134. http://dx.doi.org/10 Ratzan, S. C. (2002). The plural of anecdote is not evidence. Journal of
.1093/clipsy.9.2.112 Health Communication, 7, 169 –170. http://dx.doi.org/10.1080/
Ljótsson, B., Hedman, E., Mattsson, S., & Andersson, E. (2017). The 10810730290088058
effects of cognitive-behavioral therapy for depression are not falling: A Reichenbach, H. (1938). Experience and prediction: An analysis of the
re-analysis of Johnsen and Friborg (2015). Psychological Bulletin, 143, foundations and the structure of knowledge. Chicago, IL: University of
321–325. http://dx.doi.org/10.1037/bul0000055 Chicago Press.
Luborsky, L., Diguer, L., Seligman, D. A., Rosenthal, R., Krause, E. D., Rosen, G. M. (1999). Treatment fidelity and research on eye movement
Johnson, S., . . . Schweizer, E. (1999). The researcher’s own therapy desensitization and reprocessing (EMDR). Journal of Anxiety Disorders,
allegiances: A “wild card” in comparisons of treatment efficacy. Clinical 13, 173–184. http://dx.doi.org/10.1016/S0887-6185(98)00046-2
Psychology: Science and Practice, 6, 95–106. http://dx.doi.org/10.1093/ Rosen, G. M., & Lilienfeld, S. O. (2016). On the failure of psychology to
clipsy/6.1.95 advance self-help Acceptance and commitment therapy (ACT) as a case
McFall, R. (1991). A manifesto for the science of clinical psychology. example. Journal of Contemporary Psychotherapy, 46, 71–77. http://dx
Clinical Psychologist, 44, 75– 88. .doi.org/10.1007/s10879-015-9319-y
Meehl, P. E. (1973). Why I do not attend case conferences. Psychodiag- Rosen, R. D. (1977). Psychobabble: Fast talk and quick cure in the era of
nosis: Selected Papers (pp. 225–302). Minneapolis, MN: University of feeling. New York, NY: Atheneum.
Minnesota Press. Rosenzweig, S. (1936). Some implicit common factors in diverse methods
Meehl, P. E. (1992). Cliometric metatheory: The actuarial approach to of psychotherapy. American Journal of Orthopsychiatry, 6, 412– 415.
empirical, history-based philosophy of science. Psychological Reports, http://dx.doi.org/10.1111/j.1939-0025.1936.tb05248.x
71, 339 – 467. Sagan, C. (1980). Cosmos: “Encyclopaedia Galactica.” KCET Television.
 30 MEICHENBAUM AND LILIENFELD

Sagan, C. (1995). The demon-haunted world: Science as a candle in the Walfish, S., McAlister, B., O’Donnell, P., & Lambert, M. J. (2012). An
dark. New York, NY: Random House. investigation of self-assessment bias in mental health providers. Psycho-
Schooler, J. (2011). Unpublished results hide the decline effect. Nature, logical Reports, 110, 639 – 644. http://dx.doi.org/10.2466/02.07.17.PR0
470, 437. http://dx.doi.org/10.1038/470437a .110.2.639-644
Schwartz, S. J., Lilienfeld, S. O., Meca, A., & Sauvigné, K. C. (2016). The Waltman, S. H., Creed, T. A., & Beck, A. T. (2016). Are the effects of
role of neuroscience within psychology: A call for inclusiveness over cognitive behavior therapy for depression falling? Review and critique
exclusiveness. American Psychologist, 71, 52–70. http://dx.doi.org/10 of the evidence. Clinical Psychology: Science and Practice, 23, 113–
.1037/a0039678 122. http://dx.doi.org/10.1111/cpsp.12152
Shakow, D. (1969). Clinical psychology as science and profession: A Wampold, B. E. (2017). What should we practice? In T. Rousmaniere,
forty-year odyssey. New Brunswick, NJ: Transaction Publishers.
R. K. Goodyear, S. D. Miller, & B. E. Wampold (Eds.), The cycle of
Shermer, M. (2002). Why people believe weird things: Pseudoscience,
excellence (pp. 49 – 65). New York, NY: Wiley. http://dx.doi.org/10
superstition, and other confusions of our time. New York, NY: Holt.
.1002/9781119165590.ch3
Simon, G. E., & Ludman, E. J. (2009). It’s time for disruptive innovation
Wampold, B. E., Flückiger, C., Del Re, A. C., Yulish, N. E., Frost, N. D.,
in psychotherapy. The Lancet, 374, 594 –595. http://dx.doi.org/10.1016/
Pace, B. T., . . . Hilsenroth, M. J. (2017). In pursuit of truth: A critical
This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

S0140-6736(09)61415-X
Singer, M. T., & Lalich, J. (1996). “Crazy” Therapies: What Are They? Do examination of meta-analyses of cognitive behavior therapy. Psycho-
This document is copyrighted by the American Psychological Association or one of its allied publishers.

They Work? San Francisco, CA: Jossey-Bass. therapy Research, 27, 14 –32. http://dx.doi.org/10.1080/10503307.2016
Stanovich, K. E. (2012). How to Think Straight About Psychology. Boston, .1249433
MA: Pearson. Wampold, B. E., & Imel, Z. E. (2015). The great psychotherapy debate:
Stickgold, R. (2002). EMDR: A putative neurobiological mechanism of The evidence for what makes psychotherapy work. New York, NY:
action. Journal of Clinical Psychology, 58, 61–75. http://dx.doi.org/10 Routledge.
.1002/jclp.1129 Wampold, B. E., Mondin, G. W., Moody, M., Stich, F., Benson, K., &
Tavris, C., & Aronson, E. (2007). Mistakes Were Made (But Not by Me). Ahn, H. (1997). A meta-analysis of outcome studies comparing bona-
New York, NY: Harcourt. fide psychotherapies: Empirically, “All must have prizes”. Psychologi-
Thyer, B. A., & Pignotti, M. (2015). Science and Pseudoscience in Social cal Bulletin, 122, 203–215. http://dx.doi.org/10.1037/0033-2909.122.3
Work Practice. New York, NY: Springer. .203
Tolin, D. F. (2014). Beating a dead dodo bird: Looking at signal vs. noise Westen, D., & Bradley, R. (2005). Empirically supported complexity:
in cognitive-behavioral therapy for anxiety disorders. Clinical Psychol- Rethinking evidence-based practice in psychotherapy. Current Direc-
ogy: Science and Practice, 21, 351–362. http://dx.doi.org/10.1111/cpsp tions in Psychological Science, 14, 266 –271. http://dx.doi.org/10.1111/
.12080 j.0963-7214.2005.00378.x
Truzzi, M. (1978). On the extraordinary: An attempt at clarification. Zetetic
Witkowski, T. (2015). Psychology Gone Wrong: The Dark Sides of Science
Scholar, 1, 11–22.
and Therapy. Boca Raton, FL: BrownWalker.
Van Dam, N. T., van Vugt, M. K., Vago, D. R., Schmalzl, L., Saron, C. D.,
Olendzki, A., . . . Meyer, D. E. (2017). Mind the hype: A critical
evaluation and prescriptive agenda for research on mindfulness and
meditation. Perspectives on Psychological Science, 13, 1–26.
Waite, W. L., & Holder, M. D. (2003). Assessment of the emotional Received October 6, 2017
freedom technique. The Scientific Review of Mental Health Practice, 2, Revision received November 2, 2017
1–10. Accepted November 2, 2017 䡲