Manipulating Belief in Free Will and Its Downstream Consequences: A Meta-Analysis

1087527
review-article2022
PSRXXX10.1177/10888683221087527Personality and Social Psychology ReviewGenschow et al.
Review
Personality and Social Psychology Review
Manipulating Belief in Free Will and

1–31
© 2022 by the Society for Personality
and Social Psychology, Inc.
Its Downstream Consequences: A Article reuse guidelines:
sagepub.com/journals-permissions
Meta-Analysis DOI: 10.1177/10888683221087527

https://doi.org/10.1177/10888683221087527
pspr.sagepub.com
Oliver Genschow1* , Emiel Cracco2* , Jana Schneider1 ,

John Protzko3 , David Wisniewski2, Marcel Brass4,
and Jonathan W. Schooler5
Abstract
Ever since some scientists and popular media put forward the idea that free will is an illusion, the question has risen what
would happen if people stopped believing in free will. Psychological research has investigated this question by testing the
consequences of experimentally weakening people’s free will beliefs. The results of these investigations have been mixed,
with successful experiments and unsuccessful replications. This raises two fundamental questions: Can free will beliefs be
manipulated, and do such manipulations have downstream consequences? In a meta-analysis including 145 experiments (95
unpublished), we show that exposing individuals to anti–free will manipulations decreases belief in free will and increases
belief in determinism. However, we could not find evidence for downstream consequences. Our findings have important
theoretical implications for research on free will beliefs and contribute to the discussion of whether reducing people’s belief
in free will has societal consequences.
Keywords
free will, determinism, belief, meta-analysis, morality, cheating, social behavior, punishment
From morality to politics, public policy, intimate relation- neural activity several hundred milliseconds before partici-
ships, and punishing behavior—“most of what is distinctly pants reported being aware of this urge. These and similar
human about our life depends upon our viewing one another findings (Libet et al., 1983, 1993) have often been used as an
as autonomous persons, capable of free choice” (Harris, argument for the claim that free will does not exist.
2012, p. 1). Thus, unsurprisingly, most people believe that Despite criticisms of this argument (Brass et al., 2019;
they have free will (Baumeister et al., 2009; Nahmias et al., Saigle et al., 2018), anti–free will viewpoints have become in
2005). Whether free will actually exists, however, has been a vogue not only in academia (e.g., Greene & Cohen, 2004) but
long-standing philosophical debate (e.g., Dennett, 2015; Van also, and perhaps even more so, in popular media (e.g., Chivers,
Inwagen, 1983). This debate has reached an extremely high 2010; Griffin, 2016; Racine et al., 2017; Wolfe, 1997). This
level of sophistication outlining different theoretical posi- raises the fundamental question of whether reading anti–free
tions that span free will skepticism to complete libertarian- will viewpoints pushes people toward a deterministic world-
ism (for an overview, see Dennett, 2015). However, these view and lowers their belief in free will. Moreover, the question
philosophical arguments have rarely left academic circles arises what impact this has on society.
and therefore have had limited impact outside academia. Some philosophers have argued that undermining peo-
In the last decades, cognitive neuroscientists and psychol- ple’s belief in free will saps the basis for moral behavior and
ogists entered the debate by claiming humans’ perception of
free will is nothing more than an illusion (e.g., Crick, 1994; 1
University of Cologne, Germany
2
Harris, 2012; Wegner, 2002) and simply arises from uncon- Ghent University, Belgium
3
scious brain activity (Hallett, 2007; Libet et al., 1983; Soon Central Connecticut State University, New Britain, USA
4
Humboldt University of Berlin, Germany
et al., 2008). A seminal study supporting this view is the 5
University of California, Santa Barbara, USA
study by Libet and colleagues (1983), who measured neural *These authors shared first authorship.
activity while participants made voluntary finger move-
Corresponding Author:
ments. After each movement, participants indicated on a Oliver Genschow, Social Cognition Center Cologne, University of
clock the time at which they perceived their first urge to initi- Cologne, Richard-Strauss Str. 2, Cologne 50931, Germany.
ate a movement. Libet and colleagues found an increase in Email: oliver.genschow@uni-koeln.de
2 Personality and Social Psychology Review 00(0)
would therefore have catastrophic consequences (e.g., determinism, which captures people’s belief in “destiny” or
Smilansky, 2000, 2002). In contrast, other philosophers have “fate,” and scientific determinism, which measures belief in
argued that disbelief in free will might instead have positive genetic determinism (e.g., Paulhus & Carey, 2011) and reflects
effects, as it would cause people to abandon retribution- the belief that genes are the sole or predominant basis of per-
based morality and illusory beliefs in a just world (e.g., sonal characteristics (Keller, 2005).
Caruso, 2014; Greene & Cohen, 2004; Nadelhoffer, 2011; Irrespective of how determinism is defined, psychologists
Pereboom, 2006). Research in social and cognitive psychol- and philosophers debate whether free will and determinism
ogy as well as in neuroscience has empirically tested these are the endpoints of the same continuum or separate con-
theories by experimentally manipulating belief in free will structs. For instance, incompatibilists see free will and deter-
(for an overview, see Ewusi-Boisvert & Racine, 2018). These minism as mutually exclusive. Such a view suggests that the
studies provided evidence for the idea that free will beliefs more a person believes in free will, the less they believe in
indeed have societal consequences. There have also been a determinism (Rakos et al., 2008; Viney et al., 1982). In con-
number of failures to replicate some of these findings, how- trast, compatibilists see free will and determinism as inde-
ever. As a result, it remains unclear whether exposing people pendent constructs. Based on a compatibilistic view, if
to anti–free will viewpoints has behavioral and societal con- determinism were true, people could nevertheless be free.
sequences. In this article, we address this controversy. First, Previous research has shown that belief in free will is not
we review the literature on belief in free will, its experimen- correlated with belief in genetic determinism (Nadelhoffer
tal manipulations, and its downstream consequences. Then, et al., 2014) and that a compatibilistic view is more wide-
we test meta-analytically (a) whether beliefs related to free spread in the general public than philosophers and psycholo-
will can be manipulated and (b) whether these manipulations gists may have traditionally assumed (e.g., Monroe & Malle,
have downstream consequences. 2010a; Murray & Nahmias, 2014; Nadelhoffer et al., 2014;
Nahmias et al., 2006; Nichols, 2004, 2006; Nichols & Knobe,
2007; Rose & Nichols, 2013; Shepard & Reuter, 2012;
Belief in Free Will and Determinism Shepherd, 2012).
Philosophical definitions regarding the concept of free will
are rather diverse and complex (Carey & Paulhus, 2013). Research on Manipulating Belief
Because of this, researchers often look at laypeople’s con-
cepts of free will and to which degree laypeople believe in
in Free Will
these concepts (e.g., Nichols, 2006). These analyses indicate To test the consequences of (dis)believing in free will,
that laypeople’s free will beliefs are metacognitive judg- researchers have developed various approaches to experimen-
ments about the extent to which individuals intentionally tally manipulate people’s belief in free will. The first investi-
guide their thoughts and actions (Frith, 2012). Specifically, gations in this respect were carried out by Vohs and Schooler
belief in free will reflects the belief that people are respon- (2008). In one experiment, participants either read a passage
sible for their actions (Carey & Paulhus, 2013) because they from Francis Crick’s (1994) book The Astonishing Hypothesis,
can decide and control their own behavior (Paulhus & Carey, arguing against the plausibility of free will, or a passage from
2011). the same book that did not mention free will. In another exper-
Although conceptually related to internal locus of control iment, the authors followed a Velten-like technique (Velten,
(Rotter, 1966) and self-efficacy (Bandura, 1977), there are 1968), in which participants read and pondered either anti–
important differences between these constructs that set free free will or pro–free will statements. In both experiments, par-
will beliefs apart. Internal locus of control shares the empha- ticipants who were presented with anti–free will viewpoints
sis on internal causal attribution but represents a personality reported lower beliefs in free will and were more willing to
dimension, whereas belief in free will reflects an attitudinal cheat on a test than control participants.
orientation (Waldman et al., 1983). Self-efficacy differs from These findings inspired researchers all around the world
belief in free will in that it reflects metacognitive judgments to start investigating the consequences of experimentally
about one’s specific skill or ability (e.g., “Can I execute this reducing belief in free will by applying and adopting the
successfully?”). In contrast, belief in free will reflects a much manipulations introduced by Vohs and Schooler (2008). This
broader belief about choice and freedom (e.g., “Do I have a research suggests that weakening belief in free will increases
choice? Can I freely choose to do otherwise?”). antisocial behavior, such as prejudice (Zhao et al., 2014) or
Related to the belief in free will is the belief in determin- aggressiveness toward others (Baumeister et al., 2009), and
ism. Previous research has defined belief in determinism in decreases prosocial behavior, such as helping (Baumeister
different ways. On a general level, determinism is typically et al., 2009) or cooperation (Protzko et al., 2015). At the
interpreted as: given the past and the laws of nature, there is same time, however, exposing people to anti–free will view-
only one possible future at any moment in time (e.g., Van points can also lead to reduced retributive punishment
Inwagen, 1983). Some scholars differentiate between fatalistic (Shariff et al., 2014).
Genschow et al. 3
In addition, anti–free will manipulations have been found three explanations could account for the replication failures
to increase conformity (Alquist et al., 2013) and feelings of reported in the literature. First, it could be that the failed rep-
alienation (Seto & Hicks, 2016), and to decrease causal attri- lications are false negatives. That is, they were not able to
butions of other people’s actions (Genschow et al., 2017a), detect an effect that is actually real. Second, it might be that
the perceived meaningfulness of life (Crescioni et al., 2016; free will beliefs cannot be manipulated and that successful
Moynihan et al., 2019), perceived gratitude (MacKenzie studies in the literature are therefore false positives. Third, it
et al., 2014), counterfactual thinking (Alquist et al., 2015), could be that manipulations of belief in free will successfully
and risk-taking behavior (Schrag et al., 2016). Finally, a last affect free will beliefs, but that these manipulations are not
line of research suggests that experimentally reducing peo- causally related to other behaviors, and thus have no down-
ple’s belief in free will influences neurocognitive processes stream consequences. In this article, we investigate these
such as intentional action preparation (Rigoni et al., 2011), explanations by analyzing all available evidence in a
deliberate motor inhibition (Lynn et al., 2013; Rigoni et al., meta-analysis.
2012), and the processing of performance errors (Rigoni
et al., 2013, 2015).
In sum, there is a large body of research suggesting that
Previous Reviews and Meta-Analyses
manipulating belief in free will affects societally relevant To the best of our knowledge, this is the first comprehensive
behaviors such as cheating (Vohs & Schooler, 2008), retribu- meta-analysis on the effectiveness of manipulations related
tive punishment (Shariff & Vohs, 2014), and antisocial to free will beliefs. There are, however, two articles related to
behavior (Baumeister et al., 2009; Zhao et al., 2014), as well our analysis. First, Ewusi-Boisvert and Racine (2018) pub-
as basal neurocognitive mechanisms (Rigoni & Brass, 2014). lished a qualitative review of the literature on free will belief
Based on this research, it has been argued that one should be manipulations. Overall, the authors report a substantial
careful in how anti–free will and deterministic viewpoints amount of methodological diversity and a lack of replication
are presented to society because it may change the way peo- studies in the published literature. Moreover, the review sug-
ple interact with each other. For example, some scholars suggests that the studied samples are heavily constituted of
gested that encountering anti–free will viewpoints in the women, students, and younger participants and contain little
popular press may “move judges and jurors toward being information about the representation of ethnic minorities.
less punitive and less retributive in general” This review did not allow the accurate estimation of the
(p. 1569, Shariff et al., 2014) or “provide the ultimate excuse effect size of free will belief manipulations and its down-
to behave as one likes” (p. 54, Vohs & Schooler, 2008). stream consequences, however, as the researchers provided a
descriptive review of the literature and did not include
unpublished data.
Failed Replications Second, Genschow and colleagues (2017a) tested the
Despite the mounting evidence that manipulating belief in effectiveness of one specific manipulation (i.e., the Crick
free will influences behavior, a number of studies have manipulation) in a mini meta-analysis involving nine pub-
reported difficulties in replicating some key results (Crone & lished and unpublished experiments that were conducted in
Levy, 2019; Eben et al., 2020; Genschow et al., 2020; Giner- their own research group (N = 625 participants). Overall, the
Sorolla et al., 2016; Monroe et al., 2017; Nadelhoffer et al., authors found that participants who read the anti–free will
2020; Open Science Collaboration, 2015; Schooler et al., text had a significantly lower belief in free will than partici-
2014; Shariff & Vohs, 2014; Zwaan, 2014). For example, pants who read the control text. However, this analysis did
Monroe et al. (2017) found no effect of diminishing partici- not allow strong conclusions regarding the general effective-
pants’ belief in free will on moral behavior, judgments of ness of free will belief manipulations because only one
blame, or punishment decisions. Similarly, Nadelhoffer et al. manipulation type, a small number of experiments, one mea-
(2020) found that manipulating free will beliefs in a robust sure, and data from only one research team were investi-
way is more difficult than implied by previous work and that gated. In addition, this analysis did not explore potential
the proposed link with immoral behavior, such as cheating, reasons for why the manipulation sometimes works and
for instance, might be similarly tenuous (for similar findings, sometimes fails, nor did it investigate whether the manipula-
see Crone & Levy, 2019; Giner-Sorolla et al., 2016; Zwaan, tion has any downstream consequences.
2014).
Although these failed replications call into question the
The Present Meta-Analysis
societal relevance of belief in free will, it is not yet clear what
caused them. Before we can draw conclusions about the role The present meta-analysis aims to build on and considerably
of free will beliefs in society, it is imperative to understand extend previous work by including both published and
why some findings failed to replicate as well as which mech- unpublished evidence and addressing two main research
anisms underlie free will belief manipulations. In principle, questions:
Research Question 1: Can belief in free will be experi- measured in free will questionnaires, such as belief in dual-
mentally manipulated? ism, fatalistic determinism, and unpredictability.
Research Question 2: Does this have any downstream
consequences? Type of manipulation. Not only the scale but also the type of
manipulation differs across studies. Specifically, four types of
manipulations can be distinguished. The first method is to let
Research Question 1: Can Belief in Free
participants read a text—either a control text or a text arguing
Will Be Manipulated? against the plausibility of free will. Frequently used anti–free
In the first part of the meta-analysis, we investigate whether will texts include a passage of Francis Crick’s (1994) book The
it is possible to experimentally manipulate beliefs related to Astonishing Hypothesis (e.g., MacKenzie et al., 2014; Rigoni
free will and explore the conditions under which the manipu- et al., 2011; Shariff et al., 2014; Vohs & Schooler, 2008) or arti-
lations are effective. To this end, we investigated different cles featuring neuroscientific experiments (e.g., Harms et al.,
moderators. 2017; Protzko et al., 2016; Shariff et al., 2014), among others.
The second method is to present participants with several state-
Beliefs. The two most often used measures to test the effec- ments using a Velten-like technique (Velten, 1968). In this
tiveness of free will belief manipulations are belief in free method, participants are presented either with a variety of anti–
will scales and belief in determinism scales (Ewusi-Boisvert free will statements or with a variety of control statements (e.g.,
& Racine, 2018). It is often assumed (without giving explicit Rigoni et al., 2012; Stillman et al., 2010; Vohs & Schooler,
explanations) that anti–free will manipulations should not 2008). The third method uses a combination of text and state-
only decrease free will beliefs but should also increase beliefs ments (e.g., Seto & Hicks, 2016). Finally, the fourth method is
in determinism (for an overview, see Ewusi-Boisvert & to let participants watch videos related to (anti–)free will view-
Racine, 2018). This assumption, however, has never been points (e.g., Highhouse & Rada, 2015). To reliably manipulate
systematically tested. Therefore, in the present meta-analysis, belief in free will, it is important to know which of these manip-
we tested whether anti-free will manipulations decrease the ulations are most effective. Therefore, we directly compared
belief in free will, increase the belief in determinism, or both. them in the present meta-analysis.
Scales. Several validated scales have been developed to mea- Participant involvement. Whether the manipulation is effec-
sure belief in free will and determinism. The most frequently tive may, however, not only depend on the type of manipula-
used scales are the FWD scale (Rakos et al., 2008) and the tion but also on participants’ involvement in the manipulation.
FAD scale (Paulhus & Carey, 2011), which both stand for While some researchers merely presented participants with
Free Will and Determinism Scale, and the Free Will Inven- anti–free will viewpoints (e.g., Baumeister et al., 2009;
tory (FWI; Nadelhoffer et al., 2014). In addition, researchers Goodyear et al., 2016; Rigoni et al., 2012; Shariff et al.,
have sometimes also used self-made rating scales (e.g., Bau- 2014; Stillman et al., 2010), others engaged participants
meister et al., 2009; Moynihan et al., 2019). Less frequently more strongly by letting them, for instance, summarize or
used scales measuring beliefs related to free will and deter- rewrite the presented messages immediately after reading
minism include the Belief in Genetic Determinism scale them (Harms et al., 2017; Highhouse & Rada, 2015; Moyni-
(BGD; Keller, 2005), the Belief in Social Determinism scale han et al., 2019; Rigoni et al., 2011; Seto & Hicks, 2016;
(BSD; Rangel & Keller, 2011), and the WiF scale (Melcher, Vonasch et al., 2017). Yet other researchers told participants
2019). However, while a wide range of scales has been used, they had to attend the presented messages carefully so they
it remains unknown whether certain scales are more sensitive could summarize them at the end of the experiment (e.g.,
in picking up the effects of the manipulation than others. To Genschow et al., 2017a). An interesting question is how
address this question, the present meta-analysis tests whether deeply participants have to process the provided information
the effects of free will belief manipulations differ across for the manipulation to succeed. To answer this question, we
scales. We focus on the FWI and the FAD, as these are the investigated if the effectiveness of the manipulation depends
only two scales that tease apart belief in free will and belief on whether and when participants have to summarize or
in determinism. Moreover, they are also the two most com- rewrite the presented messages.
monly used scales and the only scales for which sufficient
data are available to reliably compare them with each other. Baseline condition. Another factor that may influence the
In addition to the sensitivity of different scales, little is manipulations’ effectiveness is the baseline condition. Past
known about whether free will belief manipulations specifi- research has shown that most people believe in free will
cally affect beliefs in free will and determinism or also influ- (Baumeister et al., 2009; Nahmias et al., 2005). As a result, a
ence other, related beliefs. Therefore, we investigated tacit assumption in the literature is that beliefs in free will
whether the influence of free will belief manipulations can only be diminished but not increased. Yet, some experi-
extends to other beliefs that are related to free will and are ments have nevertheless used not only anti–free will and
Genschow et al. 5
neutral messages but also pro–free will messages (e.g., Bau- or “attitudes.” However, this would involve pooling together
meister et al., 2009; Clark et al., 2017; Highhouse & Rada, studies with vastly different dependent variables under the
2015; Moynihan et al., 2019; Schrag et al., 2016; Seto & same denominator and would therefore run the risk of mak-
Hicks, 2016; Vohs & Schooler, 2008). This allowed us to test ing the meta-analysis uninterpretable, a problem that is well
the effectiveness of anti–free will messages both with respect known as the “apples and oranges” critique of meta-analysis
to control messages and with respect to pro–free will mes- (Carpenter, 2020). To deal with this issue, we proceeded in
sages and thereby allowed us to investigate whether belief in three steps.
free will can also be experimentally increased, in addition to In the first step, we ran a p-curve analysis across all
being decreased. dependent variables. While the aim of estimating a popula-
tion effect size makes a meta-analysis unsuited to evaluate
Measurement moment. A final moderator that may influence diverse sets of dependent variables, this is not the case for
the effectiveness of the manipulation is the moment at which p-curve. Rather than estimating a population effect size,
free will beliefs are measured. While some experiments mea- p-curve investigates whether a set of statistically significant
sured participants’ beliefs directly after the manipulation findings contains evidential value by testing whether the dis-
(Baumeister et al., 2009; Monroe et al., 2017; Seto & Hicks, tribution of p-values is consistent with the existence of a true
2016; Shariff et al., 2014; Vohs & Schooler, 2008; Vonasch effect (Simonsohn et al., 2014). Importantly, if confirmed,
et al., 2017), others measured them after participants per- this does not mean that all included studies show a true
formed a secondary task, often included to explore potential effect. Instead, it merely implies that at least one study does
downstream consequences (Clark et al., 2017; Genschow (Simonsohn et al., 2014). As such, p-curve can be applied to
et al., 2017a; Harms et al., 2017; Highhouse & Rada, 2015; diverse findings as long as they form a meaningful whole
Protzko et al., 2016; Rigoni et al., 2012; Schrag et al., 2016). (Simonsohn et al., 2015).
Measuring beliefs at the end of the experiment assumes the In a second step, we ran meta-analyses on internally
manipulation has a relatively long-lasting effect. This may, coherent sets of dependent variables. Upon reviewing the lit-
however, not necessarily be the case. For instance, it could erature, one clear set arose—namely, antisocial versus proso-
also be that free will belief manipulations do not fully change cial behavior (for an overview, see Table 1). Hence, we
people’s beliefs but rather prime an anti–free will mindset. pooled together the studies in this set and subjected them to
Such priming effects may not be as long-lasting, and measur- a meta-analysis testing whether manipulating belief in free
ing free will beliefs at the end of the experiment may there- will influences social behavior. However, pro- and antisocial
fore make it more difficult to confirm that the manipulation behavior is still a relatively broad and unspecific dependent
worked. To test this hypothesis, we investigated if the effect variable. Therefore, in a third and final step, we also ran
of the manipulation differs depending on whether beliefs are meta-analyses on three specific dependent variables that
measured immediately after the manipulation or at the end of have been used in at least five experiments: conformity, pun-
the experiment (i.e., after completing another task). ishment, and cheating.
Secondary moderator analyses. In addition to the above-men-

tioned moderators, we also considered the influence of par- Method
ticipant age, sex, the continent in which the study was
conducted (United States vs. Europe), test location (i.e., Search Strategy
online vs. laboratory), and sample type (students vs. non- The literature search for published articles was initiated in
students) as secondary moderators. July 2018 and includes studies published between January
2008—based on the publication date of the first study that
Research Question 2: Do Free Will Belief included an experimental belief in free will manipulation
(Vohs & Schooler, 2008)—and July 2019. To collect pub-
Manipulations Have Downstream Consequences? lished studies, we entered the following search terms in Web
In the second part of the meta-analysis, we investigate of Science, PubMed, and PsycINFO: (“Free will” AND
whether experimentally reducing free will beliefs influences “belie*”) OR (“Free will” AND “manipulat*”) OR (“Free
attitudes, behavior, and cognition. An interesting characteris- will” AND “experiment*”) OR (“Free will” AND “group”)
tic of the literature is that a large variety of dependent vari- OR (“Free will” AND “induc*”) OR (“Free will” AND
ables have been studied (for an overview, see Table 1). While “reduc*”) OR (“Free will” AND “threat*”) OR (“Free will”
this illustrates the breadth of the field, it also makes it diffi- AND “undermin*”) OR (“Free will” AND “weak*”) OR
cult to quantify the evidence for downstream consequences, (“Determinis*” AND “belie*”) OR (“Determinis*” AND
because for some dependent variables only one or a few “manipulat*”) OR (“Determinis*” AND “experiment*”) OR
experiments exist. In this respect, one solution could be to (“Determinis*” AND “group”) OR (“Determinis*” AND
group the variables into broad categories such as “behavior” “induc*”) OR (“Determinis*” AND “encourag*”) OR
Table 1. Overview of All Published Experiments Included in the P-Curve Analysis and/or the Meta-Analyses Assessing the Influence of
Free Will Belief Manipulations on Downstream Consequences.
Included in antisocial Included in p-curve

Experiment DV meta-analysis analysis
Vohs and Schooler (2008); Exp. 1 Cheating Yes Yes
Vohs and Schooler (2008); Exp. 2 Cheating Yes Yes
Baumeister et al. (2009); Exp. 1 Helping Yes Yes
Baumeister et al. (2009); Exp. 3 Aggression Yes Yes
Stillman and Baumeister (2010); Exp. 1 Learning No Yes
Stillman and Baumeister (2010); Exp. 2 Learning No Yes
Rangel & Keller (2011); Exp. 6 In-group preference No Yes
Rigoni et al. (2011); Exp. 1 Preconscious motor preparation No Yes
Alquist et al. (2013); Exp. 2 Conformity No Yes
Alquist et al. (2013); Exp. 3 Conformity No Yes
Evans (2013); Exp. 1 Willingness to help Yes Yes
Rigoni et al. (2013); Exp. 1 Action monitoring No Yes
MacKenzie et al. (2014); Exp. 2 Gratitude No Yes
Shariff et al. (2014); Exp. 2 Punishment No Yes
Shariff et al. (2014); Exp. 3 Punishment No Yes
Zhao et al. (2014); Exp. 2 Prejudice Yes Yes
Zhao et al. (2014); Exp. 3 Prejudice Yes Yes
Open Science Collaboration (2015); Exp. 1 Cheating Yes No
Alquist et al. (2015); Exp. 1 Counterfactual thinking No Yes
Alquist et al. (2015); Exp. 2 Counterfactual thinking and intention perception No Yes
Plaks and Robinson (2015); Exp. 4 Moral judgments No Yes
Rigoni et al. (2015); Exp. 1 Error detection No Yes
Crescioni et al. (2016); Exp. 3 Meaningfulness of life No Yes
Crescioni et al. (2016); Exp. 4 Meaningfulness of life No Yes
Goodyear et al. (2016); Exp. 1 Assigned moral responsibility No Yes
Protzko et al. (2016); Exp. 1 Cooperation Yes Yes
Schrag et al. (2016); Exp. 1 Risk behavior No Yes
Seto and Hicks (2016); Exp. 1 Self-alienation No Yes
Seto and Hicks (2016); Exp. 2 Reported authenticity No Yes
Caspar & Vuillaume (2017); Exp. 1 Immoral behavior Yes Yes
Clark et al. (2017); Exp. 4 Punitive distress No Yes
Genschow et al. (2017a); Exp. 3a Correspondence bias No Yes
Genschow et al. (2017a); Exp. 3b Correspondence bias No Yes
Hannikainen et al. (2017); Exp. 4 Preference for autonomy vs. Welfare No Yes
Hannikainen et al. (2017); Exp. 5 Approval of paternalistic policies No Yes
Monroe et al. (2017); Exp. 1 Punishment, cheating, moral judgments Yes No
Monroe et al. (2017); Exp. 3 Punishment and moral judgments No No
Vonasch et al. (2017); Exp. 6 Addiction and self-control No Yes
Ching & Xu (2018); Exp. 1 Prejudice Yes Yes
Moynihan et al. (2019); Exp. 1 Meaninglessness No Yes
Nadelhoffer et al. (2020); Exp. 3 Cheating Yes No
Nadelhoffer et al. (2020); Exp. 4 Cheating Yes No
Genschow & Vehlow (2019); Exp. 1 Victim blaming Yes Yes
Note. Nadelhoffer er al. (2019) and Genschow and Vehlow (2019) were included in the meta-analysis as unpublished studies but were published later
as Nadelhoffer et al. (2020) and Genschow and Vehlow (2021) after data analysis had already been completed. Experiments were only included in the
p-curve analysis if the effect of the manipulation on the DV was significant. All experiments that were not included in the p-curve analysis (included: “no”)
are experiments without a significant effect on the DV. Articles were included in the meta-analyses on antisocial behavior, conformity, punishment, and
cheating irrespective of whether the effect was significant or not. DV= dependent variable.
Genschow et al. 7
(“Determinis*” AND “increas*”) OR (“Determinis*” AND 4. Sufficient statistical information had to be available
“enhanc*”). to calculate the necessary effect sizes. The required
In addition to this Boolean search, we also looked for information had to be either reported in the paper or
studies that cited the Vohs and Schooler (2008) paper. obtained from the authors by email.
Furthermore, we included studies that were cited in the
recent review paper on belief in free will by Ewusi-Boisvert
Coding and Reliability
and Racine (2018). Third, unpublished data were collected
by sending requests to authors who had previously published Research Question 1: Can belief in free will be manipulated? The
articles using free will belief manipulations. Fourth, we sent eligible studies were first coded by the first author. To evalu-
around requests for unpublished data via different national ate the reliability of the coding, the third author was trained
and international mailing lists, including the mailing list of to use the coding manual and subsequently recoded 17 ran-
the German Psychology Association (DGPs), the mailing list domly selected published studies containing 27 experiments
of the Belgian Association for Psychological Science in total. To further identify errors, the initial coding was also
(BAPS), and the mailing list of the European Society for checked by the third author. Disagreements and inconsisten-
Cognitive and Affective Neuroscience (ESCAN). Finally, we cies that arose during coding were resolved by the first three
asked for unpublished data via Twitter and different open authors via discussion. Coding reliability was assessed with
fora of the Society of Personality and Social Psychology intraclass correlation coefficients (ICC) for continuous vari-
(SPSP). The search for unpublished data was terminated on ables and with kappa coefficients for categorical variables.
August 15, 2019. The reliability for continuous variables was high (all ICC ≥
.99). Similarly, the average reliability for categorical vari-
ables was high (κ = .96) and varied from .74 to 1.00. The
Screening Process coding manual is openly accessible at the Open Science
We screened titles and abstracts from 3,739 records obtained Framework (OSF; https://bit.ly/2L69prl). The following
from the literature search. In addition, 110 unpublished variables were coded:
experiments were included. Studies that were clearly not eli-
gible based on the criteria described below were excluded Publication status. We coded whether an experiment was
(i.e., n = 3570). The remaining studies were then evaluated published (m = 51) or unpublished (m = 95). An experi-
in more detail by screening the full-text articles. All together, ment was considered published when it appeared in an
279 full-text articles were assessed for eligibility. This proce- academic journal or book. Dissertations, preprints (if not
dure resulted in a database of 84 eligible studies containing published elsewhere), conference posters, and raw data
145 experiments (50 published and 95 unpublished). The were all considered unpublished. Experiments that were
number of excluded and included studies can be seen in the initially unpublished but then later published before the
Preferred Reporting Items for Systematic Reviews and Meta- meta-analysis was completed remained coded as unpub-
Analyses (PRISMA) flowchart in Figure 1. The inclusion lished experiments to account for potential differences
criteria were: between the original and published results (e.g., additional
data collection, different exclusion criteria, and different
1. The studied population had to involve healthy adults. analytical choices).
2. The study had to include an experimental manipula-
tion with the aim of reducing or increasing belief in Beliefs. We coded whether belief in free will or belief
free will or belief in determinism. in determinism was measured. Effect sizes obtained with
3. For the research question investigating the influence instruments measuring belief in free will and belief in deter-
of free will belief manipulations on free will beliefs, minism as opposite ends of a scale (e.g., the FWD) were not
but not for the research question investigating down- included. Effect sizes were coded as measuring belief in free
stream consequences, the study had to contain a will when they were obtained with the free will subscales
manipulation check administered after the manipula- of the FWI, the FAD-Plus, or the FAD. In addition, we also
tion—that is, a measure of belief in free will, deter- included self-made scales and individual items designed to
minism, or both. We focused on belief in free will and measure belief or disbelief in free will. The disbelief scales
belief in determinism because these are the two most were reverse coded.
commonly measured beliefs. Importantly, we only Effect sizes were coded as measuring belief in determin-
included measures that considered free will and ism when they were obtained with the following scales: belief
determinism as separate constructs and not measures in determinism of the FWI (Nadelhoffer et al., 2014), biologi-
that considered them as two opposite endpoints of the cal determinism of the WiF (Melcher, 2019), genetic deter-
same scale because the latter approach does not allow minism of the BGD (Keller, 2005), social determinism of the
us to distinguish between both beliefs.1 BSD (Rangel & Keller, 2011), and scientific determinism of
Figure 1. Flowchart of the screening process.

Note. The number of included or excluded studies (n) and experiments (m) are denoted. See text for additional detail.
the FAD-Plus or the FAD scale (Paulhus & Carey, 2011). In (Keller, 2005), the BSD (Rangel & Keller, 2011), or self-
addition, as for free will beliefs, we also included self-made made scales. The coding indicated that not enough experi-
scales and individual items designed to measure either belief ments (i.e., m < 3) had used the WiF, BGD, and BSD to
or disbelief in determinism. The disbelief items were reverse draw reliable conclusions. Furthermore, the self-made scales
coded. We did not include the FAD/FAD-Plus’ fatalistic did not form an internally coherent cluster. Therefore, we
determinism subscale because it does not capture determin- restricted the scales analysis to the FWI and FAD/FAD-Plus.
ism as construed by the other scales (i.e., the belief that future For the FWI and FAD/FAD-Plus, we also coded not only
events are completely determined by prior events) but rather the free will and determinism subscales but also all other
measures belief in “destiny” or “fate.” subscales to investigate whether the influence of free will
belief manipulations is specific to belief in free will and
Scales. To compare the sensitivity of different scales, we belief in determinism or also extends to other, related beliefs.
coded which of the following scales was used to measure More specifically, we coded the dualism subscale of the FWI
belief in free will and/or determinism: the FAD/FAD-Plus and the fatalistic determinism and unpredictability subscales
(Paulhus & Carey, 2011), the WiF (Melcher, 2019), the BGD of the FAD/FAD-Plus.
Genschow et al. 9
Type of manipulation. To compare different types of when at least one of the measured beliefs (i.e., free will or
manipulations, we coded whether a text, statements, a determinism) reached a p-value of p < .05. In a second step,
combination of text and statements, or a video was used to the second author checked the first author’s coding for errors.
manipulate belief in free will. Disagreements and inconsistencies that arose during coding
were resolved through discussion between both authors. The
Participant involvement. To test whether the effect of the pooling of dependent variables into the clusters described
manipulation depends on how deeply the participant had to below was done through mutual discussion.
process the stimulus material, we coded participant involve-
ment. For some studies, participants were asked to attend the P-curve. Rather than estimating a population effect size,
provided information carefully so that they could summarize p-curve investigates whether there is at least one effect in a
it at the end of the experiment. In other studies, participants set of significant effects that is not zero, by testing whether
had to process the provided information directly after the the p-values corresponding to these effects follow a flat or
manipulation by writing an essay or rewriting the provided skewed distribution (Simonsohn et al., 2014). The p-curve
messages. Yet in other studies, participants merely attended analysis assessing the overall evidence for downstream
the provided information and did not have to reproduce any consequences of free will manipulations included 35 stud-
information. For participant involvement, we coded whether ies with a total of 49 experiments (see OSF; https://bit.
participants (a) had to summarize or rewrite the provided ly/2L69prl). In all, 39 of these experiments were published
information directly after the manipulation, (b) had to sum- in academic journals or books. The remaining experiments
marize or rewrite the information at the end of the experi- were unpublished datasets. The p-values used in the p-curve
ment, or (c) simply had to attend the provided information, analysis were coded according to the guidelines laid out in
such as by merely reading a text or watching a video. Simonsohn et al. (2014). If a study reported multiple relevant
tests, we coded only the first test. We also report robustness
Baseline condition. To test whether beliefs in free will and analyses checking (a) whether the results changed when we
determinism can also be increased, in addition to decreased, used the second instead of the first test and (b) whether sig-
we coded whether the baseline condition consisted of neutral nificant effects still remained after removing the most sig-
messages or pro–free will messages. nificant test (Simonsohn et al., 2015). Note that for this latter
robustness analysis, we determined the p-value using a simu-
Measurement moment. To investigate how long-lasting lated null distribution to account for the fact that removing
the free will belief manipulations are, we coded whether the lowest p-value affects the distribution of the test statistic.
belief in free will or determinism was assessed directly after
the manipulation or after the primary dependent variable was Antisocial behavior meta-analysis. Upon reviewing the lit-
assessed. erature on downstream consequences, one clear set of vari-
ables arose—namely, antisocial versus prosocial behavior.
Secondary moderators. We also coded several secondary Hence, we pooled together the 15 studies (m = 21) in this
moderators, namely, age, sex, continent, test location, and set and subjected them to a meta-analysis testing whether
sample type. Age was coded as the mean age of participants manipulating belief in free will influences social behavior.
in the sample. Sex was coded as the proportion of female The prosocial dependent variables were helping and posi-
participants. Continent refers to whether the sample was col- tive attitudes toward minorities. The antisocial dependent
lected in the United States or on U.S.-based platforms such variables were immoral behavior, cheating, prejudice, mali-
as MTurk versus in Europe or on Europe-based platforms cious envy, and victim-blaming. Studies measuring prosocial
such as Prolific (not enough experiments were conducted on behavior or prosocial attitudes were reverse coded.
other continents to draw reliable conclusions). Test location
refers to whether the experiment was conducted in the labo- Conformity, punishment, and cheating meta-analyses. Three
ratory or online. Finally, sample type refers to whether the dependent variables were measured in at least five experi-
participants were students or panel participants (e.g., MTurk ments: cheating (n = 5; m = 8), conformity (n = 4, m = 6),
or Prolific). and punishment (n = 3; m = 9).
Research Question 2: Do free will belief manipulations have

Meta-Analytic Procedures
downstream consequences? In a first step, the first author
coded all studies measuring downstream consequences of Robust variance estimation. Analyses were performed in R
manipulating belief in free will. The following variables (v3.5.1) using the metafor (Viechtbauer, 2010) and robumeta
were coded: the dependent variable, whether the study (Fisher & Tipton, 2015) packages. The data were analyzed
included a manipulation check, whether the manipulation using random-effects models because we considered hetero-
check was significant, and whether the experiment was pub- geneity to be likely and because random-effects models con-
lished. A manipulation check was considered significant verge on fixed-effects models in the absence of heterogeneity
(Field & Gillett, 2010). A critical assumption of such ran- df is ≥ 4 (Tipton, 2015). As a result, we only report t tests
dom-effects models is that the included effect sizes are statis- with more than 4 df. In addition, for continuous moderators
tically independent (Lipsey & Wilson, 2001), and violating (e.g., age), we removed outlier values exceeding the weighted
this assumption is known to inflate the false-positive rate mean by more than 3 SDs because such outliers strongly
(Tanner-Smith et al., 2016). In the current meta-analysis, reduce the available degrees of freedom and hence statistical
however, many studies measured multiple dependent vari- power (Tanner-Smith et al., 2016). Weighted means were
ables or included multiple baseline conditions and therefore calculated by dividing a weight of 1 equally among the dif-
yielded more than one relevant effect size. As a result, the ferent values provided by the same experiment.
included effect sizes were not independent. To control for
this dependency, we decided to empirically estimate stan- Effect size estimation. All included studies used a between-
dard errors that do not assume independent effect sizes using subjects design to test the effect of the manipulation. Hedge’s
robust variance estimation (RVE; Fisher & Tipton, 2015; g was used as a measure of effect size. Effects were coded so
Hedges et al., 2010; Tanner-Smith et al., 2016; Tanner-Smith that positive effect sizes corresponded to stronger beliefs in
& Tipton, 2014). free will/determinism (research question 1) and larger values
In meta-analyses, effect sizes are typically weighted by on the outcome measures (research question 2) in the anti–
their standard error. Within the RVE framework, two ways to free will condition than in the reference condition. Hedge’s g
calculate such weights have been proposed: hierarchical was calculated by first calculating Cohen’s d and then cor-
effects weights and correlated effects weights. The former is recting these values using the escalc function of the metafor
most appropriate when dependency originates mainly from package in R (Viechtbauer, 2010). When means and standard
effect sizes being nested in larger units (e.g., research deviations were reported, Cohen’s d was calculated as
groups), whereas the latter is more appropriate when depen- follows:
dency originates mostly from a single study providing mul-
tiple effect sizes. While RVE provides asymptotically M anti − M ref (1)
unbiased standard errors regardless of how the weights are ds = .
calculated, the choice of weights does influence statistical ( nanti − 1) 2
SDanti (
+ nref − 1) 2
SDref
efficiency (Hedges et al., 2010). As we expected dependency nanti + nref − 2

to arise mainly from studies providing multiple effect sizes,
we used correlated effects weights (Fisher & Tipton, 2015).
where anti refers to the anti–free will condition and ref to the
These weights are inverse variance weights, where the
reference (i.e., baseline) condition. When standard errors
denominator of the weight assigned to each effect size in an
were provided instead of standard deviations, these were
experiment is determined by the average variance in the
transformed to standard deviations and Cohen’s d was calcu-
experiment multiplied by the number of effect sizes provided
lated using Equation 1. When insufficient information was
by that experiment. Importantly, this ensures that the total
reported to use Equation 1, we instead calculated Cohen’s d
weight of each experiment does not depend on the number of
from the t- or F-value as follows:
effect sizes it provides (e.g., Cracco et al., 2018). Of note, to
estimate variability, a parameter ρ representing the correla-
tion among the effect sizes has to be specified (Fisher & 1 1 1 1
ds = t + or F + . (2)
Tipton, 2015). This parameter is assumed to be the same for nanti nref nanti nref

all experiments and typically has a negligible influence on
the results. In the current meta-analysis, we used the default
When we could not retrieve sufficient information to calcu-
value of the robumeta package (i.e., ρ = 0.80) but also report
late an effect size, we contacted the corresponding author for
sensitivity analyses where we vary this value.
the necessary information. Nevertheless, despite our efforts,
Importantly, one problem with RVE is that it has an
we could not calculate the effect size for all coded studies.
inflated false-positive rate when the number of studies is
Specifically, for the first research question, we could not cal-
moderate to small or when skewed or unbalanced moderators
culate any of the effect sizes for four experiments from three
are included (Tipton, 2015; Tipton & Pustejovsky, 2015). As
studies and only part of the effect sizes for three experiments
a solution, small-sample (i.e., a limited number of studies in
from three studies. For the second research question, we
the meta-analysis) corrections have been proposed for both t
were not able to code five experiments from five studies.
tests (Tipton, 2015) and F tests (Tipton & Pustejovsky,
Effects for which we could not calculate an effect size were
2015). As it is difficult to know when these corrections
not included in the meta-analysis.
should be implemented and when they should not, it has been
recommended to implement them for all RVE analyses,
regardless of the meta-analytical sample size (Tipton, 2015). Outliers. Outliers were defined as effect sizes exceeding the
In the current meta-analysis, we follow this recommenda- weighted mean effect size, calculated as before, by more
tion. Importantly, corrections to the t test are only valid when than 3 SDs. For the first research question, this procedure
Genschow et al. 11
identified one outlier for free will beliefs (g = −2.89) and approaches, such as three-parameter selection models
one outlier for determinism beliefs (g = 4.00). These outliers (3PSM), it is easily implemented within the RVE framework
were replaced by the effect size 3 SDs above the weighted (Rodgers & Pustejovsky, 2020). Moreover, simulation stud-
mean effect size for free will beliefs (g = −1.63) and deter- ies have shown that PET-PEESE performs reasonably well
minism beliefs (g = 2.09). There were no outliers for the under most circumstances (Carter et al., 2019). In PET-
second research question. In addition to replacing outliers, PEESE, a bias-corrected effect is calculated by taking the
we also tested the influence of each individual effect size on intercept of two RVE meta-regressions that, respectively,
the average effect size using a leave-one-out cross-validation regress the effect sizes onto their standard error (PET) and
procedure. This indicated that leaving out individual effect variance (PEESE). PET and PEESE have complementary
sizes did not strongly influence the average effect size and strengths and weaknesses: Whereas PET tends to be overly
mostly influenced it similarly in both directions for both the conservative when there is a true effect, PEESE tends to be
first (free will beliefs: Δgmin = −0.01, Δgmax = 0.01; deter- overly liberal when there is no true effect (Stanley & Dou-
minism beliefs: Δgmin = −0.02, Δgmax = 0.01) and second couliagos, 2014). Therefore, we report both measures, as has
research questions (antisocial behavior: Δgmin = −0.03, Δgmax recently been recommended (Carter et al., 2019). Impor-
= 0.04; cheating: Δgmin = −0.11, Δgmax = 0.12; conformity: tantly, in some cases, PET and PEESE overcorrect effect
Δgmin = −0.04, Δgmax = 0.11; punishment: Δgmin = −0.05, sizes, reversing their sign. Because these overcorrections are
Δgmax = 0.05). not meaningful (Carter et al., 2019), we report the corrected
effect size as 0 when this occurs. Finally, in addition to using
Moderator correlations. To control for confounded modera- bias correction methods, we also compared published with
tors (Field & Gillett, 2010; Lipsey & Wilson, 2001), we com- unpublished studies and tested if the effects remained signifi-
puted the weighted associations, calculated as before, cant when only unpublished studies were considered.
between the different moderators included in the analysis of
the first research question with r for continuous–continuous Sensitivity analyses. To explore the robustness of our main
and continuous–dichotomous pairs, with multiple R for con- results, we conducted four sensitivity analyses investigating
tinuous–polytomous pairs, and with Cramér’s V for dichoto- how variations to our analysis procedure influenced (a) the
mous–dichotomous, dichotomous–polytomous, and effect size of the manipulation and (b) the estimated publica-
polytomous–polytomous pairs (Cracco et al., 2018). For all tion bias. First, we repeated the analyses while varying the ρ
these measures, 0 means no relationship and 1 means a per- parameter of the RVE models from 0 to 1 in steps of 0.2.
fect relationship. When two moderators correlated >.50, we Second, we repeated the analyses after computing effect
tested whether the moderator effects (if any) remained after sizes preferentially from the reported statistics instead of
controlling for the confounded moderator. When the contin- from the means and standard deviations, as sometimes stud-
gency table of two confounded moderators did not contain ies reported both. Although these two types of information
empty cells, we controlled for their confounding influences should in principle lead to the same effect size, this is not
by including both moderators in the same meta-regression always the case. Therefore, we decided to test how changing
model. Using sum coding, this allowed us to test the average the type of data used to calculate the effect sizes influenced
effect of each moderator across the levels of the other mod- the results. Third, we repeated the analyses without replacing
erator (for categorical moderators). When the contingency outlier values. Finally, we used a multilevel meta-analytical
table did contain empty cells, we controlled for confounding approach (Cheung, 2014) instead of RVE to test whether dif-
by restricting the analysis of Moderator A to the level of ferent methods of dealing with dependent effect sizes yielded
Moderator B where the levels of Moderator A were most bal- similar effect sizes. A sensitivity analysis was considered to
anced to maximize statistical power (Tipton, 2015; Tipton & produce results identical to the main analysis as long as the
Pustejovsky, 2015). difference between the corresponding estimates was ≤0.01.
Publication bias. For both research questions, we tested for

publication bias and other small-study effects using a hierar- Results
chical Egger’s regression test (Sterne & Egger, 2005). More Research Question 1: Can the Belief in Free
specifically, we ran an RVE meta-regression predicting effect
Will Be Manipulated?
sizes from their standard error (Rodgers & Pustejovsky,
2020). Publication bias leads to a positive relationship Effect on belief in free will and belief in determinism. We first
between effect sizes and standard errors because only large tested the overall effectiveness of the manipulation for both
effect sizes are statistically significant in studies with a large free will and determinism beliefs. This indicated that expos-
standard error (small N). To correct for potential publication ing participants to anti–free will manipulations decreased
bias, we used the PET-PEESE (precision-effect test and pre- belief in free will, g = −0.29, t(111) = −8.74, p < .001, 95%
cision-effect estimate with standard errors) approach (Stan- confidence interval [CI] = [−0.35, −0.22], m = 119, k = 148,
ley & Doucouliagos, 2014) because unlike other popular and increased belief in determinism, g = 0.17, t(46.5) = 4.33,
Figure 2. Funnel plots for belief in free will (left) and belief in determinism (right).
Note. Effect sizes from the same study are plotted in the same color with the same symbol.
p < .001, 95% CI = [0.09, 0.24], m = 53, k = 61. Next, we and belief in determinism, t(18.9) = 2.14, p = .046 (see Table
compared the influence of the manipulation on these two 3). Importantly, however, the effect size of both free will
beliefs by reverse-coding the determinism belief effect sizes beliefs, t(86.4) = −6.58, p < .001, and determinism beliefs,
and comparing them with the free will belief effect sizes. This t(34.4) = 2.75, p = .010, remained significant even when
revealed a borderline nonsignificant difference, with the only unpublished studies were included in the analysis. Simi-
manipulation having a slightly stronger effect on free will larly, correcting the free will belief effect size with PET (gz =
beliefs than on determinism beliefs, t(66) = −1.87, p = .066, −0.33, p < .001) or PEESE (gz = −0.31, p < .001) revealed
m = 124, k = 209. Finally, we looked at the I² coefficients, that the effect size was still significant. Correcting the deter-
which indicated that a substantial portion of the variance for minism belief effect size with PET made the effect disappear
both belief in free will (I² = 79%) and belief in determinism (gz = 0.06, p = .617), but correcting it with PEESE did reveal
(I² = 67%) was due to heterogeneity in the effect sizes.2 In a significant effect (gz = 0.13, p = .040).
sum, these analyses indicate that anti–free will manipulations Taken together, we find evidence for publication bias, but
reduce the belief in free will and increase the belief in also find that publication bias is unlikely to explain the effect
determinism. of the manipulation on free will beliefs. Although the effect
on determinism beliefs disappeared after applying PET cor-
Publication bias and small-study effects. A visual inspection of rection, it was still significant after applying PEESE correc-
the funnel plots (see Figure 2) revealed a largely symmetrical tion, and an analysis including only unpublished studies
effect size distribution for both free will and determinism likewise revealed a significant determinism effect. Given
beliefs. In line with this visual inspection, a hierarchical that PET is known to be overly conservative when there is a
Egger test found no relationship between the standard error true effect (Stanley & Doucouliagos, 2014), the combined
and the effect size for either free will, t(41.3) = 0.58, p = evidence suggests that not just belief in free will but also
.565, or determinism beliefs, t(19.4) = 0.76, p = .457. Impor- belief in determinism is affected by the manipulation, even
tantly, however, this does not necessarily mean that there is no after accounting for publication bias.
publication bias in the literature. Indeed, our analysis included
a large number of unpublished studies both for the free will Primary moderators
(m = 93, k = 116) and for the determinism analysis (m = 39, Scales. We first investigated whether the effectiveness of
k = 44), potentially masking the presence of publication bias. the manipulation depends on whether free will and determin-
In support of this hypothesis, an analysis including publica- ism were measured using the FWI scale or using the FAD
tion status as a moderator showed that the effect of the manip- scale. This revealed no significant difference between both
ulation was larger in published than in unpublished studies for scales for free-will beliefs, t(74.9) = 1.53, p = .130 (see
both belief in free will, t(36.8) = −3.43, p = .002 (see Table 2), Table 2), but a stronger effect when belief in determinism was
Genschow et al. 13
Table 2. Free Will Moderator Statistics.
Moderator g/β SE 95% CI m k F/t (df) p

Publication Status 3.43 (36.8) .002
Unpublished –0.23a 0.04 [–0.30, –0.16] 93 116
Published –0.51b 0.07 [–0.66, –0.36] 26 32
Questionnaire 1.53 (74.9) .130
FWI –0.23a 0.04 [–0.31, –0.14] 39 40
FAD –0.13a 0.04 [–0.22, –0.05] 46 59
Manipulation Type 6.70 (3, 26.3) .002
Text –0.18a 0.03 [–0.24, –0.12] 58 63
Statements –0.29a 0.06 [–0.41, –0.17] 35 54
Text and Statements –0.79b 0.14 [–1.12, –0.45] 8 9
Video –0.41a 0.11 [–0.64, –0.17] 12 12
Text Type 0.71 (13.2) .493
Crick –0.17a 0.03 [–0.23, –0.11] 48 51
Other –0.22a 0.06 [–0.35, –0.08] 10 12
Involvement 10.90 (2, 69.5) <.001
None –0.22a 0.06 [–0.34, –0.10] 33 43
Report Before –0.46b 0.06 [–0.59, –0.34] 44 53
Report After –0.14a 0.03 [–0.20, –0.09] 32 39
Measurement Moment 2.32 (92.3) .022
Before –0.38a 0.07 [–0.52, –0.25] 51 68
After –0.21b 0.03 [–0.27, –0.15] 49 58
Baseline 4.11 (106.7) <.001
Control –0.18a 0.03 [–0.23, –0.12] 72 80
Pro–free will –0.42b 0.06 [–0.53, –0.31] 64 67
Age 0.04 0.14 [–0.25, 0.32] 64 80 0.27 (28.2) .793
Sex (% Female) 0.09 0.16 [–0.24, 0.42] 66 82 0.53 (31.5) .598
Continent 2.50 (101.0) .014
Europe –0.19a 0.05 [–0.29, –0.09] 54 62
United States –0.36b 0.05 [–0.45, –0.27] 62 83
Test Location 0.45 (77.2) 0.655
Lab –0.29a 0.05 [–0.40, –0.18] 42 49
Online –0.26a 0.05 [–0.36, –0.15] 57 69
Sample 0.81 (62.6) 0.422
Students –0.30a 0.05 [–0.39, –0.21] 50 63
Panel –0.37a 0.08 [–0.53, –0.21] 29 32
Note. Different subscripts indicate p < .05. All statistical tests are unsigned. g/β = Hedge’s g/beta coefficient; 95% CI = 95% confidence interval;
m = number of experiments; k = number of effect sizes; FWI = Free Will Inventory; FAD = Free Will and Determinism scale.
measured with the FAD than with the FWI, t(21.8) = 2.29, Next, we investigated whether free will manipulations
p = .031 (see Table 3). However, scale correlated highly with also influenced the other subscales of the FWI and FAD. The
measurement moment for determinism beliefs (see Table 4). FWI analysis indicated that the manipulation influenced not
In particular, studies that used the FWI tended to measure only belief in free will, t(25.9) = −3.73, p < .001, and belief
beliefs at the end of the experiment rather than right after in determinism, t(26.9) = 2.27, p = .031, but also belief in
the manipulation, whereas studies using the FAD were more dualism, t(25.8) = −6.56, p < .001. More specifically, it
balanced with respect to measurement moment. To control indicated that belief in free will and belief in dualism
for this confound, we fitted a model including both scale decreased, whereas belief in determinism increased follow-
and measurement moment. This revealed that the difference ing anti–free will manipulations. A direct comparison of the
between the FWI and FAD for determinism beliefs was no size of these three effects revealed a significant main effect
longer significant, t(15.8) = 1.43, p = .172, when measure- of subscale, F(2, 31.6) = 4.43, p = .020, with significantly
ment moment was controlled, although the numerical pattern larger effect sizes for dualism than for determinism, t(27.7) =
went in the same direction as before. 2.87, p = .008, but no significant difference between free
Table 3. Determinism Moderator Statistics.

Publication Status 2.14 (18.9) .046
Unpublished 0.12a 0.05 [0.03, 0.21] 39 44
Published 0.30b 0.07 [0.15, 0.44] 14 17
Questionnaire 2.29 (21.8) .032
FWI 0.11a 0.04 [0.02, 0.19] 30 31
FAD 0.30b 0.07 [0.14, 0.47] 16 18
Manipulation Type 1.83 (2, 15.7) .193
Text 0.09a 0.04 [0.02, 0.17] 30 32
Statements 0.26a 0.08 [0.08, 0.43] 15 20
Video 0.26a 0.15 [–0.12, 0.64] 7 7
Involvement 0.61 (2, 29.3) .549
None 0.23a 0.07 [0.08, 0.37] 16 19
Report Before 0.18a 0.11 [–0.05, 0.41] 14 17
Report After 0.14a 0.04 [0.06, 0.22] 21 22
Measurement Moment 0.95 (33.2) .349
Before 0.22a 0.08 [0.05, 0.39] 18 22
After 0.13a 0.05 [0.03, 0.23] 28 29
Baseline 1.05 (25.0) .304
Control 0.13a 0.03 [0.06, 0.20] 40 43
Pro–free will 0.25a 0.10 [0.02, 0.47] 18 18
Age 0.03 0.13 [–0.25, 0.30] 42 49 0.21 (19.1) .838
Sex (% Female) –0.03 0.19 [–0.43, 0.38] 45 52 0.14 (17.3) .893
Continent 0.99 (45.2) .329
Europe 0.13a 0.06 [0.01, 0.25] 30 35
United States 0.21a 0.05 [0.10, 0.31] 23 26
Test Location 0.55 (42.6) .586
Lab 0.19a 0.06 [0.07, 0.32] 27 33
Online 0.15a 0.05 [0.04, 0.26] 22 23
Sample 0.62 (28.7) .543
Students 0.19a 0.06 [0.07, 0.31] 31 37
Panel 0.15a 0.05 [0.03, 0.26] 14 15
Note. Different subscripts indicate p < .05. All statistical tests are unsigned. g/β = Hedge’s g/beta coefficient; 95% CI = 95% confidence interval; m =
number of experiments; k = number of effect sizes; FWI = Free Will Inventory; FAD = Free Will and Determinism scale.
Table 4. Weighted Correlations Among Moderator Variables.
Moderator Published Scale Type Involvement Moment Baseline Age Sex Continent Location Sample
Published 1.00 .43 .32 .25 .06 .00 .16 .16 .25 .08 .03
Scale .28* 1.00 .40 .31 .52* .15 .27 .38 .29 .28 .26
Type .20 .36*** 1.00 .59*** .55** .75*** .23 .27 .51*** .10 .27
Involvement .19 .37*** .42*** 1.00 .39* .69*** .14 .25 .50** .21 .36*
Moment .08 .42*** .42*** .43*** 1.00 .01 .11 .04 .38** .06 .05
Baseline .13 .25 .66*** .59*** .18 1.00 .05 .20 .24 .07 .09
Age .04 .30 .34* .10 .17 .01 1.00 .42** .25 .84*** .75***
Sex .09 .18 .39** .23 .10 .08 .55*** 1.00 .36** .43** .66***
Continent .26** .21 .29* .26* .02 .11 .24* .43*** 1.00 .42** .69***
Location .02 .19 .18 .18 .08 .07 .82*** .44*** .42*** 1.00 .86***
Sample .15 .23 .33* .45*** .05 .13 .81*** .63*** .50*** .72*** 1.00
Note. The lower half of the matrix shows correlations for free will beliefs and the upper half of the matrix shows correlations for determinism beliefs.
Relations between continuous–continuous variables and between continuous–dichotomous variables were assessed with Pearson’s r. Relations between
continuous–polytomous variables were assessed with multiple R. Relations between dichotomous–dichotomous, dichotomous–polytomous, and
polytomous–polytomous variables were assessed with Cramér’s V. Correlations > .50 are indicated in bold font. These correlations were controlled for
whenever one of the two involved moderators produced a significant effect.
*p < .05. **p < .01. ***p < .001.
Genschow et al. 15
Table 5. FWI and FAD Statistics.

FWI 4.43 (2, 31.6) .020
Free Will 0.25a,b 0.07 [0.11, 0.38] 39 40
Determinism 0.11a 0.05 [0.01, 0.21] 30 31
Dualism 0.29b 0.05 [0.20, 0.39] 28 29
FAD 6.52 (3, 9.8) .011
Free Will 0.12a 0.04 [0.04, 0.21] 46 59
Scientific Determinism 0.40b 0.08 [0.22, 0.57] 16 18
Fatalistic Determinism 0.13a 0.04 [0.02, 0.23] 13 14
Unpredictability –0.03c 0.05 [–0.13, 0.07] 12 13
Note. Different subscripts indicate p < .05. Effect sizes are coded such that larger effects in the expected direction lead to more positive effect sizes. All
statistical tests are unsigned. g/β = Hedge’s g/beta coefficient; 95% CI = 95% confidence interval; m = number of experiments; k = number of effect
sizes; FWI = Free Will Inventory; FAD = Free Will and Determinism scale.
will and determinism, t(34.4) = 1.86, p = .072, or between p = .050. None of the other conditions differed significantly
free will and dualism, t(33.9) = 0.57, p = .571 (see Table 5). from each other (all ps ≥ .057; see Table 2). Next, we also
The FAD analysis likewise showed that the manipulation compared the effectiveness of the Crick text, which has been
changed not only belief in free will, t(33) = −2.88, p = .007, used most often in the literature, with the effectiveness of
and belief in scientific determinism, t(7.8) = 5.26, p < .001, other texts that have been used. This revealed no difference
but also belief in fatalistic determinism, t(5.3) = 3.10, p = for free will beliefs, t(13.2) = −0.71, p = .493 (see Table 2).
.025. More specifically, belief in free will decreased and The determinism analysis did not have enough degrees of
belief in scientific and fatalistic determinism increased fol- freedom (i.e., df = 2.15) to interpret.
lowing anti–free will manipulations. No effect was found on Importantly, however, manipulation type correlated
belief in unpredictability, t(8.3) = 0.67, p = .522. Comparing strongly with baseline condition for free will beliefs (see
the effectiveness of the manipulation across subscales again Table 2). In particular, a closer look at the coding revealed
revealed a significant main effect, F(3, 9.8) = 6.51, p = that studies combining a text with statements and studies
.011. Post hoc tests showed that the effect on scientific deter- using videos always had a pro–free will baseline. Therefore,
minism was stronger than on all other subscales (all ps ≤ to test if this could explain the effect of manipulation type,
.007) and that the effect on unpredictability was significantly we did a control analysis restricting the analysis to those
weaker than the effect on the other subscales (all ps ≤ .045; studies using a pro–free will baseline. While this changed the
see Table 5). main effect of manipulation type from significant to margin-
In sum, the analysis comparing the different scales indi- ally significant, F(3, 21.9) = 2.86, p = .060, it did not sub-
cated that anti–free will manipulations have similar effects stantially change the pattern of results. In particular,
irrespective of whether belief in free will and belief in deter- manipulations combining a text with statements were still
minism are measured with the FWI or with the FAD. significantly more effective than manipulations only using
Interestingly, the results also showed that the effects of the statements (p = .016), and manipulations only using a text
manipulation are not specific to belief in free will and belief (p = .016) were marginally more effective than manipula-
in scientific determinism, but also extend to belief in dualism tions using a video (p = .061).
and belief in fatalistic determinism, albeit not to belief in Taken together, attempts to manipulate beliefs in free will
unpredictability. are most effective when combining texts arguing against free
will with statements that help strengthen the message. They
Type of manipulation. We first compared the effectiveness are more effective than only using statements, texts, or
of the different manipulations (i.e., text, statements, text and videos.
statements combined, or video). This revealed a significant
effect of manipulation type on free will beliefs, F(3, 26.3) = Participant involvement. For participant involvement, there
6.70, p = .002 (see Table 2), but not on determinism beliefs, was a significant main effect on free will beliefs, F(2, 69.5) =
F(2, 15.7) = 1.83, p = .193 (see Table 3). 10.9, p < .001 (Table 2), but not on determinism beliefs,
A further analysis of the free will belief effect showed that F(2, 29.3) = 0.61, p = .549 (see Table 3). Further analysis of
manipulations combining a text with statements were more the free will effect showed that the manipulation was more
effective than manipulations only presenting statements, effective when participants had to report on the content of the
t(11.4) = −3.26, p = .007, only presenting a text, t(9.5) = manipulation immediately after the manipulation compared
−4.17, p = .002, or only showing a video, t(14.6) = −2.14, with when they had to report on the content at the end of the
experiment, t(63.8) = −4.67, p < .001, or not at all, t(61.9) = This suggests that belief in free will can not only be reduced
−2.79, p = .007. The latter two conditions, however, did not but can also be increased by experimental manipulations.
differ, t(58.5) = 1.19, p = .240. Importantly, like manipu-
lation type, participant involvement also correlated highly Secondary moderators
with baseline condition for free will beliefs. Visual inspec- Age. The mean age of the sample did not influence the
tion showed that most studies requiring participants to report effectiveness of the manipulation for either free will beliefs,
on the content of the manipulation immediately after the t(28.2) = 0.27, p = .793 (see Table 2), or determinism
manipulation also used a pro–free will baseline, whereas beliefs, t(19.1) = 0.21, p = .838 (see Table 3).
other studies tended to use a neutral baseline. To control for
this confound, we fitted a model including both participant Sex. The proportion of female participants in the sample
involvement and baseline condition, which revealed that par- did not influence the effect of the manipulation on either free
ticipant involvement remained significant, F(2, 50) = 4.71, will beliefs, t(31.5) = 0.53, p = .598 (see Table 2), or deter-
p = .013. minism beliefs, t(17.3) = −0.14, p = .893 (see Table 3).
In conclusion, anti–free will manipulations have the
strongest effects when experimental procedures are rehearsed Continent. The manipulation had a stronger effect on
or verified directly after the manipulation, suggesting that samples collected in the United States or on U.S.-based plat-
participant involvement increases its effectiveness. forms (e.g., MTurk) than on samples collected in Europe
or on Europe-based platforms (e.g., Prolific) for free will
Measurement moment. For measurement moment, anti– beliefs, t(101) = −2.50, p = .014 (see Table 2), but not for
free will manipulations had a larger influence on free will determinism beliefs, t(45.2) = 0.99, p = .329 (see Table 3).
beliefs, t(92.3) = −2.32, p = .022 (see Table 2), when the
belief was assessed before compared with after the primary Test location. Whether the experiment was conducted in
dependent variable was measured (i.e., downstream conse- the lab or online did not influence the effect of the manipula-
quences). The same statistical test was not significant for tion on either free will beliefs, t(77.2) = 0.45, p = .655 (see
determinism beliefs, t(33.2) = 0.95, p = .349 (see Table 3). Table 2), or determinism beliefs, t(42.6) = −0.55, p = .586
In sum, anti–free will manipulations have the strongest (see Table 3).
effects when free will beliefs are measured directly after the
experimental manipulation instead of at the end of the study. Sample type. The effectiveness of the manipulation did
This suggests the effects of the manipulation fade over the not differ between student samples and samples collected via
course of the experimental session. online platforms for either free will beliefs, t(62.6) = 0.81,
p = .422 (see Table 2), or determinism beliefs, t(28.7) =
Baseline condition. Anti–free will manipulations had a 0.62, p = .543 (see Table 3).
larger effect on free will beliefs when the baseline was a
pro–free will condition than when it was a neutral condition,
Sensitivity Analyses
t(106.7) = −4.11, p < .001 (Table 2). A similar effect was
also apparent for determinism beliefs, but did not reach sta- To test the robustness of our main results, we ran four sensi-
tistical significance, t(25) = 1.05, p = .304 (see Table 3). tivity analyses investigating how changing the analysis pro-
As discussed earlier, however, baseline condition correlated cedure influenced (a) the effect size of the manipulation and
highly with manipulation type and manipulation task. As (b) the evidence for publication bias. First, we varied the ρ
there were no studies in some cells of the Baseline Condition parameter between 0.1 and 1.0 in steps of 0.2. This resulted
× Manipulation Type cross-table, we decided to control for in identical effect sizes for both free will beliefs and deter-
manipulation type by looking at the effect of baseline condi- minism beliefs. Second, we calculated effect sizes preferen-
tion for the manipulation type where baseline condition was tially from the test statistics instead of from the means and
most balanced (i.e., statements) to optimize power (Tanner- standard deviations. This again resulted in identical estimates
Smith et al., 2016). This revealed that even when restricting for both free will beliefs and determinism beliefs. Third, we
the analysis to those studies using statements, anti–free will repeated the analysis without replacing outlier values. This
messages were more effective when the baseline condition led to slightly larger effect sizes for determinism beliefs (gz
was a pro–free will condition than when it was a neutral con- = 0.19) but identical effect sizes for free will beliefs. Finally,
dition (p = .032). To control for participant involvement, we conducted a multilevel meta-analysis instead of using
we fitted a model including both baseline condition and RVE. This resulted in identical effect sizes for both free will
participant involvement, which did not change the results beliefs and determinism beliefs. Thus, none of the four
(p = .024). changes to our analysis procedure had a notable influence on
Taken together, comparing anti–free will messages with the manipulation’s effect size. Similarly, the same four
pro–free will messages results in larger effect sizes than changes also did not have a notable influence on the publica-
comparing anti–free will messages with neutral messages. tion bias results, except that the PEESE-corrected effect size
Genschow et al. 17
Figure 3. P-curve on the entire data set (left) and on the studies with a significant manipulation check (right).
Note. See the OSF folder (https://bit.ly/2L69prl) for the evidence tables.
for determinism beliefs was no longer significant when outli- attitudes, and cognition revealed that both the full (zfull =
ers were not replaced (gz = 0.12, p = .201). For a full over- −3.54, pfull < .001) and half p-curve (zhalf = −1.38, phalf =
view of the results of the different sensitivity analyses, see .084) had a p-value < .10, and hence that the set of studies
OSF: https://bit.ly/2L69prl. contained evidential value. However, a visual inspection of
the p-curve revealed that it was almost entirely flat (see Figure
Summary of the effectiveness of free will belief manipulations. In 3). Although the flatness test did not find evidence for the
sum, our analyses indicate that anti–free will manipulations absence of an evidential value (zhalf = 5.86, phalf > .999), a
are able to successfully influence belief in free will and robustness check, as recommended by Simonsohn et al.
related beliefs. The strongest effects occur when anti–free (2015), indicated that removing the single most significant
will texts (e.g., the Crick text) are paired with anti–free will p-value across all included studies (i.e., Ching & Xu, 2018)
statements. Manipulation effects are strongest when mea- was sufficient to render the test of evidential value nonsig-
sured directly after the manipulation, and group differences nificant (zhalf = −0.70, phalf = .128). A second robustness
are largest when the anti–free will condition is compared check indicated that evidential value was stronger (zfull =
with a pro–free will condition. However, we found no mod- −4.57, pfull < .001, zhalf = −2.49, phalf = .006) when the sec-
eration by the demographics investigated here. ond rather than the first result was used from studies report-
ing multiple relevant results, and this was still true even after
Research Question 2: Do Free Will Belief removing the most significant test.
A potential explanation for why the main analysis did not
Manipulations Have Downstream Consequences?
find clear evidence for downstream effects could be that
P-curve analysis. A p-curve is considered to contain eviden- some of the included studies were not able to confirm that
tial value if the null-hypothesis that a zero true effect size their manipulation changed participants’ belief in free will.
underlies all effect sizes can be rejected. This null-hypothe- That is, a study cannot be expected to have downstream con-
sis can be rejected if the half p-curve, considering only p-val- sequences if it did not successfully manipulate the belief in
ues <.025, is significantly right-skewed at p < .05 or if the free will in the first place. To test this hypothesis, we ran a
full p-curve, considering all p-values < .05, and half p-curve second p-curve analysis including only those studies with a
are both right-skewed at p < .10. Similarly, a p-curve is con- statistically significant manipulation check (i.e., p < .05).
sidered to signal the absence of evidential value if the half Across 23 tests, this revealed no evidence for either the pres-
p-curve is significantly flatter than the curve expected with ence (zhalf = −1.09, phalf = .138) or absence (zhalf = 3.66, phalf =
33% power at p < .05 or if both the full and the half p-curve .999) of evidential value (Figure 3). A robustness check
are flatter than 33% power at p < .10 (Simonsohn et al., using the second instead of the first reported test in studies
2015). with multiple relevant tests indicated the presence of eviden-
A first p-curve analysis with 49 experiments testing the tial value (zfull = −1.87, pfull = .031, zhalf = −1.48, phalf =
influence of anti–free will manipulations on behavior, .069), but removing the single most significant test was again
Figure 4. Forest plot of the effect of the manipulation on antisocial behavior.
Note. Prosocial effect sizes are reverse coded. Published studies are annotated with “+” and unpublished studies with “-.” Nadelhoffer er al. (2019)
and Genschow and Vehlow (2019) were included in the meta-analysis as unpublished studies but were published later as Nadelhoffer et al. (2020) and
Genschow and Vehlow (2021) after data analysis had already been completed. CI = confidence interval.
sufficient to make this evidence disappear (zhalf = −0.71, Antisocial behavior. A random-effects meta-analysis with
phalf = .103). robust variance estimation indicated an effect size of g =
Taken together, the p-curve analysis finds only little evi- 0.33 for the influence of anti–free will manipulations on
dence for the hypothesis that manipulating belief in free will anti–social behavior, t(21.1) = 3.10, p = .005, 95% CI =
has downstream consequences on behavior, attitudes, and [0.11, 0.55], m = 23, k = 33.3 However, a visual inspection
cognition. That is, although some analyses provided weak of the forest plot (see Figure 4) suggested that this significant
evidence for downstream consequences, this evidence disap- effect was mostly driven by studies with smaller sample
peared in all but one case when the single lowest p-value was sizes, whereas studies with larger sample sizes tended to
excluded (Simonsohn et al., 2015). At the same time, there report null results. This was confirmed by a hierarchical
was also no conclusive evidence for the absence of an effect. Egger test, indicating a significant positive relationship
Hence, the p-curve found no clear evidence for, but also not between the effect size and the standard error, β = 3.01,
against, the hypothesis that free will belief manipulations t(12.8) = 3.52, p = .004. In other words, while the meta-
have downstream consequences. analysis suggested an effect of the manipulation on antisocial
Genschow et al. 19
Figure 5. Forest plot of the effect of the manipulation on cheating.

Note. Published studies are annotated with “+” and unpublished studies with “-.” Nadelhoffer er al. (2019) and Genschow & Vehlow (2019) were
included in the meta-analysis as unpublished studies but were published later as Nadelhoffer et al. (2020) and Genschow and Vehlow (2021) after data
analysis had already been completed. CI = confidence interval.
behavior, additional analyses indicated that this effect may (g = 0.37, m = 13, k = 20) revealed no significant difference,
have been driven by small-study bias. Correcting the effect t(19.4) = −0.34, p = .736. If anything, the effect size was
size for this bias using PET (g = 0.00) or PEESE (g = 0.00) even numerically smaller in the set of studies reporting a suc-
resulted in substantially reduced and nonsignificant effect cessful manipulation check, and this effect size was not sig-
sizes. Similarly, even though there was no significant differ- nificant, t(8.7) = 2.11, p = .065, 95% CI = [−0.02, 0.61].
ence between published (g = 0.50) and unpublished (g = In sum, the analysis showed that the effect of anti–free
0.15) studies, t(19.6) = 1.76, p = .095, the effect was will manipulations on antisocial behavior was no longer sig-
descriptively larger for published studies and was no longer nificant after controlling for publication and small sample
significant if only unpublished studies were included in the biases. This was true even when we only included studies
analysis, t(8.6) = 1.16, p = .279. Taken together, this indi- that found a significant effect of the manipulation on belief in
cates that there is insufficient evidence that anti–free will free will and indicates that there is insufficient evidence for
manipulations influence antisocial behavior. the idea that manipulating belief in free will influences anti-
However, including all studies in a meta-analysis of this social behavior.
sort may be problematic because studies that did not change
the belief in free will (e.g., failed at the manipulation check) Cheating, conformity, and punishment. No effects of anti–free
could not hope to have downstream consequences on antiso- will manipulations were found on cheating, g = 0.39, t(6.8)
cial behavior. That said, a moderator analysis comparing = 1.81, p = .114, 95% CI = [−0.12, 0.91], m = 8, k = 9,
effect sizes in studies with a significant manipulation check (g conformity, g = 0.26, t(4.7) = 1.63, p = .168, 95% CI =
= 0.30, m = 10, k = 13) to effect sizes in studies with no [−0.16, 0.68], m = 6, k = 9, or punishment, g = −0.15, t(7.5)
manipulation check or a nonsignificant manipulation check = −1.60, p = .151, 95% CI = [−0.36, 0.07], m = 9, k = 9.
Figure 6. Forest plot of the effect of the manipulation on conformity.

Note. Published studies are annotated with “+” and unpublished studies with “-.”CI = confidence interval.
In line with the antisocial behavior meta-analysis, visual Finally, we investigated if the effect was modulated by
inspection of the forest plots (see Figures 5–7) indicated that whether a significant manipulation check was reported and if
studies with smaller sample sizes tended to report significant it was significant when including only those studies report-
effects, whereas studies with larger sample sizes tended to ing a significant manipulation check. However, for cheating
report null effects. This was confirmed by a hierarchical and conformity, there were insufficient studies to test either
Egger test, showing significant relationships between the of these two hypotheses (all df ≤ 3.77). Although the cheat-
effect size and the standard error for all three dependent vari- ing effect size was descriptively larger for studies reporting a
ables (all p ≤ .014). Correcting the effect sizes for these significant manipulation check (g = 0.94, m = 3, k = 4) than
small-study biases using PET (all g = 0.00) or PEESE (all g for studies not reporting a significant manipulation check (g
= 0.00) eliminated all possible evidence. However, the = 0.02, m = 5, k = 5), this difference should not be taken at
Egger and PET/PEESE results should be interpreted with face value, given the limited number of studies and given
care, as the limited dfs (all ≤ 4.1) likely led to an inflated that three of the four included effect sizes with a significant
false-positive rate (Tipton, 2015). Similarly, for cheating and manipulation check came from the same paper (i.e., Vohs &
conformity, there were not enough studies to reliably test if Schooler, 2008). The conformity effect size was descrip-
the effect remained significant when including only the tively smaller for studies reporting a significant manipula-
unpublished studies (both df ≤ 3.9), but in both cases, effect tion check (g = 0.10, m = 3, k = 4) than for studies not
sizes were numerically smaller for unpublished (gcheating = reporting a significant manipulation check (g = 0.49, m = 3,
0.23, gconformity = 0.04) than for published studies (gcheating = k = 5). For punishment, the difference between studies that
0.69, gconformity = 0.90). For punishment, effect sizes were did or did not report a significant manipulation check was
also numerically smaller for unpublished studies (g = −0.10) not significant, t(6.0) = 0.24, p = .821. There were not
than for published studies (g = −0.23), but this difference enough studies to test whether the effect remained significant
did not reach significance, t(6.1) = −0.61, p = .563. when only including studies reporting a significant
Genschow et al. 21
Figure 7. Forest plot of the effect of the manipulation on punishment.

Note. Published studies are annotated with “+” and unpublished studies with “-.”CI = confidence interval.
manipulation check (df = 3.9). However, if anything, the from the means and standard deviations. This led to a slightly
effect was numerically smaller in studies that reported a sig- larger effect size for conformity (gz = 0.28, p = .138), but
nificant manipulation check (g = −0.13, m = 5, k = 5) than did not change the effect sizes of the other dependent vari-
in studies that did not report a significant manipulation check ables. Finally, we conducted a multilevel meta-analysis
(g = −0.18, m = 4, k = 4). instead of using RVE. This resulted in slightly higher effect
In conclusion, after controlling for small-study biases, the sizes for antisocial behavior (gz = 0.35, p = .004), cheating
effect sizes of free will belief manipulations on cheating, (gz = 0.43, p = .093), and conformity (gz = 0.33, p = .128)
conformity, and punishment became statistically indistin- but did not change the effect size for punishment. Thus, in
guishable from zero. line with the first research question, varying the analysis
strategy did not have a notable influence on the results. The
Sensitivity analyses. To test the robustness of our main results, same was also true for the publication bias results, both in the
we used the same approach as for the first research question. sense that all sensitivity analyses indicated evidence for such
That is, for all four dependent measures, we investigated a bias and in the sense that correcting the bias made the
how changing the analysis procedure influenced (a) the effects disappear regardless of the analysis approach. For a
effect size of the manipulation and (b) the evidence for pub- full overview of the sensitivity analysis results, see OSF:
lication bias. In contrast to the first research question, how- https://bit.ly/2L69prl.
ever, we now ran only three instead of four sensitivity
analyses because there were no outlier values in the data of
Discussion
the second research question. First, we varied the ρ parame-
ter from 0.1 to 1.0 in steps of 0.2. This led to identical effect Past research has shown that reducing individuals’ belief in
sizes for all four dependent measures. Second, we calculated free will affects societally relevant behaviors such as cheat-
effect sizes preferentially from the test statistics instead of ing (Vohs & Schooler, 2008) and other antisocial behaviors
(Baumeister et al., 2009; Zhao et al., 2014). These and simi- demonstrate that belief in free will can be reduced as well as
lar results have been used by some scholars as an argument increased by experimental means.
that anti–free will and deterministic viewpoints should be Second, the changes in the beliefs were relatively weak.
kept away from society, because they may change the way An open question is whether the conceptualization of free
people interact with each other. However, a number of stud- will in the manipulation influences the magnitude of the
ies reported difficulties in replicating some of the key results effect. Indeed, philosophical definitions of free will are
in the field (Genschow et al., 2020; Giner-Sorolla et al., rather diverse and complex (Carey & Paulhus, 2013) and
2016; Monroe et al., 2017; Open Science Collaboration, may thus be impalpable for laypeople. If true, free will belief
2015; Schooler et al., 2014; Shariff & Vohs, 2014). While manipulations relying on such definitions are unlikely to
these failed replications call into question the societal rele- have a strong impact on laypeople’s beliefs. To mitigate this
vance of belief in free will, it is not yet clear what caused problem, future research could investigate whether manipu-
them. In this article, we tested three possible explanations lations that focus more specifically on laypeople’s concepts
that may account for the failed replications. First, it could be of free will produce stronger effects.
that the failed replications are false negatives. That is, they Third, from a methodological point of view, it is impor-
were not able to detect an effect that is actually real. Second, tant to know which manipulations most reliably alter belief
it could be that the manipulations commonly used in the lit- in free will. By comparing all the different manipulations
erature do not alter individuals’ belief in free will and thus used in previous research, our analysis suggests that an
have no impact on other behaviors. Third, it could be that approach in which participants have to both read a text and
free will belief manipulations reduce belief in free will, but reproduce statements produces the largest effects.
that this does not have any downstream consequences. Fourth, an interesting and open question was how deeply
Our meta-analysis favors the third explanation. That is, participants have to process the stimulus information for the
we found that beliefs related to free will can be effectively manipulation to succeed. To answer this question, we inves-
manipulated by commonly used experimental manipula- tigated whether the effectiveness of the manipulation
tions, although the effects were rather small, with g = −0.29 depended on the degree to which participants had to engage
for free will beliefs and g = 0.17 for determinism beliefs. with the task (e.g., by summarizing or rewriting presented
However, we did not find evidence for downstream conse- messages). Manipulations worked best when participants
quences on attitudes, behavior, or cognition. In the remainder had to summarize the stimulus information directly after the
of this article, we discuss the implications of our results for manipulation. In other words, actively processing the pro-
the processes underlying free will belief manipulations, the vided information (e.g., by summarizing it) increased the
potential reasons for why these manipulations did not have effect of the manipulation. This indicates that participants’
downstream consequences, the societal implications of the involvement in the task strengthens the effect of free will
meta-analysis, possible steps for future research, and the belief manipulations—a finding that is in line with previous
limitations of the present meta-analysis. research on the self-generation effect (Slamecka & Graf,
1978) where self-generated information increases memory
Processes Underlying Free Will Belief performance of that information.
Fifth, past research left unanswered the question of how
Manipulations long the effects of free will belief manipulations last.
The analysis of our first research question on the effective- Measuring beliefs at the end of the experiment assumes the
ness of free will belief manipulations indicated that beliefs manipulation lasts through the duration of the experiment;
related to free will can be influenced by experimental manip- this is not necessarily the case, however. For instance, it could
ulations. Although we found evidence for publication bias, also be that free will belief manipulations do not fully change
we also found that this cannot explain the effect. To investi- people’s beliefs but merely activate an anti–free will mindset
gate the conditions under which these manipulations were for a brief moment. To test how long-lasting the effects of free
effective, we conducted several moderator analyses. The will belief manipulations are within the span of an experi-
results of these analyses further the understanding of free ment, we investigated if the effect of the manipulation dif-
will belief manipulations manifold. fered depending on whether beliefs were measured
First, previous research indicated that most people believe immediately after the manipulation or at the end of the experi-
that they have free will (Baumeister et al., 2009; Nahmias ment (i.e., after completing another task). At least for belief in
et al., 2005). Based on this finding, it has been assumed that free will, the effect reduced over time. While the same was
free will beliefs can only be decreased (for an overview, see not true for belief in determinism, it is important to note that
Ewusi-Boisvert & Racine, 2018). While this assumption has deterministic beliefs were generally less influenced by mod-
never been systematically investigated, the present meta- erators. This could mean that determinism beliefs are more
analysis allowed us to test whether experimental manipula- robust than beliefs in free will, but there are also several alter-
tions can also increase the belief in free will. The results native explanations. For example, the meta-analysis included
Genschow et al. 23
fewer studies measuring belief in determinism than belief in effects on belief in free will and/or determinism) were used
free will and the determinism moderator analyses may there- in the analysis. This indicates that there is currently insuffi-
fore have had less statistical power to detect such effects cient evidence for downstream consequences.
(Hempel et al., 2013). Alternatively, it could be that because How can this be reconciled with the fact that some pri-
belief in determinism was less influenced by the manipulation mary studies did find downstream consequences? The results
as such, there was a floor effect, making it difficult to further of our meta-analysis suggest that one likely candidate is pub-
reduce the effect of the manipulation. lication bias. Indeed, the results of the Egger test indicated
Sixth, the meta-analysis yielded a borderline nonsignifi- that studies with larger effect sizes tended to be studies with
cant trend indicating that anti-free will manipulations influ- higher standard errors. This is a typical sign of publication
enced beliefs in free will slightly more strongly than beliefs bias because studies with small samples or noisy measure-
in determinism. Given that this effect was not very strong ments only reach statistical significance if the effect size is
and not significant based on conventional levels of signifi- large (Sterne & Egger, 2005). Hence, a potential reason why
cance, it should be interpreted with care. Nevertheless, if we could not confirm previous studies finding downstream
confirmed by future research, a potential explanation could consequences is that these studies are false positives.
be that existing manipulations target beliefs in free will more Nevertheless, it is important to note that absence of evi-
strongly than beliefs in determinism. As beliefs in free will dence does not equal evidence of absence. Indeed, the
are not typically correlated with belief in determinism p-curve analysis lumped together a variety of different vari-
(Nadelhoffer et al., 2014), it is not surprising that such ables, and the meta-analysis focused only on a limited sub-
manipulations would have different effects on both beliefs. sample of outcomes for which there were enough data points.
The stronger effect of anti–free will manipulations on free Thus, we cannot rule out that there are other specific vari-
will beliefs could thus indicate that these manipulations spe- ables that are influenced by free will belief manipulations.
cifically influence the belief in free will and only influence For example, we did not have enough data to investigate the
other beliefs to a lesser degree. However, in contrast to this influence of free will belief manipulations on neurocognitive
view, the results of our meta-analysis suggest that the influ- processes (Rigoni et al., 2011, 2012, 2015), feelings of alien-
ence of anti–free will manipulations tends to be rather unspe- ation (Seto & Hicks, 2016), attributions of other people’s
cific. For example, we found that the effect of anti–free will actions (Genschow et al., 2017a), perceived meaningfulness
manipulations is at least equally strong, if not stronger, on of life (Crescioni et al., 2016; Moynihan et al., 2019), per-
belief in dualism than on belief in free will. Similarly, we ceived gratitude (MacKenzie et al., 2014), counterfactual
found that belief in free will manipulations influence not thinking (Alquist et al., 2015), or risk-taking behavior
only belief in free will and belief in scientific determinism (Schrag et al., 2016). For these and potential other variables
but also belief in fatalistic determinism. There are different that have not been studied yet, it remains open to what degree
possible explanations for these unspecific effects. Some of they are influenced by free will belief manipulations,
the beliefs are correlated with each other and are thus quite although one of the effects on neurocognitive processes
unspecific themselves. For example, belief in free will cor- recently failed to replicate (Eben et al., 2020).
relates positively with belief in dualism (Nadelhoffer et al., It is also possible that the failure to find robust evidence
2014; Wisniewski et al., 2019). In addition, it is likely that for previously reported downstream effects of free will belief
the manipulations affect not only beliefs related to free will, manipulations has to do with subtle differences in the man-
but also other psychological and cognitive factors. For, ner in which the experimental protocols were implemented
instance, it is conceivable that challenging the fundamental by the research teams that were versus were not successful in
belief that free will exists leads to a general feeling of confu- finding such effects. Differences in experimental outcomes
sion and uncertainty about the world. between labs might reflect variations in the implementation
of effective experimental procedures (e.g., ensuring that par-
Insufficient Evidence for Downstream ticipants were sufficiently motivated, believed the experi-
mental ruse, and followed the protocol), could be due to the
Consequences and Its Potential Reasons inclusion of artifacts that produced false-positive effects
While we found clear evidence that beliefs related to free (e.g., introducing experimenter demands characteristics), or
will can be experimentally manipulated, there was little evi- could be driven by the belief of the research teams to find an
dence that these free will manipulations have downstream effect (Doyen et al., 2012). Future studies, perhaps by
consequences. That is, both p-curve analyses across all research teams that have previously been effective in finding
dependent variables and meta-analyses on internally coher- downstream effects, might profitably examine whether pro-
ent sets of dependent variables found either no evidence for cedural differences can be identified that reliably differenti-
downstream consequences or weak evidence that, in all but ate between when such effects are versus are not observed.
one instance, disappeared when correcting for extreme val- Even if currently employed manipulations do turn out, as
ues or small-sample effects. This remained true even if only the present findings suggest, to not produce robust down-
studies with significant manipulation checks (i.e., significant stream effects, this does not necessarily mean that such
effects cannot happen. It might be that existing manipula- perspective, downstream consequences may be visible only
tions are not suited for this purpose. In other words, it might when they are measured directly after the manipulation.
well be that experimentally reducing beliefs in free will has Future research should test this hypothesis more directly.
downstream consequences but that established free will Finally, it is possible that some participants are not com-
belief manipulations are not able to produce them. There are mitted to a particular belief about free will but nevertheless
several reasons why this might be the case. As mentioned have strong moral convictions about behaviors like cheating
earlier, the manipulations are rather unspecific and do not (see, for example, Dubljević, 2013), for instance. In this
only affect belief in free will and determinism, but also other case, manipulating belief in free will may influence beliefs in
beliefs. This is problematic for at least two reasons. On the free will without having downstream consequences, because
one hand, an unspecific manipulation may be detrimental in moral beliefs are held separate from their other beliefs.
finding downstream consequences because the different fac- Future research could try to test this hypothesis in more
tors influenced by the manipulation may counteract the effect detail.
of free will beliefs on the dependent variable. On the other Taken together, many possible explanations exist for why
hand, an unspecific manipulation opens the question to we could not find evidence for downstream consequences in
which degree downstream consequences (if any) are actually this meta-analysis. An important task for future research will
driven by free will beliefs or rather by other beliefs and psy- be to investigate these explanations and to determine whether
chological variables. and under which circumstances free will belief manipula-
In addition, existing manipulations of free will beliefs tions influence attitudes, behavior, and cognition.
have only weak effects on free will (g = −0.29) and deter-
minism (g = 0.17) beliefs. Given that any effect of anti–free
will manipulations on attitudes and behavior is likely smaller
Societal Implications
than their effect on the beliefs they purport to change, this Whether free will exists is part of a long-standing philosoph-
makes finding evidence for downstream consequences par- ical debate (e.g., Dennett, 2015; Van Inwagen, 1983).
ticularly challenging. Indeed, research on the relationship However, ever since cognitive neuroscientists and psycholo-
between behavioral intentions and actual behavioral change gists started claiming that humans’ perception of free will is
shows that the influence of behavioral interventions on nothing more than an illusion (e.g., Crick, 1994; Harris,
behavior is roughly twice as small as their influence on 2012; Wegner, 2002), anti–free will viewpoints have become
behavioral intentions (Webb & Sheeran, 2006). The same in vogue not only in academia (e.g., Greene & Cohen, 2004)
problem is likely true for belief in free will and its down- but also in popular media (e.g., Chivers, 2010; Griffin, 2016;
stream consequences. In this sense, it may be helpful to relate Racine et al., 2017; Wolfe, 1997). When psychological
our findings to the idea of “attitude strength” (Petty & research found that presenting individuals with such anti–
Krosnick, 2014). Attitude strength is defined “as the extent to free will viewpoints influences fundamental behavior, cogni-
which attitudes manifest the qualities of durability and tion, and attitudes, the question arose whether the public
impactfulness” (Petty & Krosnick, 2014; p. 3). Durability press should publish such anti–free will viewpoints. While
refers to the degree to which an attitude can be changed over some philosophers argue that undermining people’s belief in
time and how strongly it resists attacks (e.g., persuasion free will would have catastrophic consequences, as free will
attempts). Impactfulness relates to how strongly an attitude forms the basis for moral behavior (e.g., Smilansky, 2000,
influences information processing, judgments, and behav- 2002), other philosophers argue that disbelieving in free will
iors. Applying the idea of attitude strength to beliefs in free might also have positive effects because it could lead to
will may help to explain why we did not find evidence for abandoning retribution-based morality and illusory beliefs in
downstream consequences. That is, free will beliefs are most a just world (Caruso, 2014; Greene & Cohen, 2004;
likely to influence attitudes, behavior, and cognition when Nadelhoffer, 2011). The present research adds to this debate
they are strong. However, people with strong beliefs are least by suggesting that confronting individuals with anti–free
likely to be influenced by manipulation. Moreover, even if will viewpoints might not have as strong consequences as
people were influenced by the manipulation, our meta-anal- has been previously assumed.
ysis indicates that these new beliefs are likely to be weak. Although these manipulations affect people’s beliefs in
Given that weak beliefs are less likely to influence behavior, free will, we did not find evidence for effects on behavior,
this might explain why we did not find evidence for down- cognition, or attitudes. This is in line with recent findings
stream consequences in the current meta-analysis. indicating that although professional judges’ beliefs in free
Related to this idea, another potential reason for the dif- will are influenced by reading anti–free will texts, reading
ficulty to find downstream effects could be that the effect of these viewpoints does not influence their judgments
the manipulation on free will beliefs fades over time, as our (Genschow et al., 2020). However, it is important to keep in
results indicate. As a result, it is possible that in some experi- mind that all these findings only speak to the effect of pre-
ments, the effect of the manipulation had already disappeared senting individuals with a single anti–free will viewpoint.
when the dependent variable was measured. From this While such a short exposure may not have downstream
Genschow et al. 25
consequences, it remains an open question whether more Therefore, in addition to using bias correction methods, we
concentrated and repeated presentations of anti–free will also compared published with unpublished studies and tested
messages, as may happen in real-life, could nevertheless if the effects remained significant if only unpublished studies
have important consequences. Indeed, preliminary support were considered. While no single method is perfect by itself,
for this possibility comes from the results of the present we believe that by combining these different methods, we
meta-analysis, demonstrating that a combined manipulation were able to get a clearer overview of the underlying true
of presenting participants with a text and statements has the effects and the degree to which they were inflated by publi-
strongest impact on individuals’ belief in free will. It would cation bias.
be interesting to test to what extent repeated exposure to Second, the current meta-analysis included a large num-
anti–free will messages may have stronger effects on indi- ber of unpublished studies. This approach is consistent with
viduals’ belief in free will and thereby also lead to down- recent calls to include unpublished research in meta-analyses
stream consequences. as a means to counter publication bias (Polanin et al., 2016).
In the same vein, although we did not find sufficient sup- However, a potential downside is that the unpublished stud-
port for the idea that conventional belief in free will manipu- ies may have been of inferior quality to the published
lations have downstream consequences, it is important to research. One way to rule this out is to code study quality.
note that we do not necessarily argue that belief in free will Existing tools focus strongly on clinical intervention studies,
itself has no impact. That is, on an interindividual level, however, and are therefore not easily applied to the research
belief in free will may well contribute to societally relevant synthesized here (Armijo-Olivo et al., 2012). Instead, the
behaviors such as retributive punishment and anti- or proso- main indicators of study quality in research on free will
cial behavior—to name just a few examples. Indeed, several belief manipulations (and other experimental social psycho-
studies could reliably replicate the link between belief in free logical research) are blinded and random assignment, sample
will and retributive punishment on a correlational level size, and the use of validated manipulations and scales. With
(Genschow et al., 2017a; Martin et al., 2017). Similar strong respect to blinded and random assignment, we had initially
correlations were also found between belief in free will and planned to include a moderator coding whether the experi-
job satisfaction (Feldman et al., 2018), intentional attribu- menter was blind to the manipulation. However, after com-
tions (Genschow et al., 2017a, 2019a; Genschow & Lange, pleting the coding, it was clear that this variable was almost
in press), as well as between free will beliefs and just world completely confounded with study location, such that online
beliefs, religious worldviews, and a conservative worldview studies were blinded, whereas papers on studies conducted in
(Carey & Paulhus, 2013; Genschow & Vehlow, 2021). This the lab did not provide information about this variable.
suggests that on a correlational level, belief in free will may Indeed, experimenter blindness could only be coded for
well be connected to societally relevant behaviors. seven studies conducted in the lab, of which three were our
Taken together, there is a debate about whether anti–free own unpublished studies. A similar problem exists for ran-
will viewpoints should be discussed in the public media. Our dom assignment. While the random assignment is standard
findings suggest that the influence on society may be weaker in the field, whether or not group assignment was indeed ran-
than previously assumed. In this respect, we would like to dom is often not explicitly reported in the paper, making it
argue that discussions about the implications of believing in difficult to code.
free will should distinguish between scientific facts and phil- The three other indicators of study quality (i.e., sample
osophical speculations (Schooler, 2010) as well as acknowl- size, manipulation, and scale) were explicitly addressed in the
edge methodological limitations of the cited research (Racine meta-analysis. With respect to sample size, the average num-
et al., 2017). ber of participants per group was slightly larger in unpublished
(N = 96) than in published studies (N = 85). With respect to
the manipulation, the best validated method to manipulate
Limitations of the Present Meta-Analysis
belief in free will is the Crick text. Of the 55% published and
There are a few limitations to this meta-analysis that call for 45% unpublished studies using a text to manipulate beliefs in
a careful discussion. First, all publication bias correction free will, 59% of the published and 91% of the unpublished
methods have downsides (Carter et al., 2019; Stanley, 2017) studies used the Crick text. Finally, with respect to the scale,
and should hence be interpreted with care. We used PET and 53% of the published and 74% of the unpublished studies used
PEESE because they are easily incorporated within the RVE a validated scale. Hence, a deeper analysis of the study quality
framework (Rodgers & Pustejovsky, 2020) and have been indicates that, if anything, unpublished studies were of higher
shown to retain reasonable false-positive rates across a wide quality than published studies. More generally, methodologi-
range of scenarios (Carter et al., 2019). However, PET- cal differences between published and unpublished studies in
PEESE can suffer from low power, especially when sample the current meta-analysis were small. This was also confirmed
sizes are small, heterogeneity is high, or when there is either by our analysis of moderator correlations (see Table 4), which
very little or very heavy use of questionable research prac- found no evidence for substantial correlations between publi-
tices (Carter et al., 2019; Rodgers & Pustejovsky, 2020). cation status and any of the coded variables.
Third, although no correlations were observed with publi- (Sarkissian et al., 2010; Wisniewski et al., 2019), there is
cation status, large correlations were observed for other extensive literature showing cross-cultural differences in
moderators. Such confounding of moderators is inevitable in how individuals construe agency and choice (e.g., Furnham
meta-analytic research (Lipsey & Wilson, 2001) and was et al., 1994; Markus & Kitayama, 2003). In line with a
addressed by performing control analyses analyzing con- recent call for cross-cultural replications in social psychol-
founded moderators together. Whereas most effects remained ogy (Genschow et al., 2021), an interesting avenue for
even after controlling for these confounds, truly controlling future research could therefore be to explore the influence
for moderator overlap is difficult and these findings should of the manipulation in different continents and cultures.
hence be tested more directly in future empirical work. Such research could test not only whether there is a differ-
Finally, meta-analytical moderator analyses are known to ent influence of free will belief manipulations on the beliefs
often be underpowered (Hempel et al., 2013). It is therefore themselves but also whether cultural differences moderate
possible that some of the nonsignificant moderator effects the degree to which these manipulations have downstream
found here could be explained by a lack of power. However, consequences.
it is worth noting that at least for the meta-analysis on beliefs Fourth, besides cultural differences and the moderators
in free will and determinism, most analyses included a rela- investigated in our meta-analysis, there might be other fac-
tively large number of studies (see Tables 2 and 3). While we tors that moderate the influence of free will belief manipu-
are not aware of research that has systematically assessed the lations on beliefs, attitudes, behavior, and cognition.
statistical power of RVE meta-analyses under different con- Indeed, there is a discussion regarding the degree to which
ditions, previous simulation work using regular meta-analyt- hidden moderators and high context sensitivity influence
ical approaches suggests that given the parameters of the psychological effects and account for the success of repli-
current meta-analysis, many of the moderator analyses were, cations (e.g., Stroebe & Strack, 2014; Van Bavel et al.,
in fact, well powered to detect even fairly small effect sizes 2016). Future research should investigate hidden modera-
(Hempel et al., 2013). tors and test how context-dependent free will belief manip-
ulations are.
How to Move Forward? Possible Steps and Fifth, an interesting question that we did not investigate in
this meta-analysis is the extent to which the dependent mea-
Recommendations for Future Research sures investigated here can be influenced by other variables.
Our results offer several promising routes for future research. For example, it could be that psychological variables related
First, future research should continue investigating the exact to free will such as locus of control (Rotter, 1966) or self-
underlying mechanisms of free will belief manipulations to fur- efficacy (Bandura, 1977), for example, have stronger down-
ther increase the understanding of these manipulations. For stream consequences. Directly comparing the effects of these
example, recent research suggests that it is important to con- different variables would be interesting, because there are
sider not just beliefs but also attitudes toward free will (Cracco subtle differences between them that could tell us exactly
et al., 2020). From this perspective, a potential avenue for what determines the studied behavior. For example, whereas
future research could be to investigate whether attitudes toward belief in free will can be defined as an attitude, locus of con-
free will alter the effect of anti–free will manipulations. trol is a personality dimension (Waldman et al., 1983).
Second, to better test the downstream consequences of Hence, if the locus of control is associated more strongly
free will belief manipulations, researchers should aim to with cheating than belief in free will, this could indicate that
develop manipulations that (a) more specifically manipulate internal attribution traits are more important for cheating
belief in free will and determinism, but not other factors and than internal attribution attitudes. While such an analysis
(b) lead to larger effect sizes, for example, by using dual- goes beyond the scope of the present meta-analysis, future
approach manipulations where participants read and repeat research could investigate whether free will belief manipula-
the presented messages. Implementing these changes may tions relate to different behaviors, attitudes, and thoughts
allow testing whether belief in free will manipulations has a than manipulations focusing on other psychological
meaningful societal relevance. variables.
Third, future research could also investigate the some- Finally, although we did not find sufficient support for the
what surprising results obtained in this meta-analysis. For hypothesis that free will belief manipulations have down-
example, an interesting finding is that anti–free will manip- stream consequences, it might still be that on a correlational
ulations appeared to have stronger effects in the United level, free will beliefs relate to individuals’ behavior, cogni-
States than in Europe. A potential explanation for this effect tion, and attitudes. Future research could, thus, (a) test to
might be that individual responsibility and personal agency which degree previous findings reported in the literature can
is more central to U.S. culture than to European culture. be replicated on a correlational level and (b) investigate
Indeed, although previous investigations suggest that irre- whether other relevant societal factors are modulated by
spective of culture most people believe in free will individuals’ belief in free will.
Genschow et al. 27
Summary Notes
The present meta-analysis finds support for the idea that 1. Based on this approach we had to exclude only two published
and two unpublished experiments.
beliefs related to free will can be experimentally manipu-
2. Note that robust variance estimation (RVE) uses a simplistic
lated. These effects are stronger when participants are pre- method to estimate I² and that this should therefore be seen
sented with a combination of texts and statements, when they as a rough indicator of heterogeneity rather than as a precise
have to report on the content of the manipulation, and when estimate (Tanner-Smith et al., 2016).
belief in free will is measured directly after the manipulation. 3. Note that after data collection had already been completed,
Moreover, beliefs related to free will can be experimentally Many Labs 5 was published, containing five replications of
increased as well as decreased. However, the used manipula- Vohs and Schooler’s (2008) Experiment 1 (Buttrick et al.,
tions produce rather weak effects and are rather unspecific in 2020). While these five experiments are not included in the
the sense that beliefs and concepts other than belief in free meta-analysis, we did check whether adding them changed the
will are also affected. Although we find support for the idea results. For antisocial behavior, the estimated effect size with
that individuals’ belief in free will can be experimentally these five experiments included was g = 0.29 (m = 28, k = 38,
p = .004). For cheating, it was g = 0.27 (m = 13, k = 14, p =
manipulated, the meta-analysis did not find evidence for the
.082). In both cases, a hierarchical Egger test found evidence
idea that these manipulations have meaningful downstream for small-study bias (both p = .004) and both the precision-
consequences. These findings call into question prior claims effect test (PET) and the precision-effect estimate with stan-
of a causal relationship between belief in free will and atti- dard errors (PEESE) indicated that the corrected effect size was
tudes, behavior, and cognition (a number of which were g = 0. In other words, the antisocial behavior and cheating
made by authors of this article). Further research is war- meta-analyses yielded very similar results with and without
ranted to determine whether free will beliefs are related to these five experiments.
attitudes, behavior, and cognition on an interindividual (cor-
relational) level and might have downstream consequences
when stronger and more specific manipulations are used. References
References marked with an asterisk indicate studies included in the
Authors’ Note meta-analysis.
Coding manual, data, and analysis scripts are open accessible at the *Akin, M., Braun, I., Daenhardt, K., Fischer, S., Kollmann, O.,
Open Science Framework (OSF; https://bit.ly/2L69prl) Müller, F., Pistillo, E., Schmidt, K., Stritzel, C., Ueffing, D.,
We did not apply for ethical approval because we did not con- Weber, A., Werkle, A., & Wilke, J. (2016). Macht der Glaube
duct a study on humans, but rather reviewed existing literature in a an Willensfreiheit dankbarer? [Does the belief in free will
meta-analysis. increase gratefulness?] [Unpublished thesis].
*Alquist, J. L., Ainsworth, S. E., & Baumeister, R. F. (2013).
Author Contributions Determined to conform: Disbelief in free will increases confor-
mity. Journal of Experimental Social Psychology, 49, 80–86.
Oliver Genschow developed the research idea; all co-authors gave *Alquist, J. L., Ainsworth, S. E., Baumeister, R. F., Daly, M., &
feedback on the basic research idea; Oliver Genschow conducted Stillman, T. F. (2015). The making of might-have-beens: Effects
the literature research; Oliver Genschow, Emiel Cracco, and John of free will belief on counterfactual thinking. Personality and
Protzko collected unpublished data; Oliver Genschow, Emiel Social Psychology Bulletin, 41, 268–283.
Cracco, and Jana Schneider screened and coded the data; Emiel Armijo-Olivo, S., Stiles, C. R., Hagen, N. A., Biondo, P. D., &
Cracco conducted the analyses; Oliver Genschow & Emiel Cracco Cummings, G. G. (2012). Assessment of study quality for sys-
drafted the paper, all authors gave feedback on the drafted tematic reviews: A comparison of the Cochrane collaboration
manuscript. risk of bias tool and the effective public health practice project
quality assessment tool: Methodological research. Journal of
Declaration of Conflicting Interests Evaluation in Clinical Practice, 18, 12–18.
The author(s) declared no potential conflicts of interest with respect *Ball, E.-C., Degen, D., Fohrer, A., Grunwald, M., Hintze, S.,
to the research, authorship, and/or publication of this article. Höhn, V., Kästel, M., Koziol, P., Matz, M., Papadakis, A.,
Porth, T., Renner, M., Seyder, M., Willems, S., & Ziske, A.
Funding (2016). Der Glaube an den freien Willen und die Neigung zur
Konformität [The belief in free will and the tendency to confor-
The author(s) received no financial support for the research, author- mity.] [Unpublished thesis].
ship, and/or publication of this article. *Balodis, K. (2016). Influence of belief in free will on intertemporal
decisions [Unpublished bachelor thesis].
ORCID iDs Bandura, A. (1977). Self-efficacy: Toward a unifying theory of
Oliver Genschow https://orcid.org/0000-0001-6322-4392 behavioral change. Psychological Review, 84, 191–215.
*Bastart, J., Redersdorff, S., & Martinot, D. (2015). Le libre arbitre
Emiel Cracco https://orcid.org/0000-0003-4043-5992
au service du jugement émis envers des victimes de sex-
Jana Schneider https://orcid.org/0000-0003-3531-1696 ism [Free will in the service of judging victims of sexism].
John Protzko https://orcid.org/0000-0001-5710-8635 Psychologie Française, 60, 223–236.
*Baumeister, R. F., Masicampo, E., & DeWall, C. N. (2009). *Crescioni, A. W., Baumeister, R. F., Ainsworth, S. E., Ent, M., &
Prosocial benefits of feeling free: Disbelief in free will Lambert, N. M. (2016). Subjective correlates and consequences
increases aggression and reduces helpfulness. Personality and of belief in free will. Philosophical Psychology, 29, 41–63.
Social Psychology Bulletin, 35, 260–268. Crick, F. (1994). The astonishing hypothesis: The science search
*Becker, L., Kandt, P., Leupelt, A., & Matthies, I. (2018). Das for the soul. Touchstone.
Ebenbild Gotttes oder ein Sklave seiner Triebe? [Unpublished *Crone, D. L., & Levy, N. L. (2019). Are free will believers nicer
thesis]. people?(Four studies suggest not). Social Psychological and
*Blatz, L., Genschow, O., & Crusius, J. (2017). Untitled data [Data Personality Science, 10, 612–619.
set]. *Darrall, A. (2017). Freiheit! Für unsere Handlungen, aber nicht
*Bomke, N., Goldapp, M., & Schlack, A. (2017). Der Glaube an für unseren Willen. Konsequenzen einer philosophischen
den freien Willen, Dualismus und Ambivalenz [Unpublished Theorie zum freien Willen [Freedom! For our actions, but not
thesis]. for our will. Consequences of a philosophical theory of free
*Braem, S., & Vermeylen, L. (2016). Untitled data [Data set]. will.] [Unpublished master’s thesis].
Brass, M., Furstenberg, A., & Mele, A. R. (2019). Why neurosci- Dennett, D. C. (2015). Elbow room: The varieties of free will worth
ence does not disprove free will. Neuroscience & Biobehavioral wanting:. MIT Press.
Reviews, 102, 251–263. *De Souter, L. (2017). The influence of belief in free will on pseudo-
Buttrick, N. R., Aczel, B., Aeschbach, L. F., Bakos, B. E., random behavior [Unpublished master thesis].
Brühlmann, F., Claypool, H. M., Hüffmeier, J., Kovacs, M., *Diel, K., & Genschow, O. (2018). Untitled data [Data set].
Schuepfer, K., Szecsi, P., Szuts, A., Szöke, O., Thomae, M., Doyen, S., Klein, O., Pichon, C.-L., & Cleeremans, A. (2012).
Torka, A., Walker, R. J., & Wood, M. J. (2020). Many labs Behavioral priming: It’s all in the mind, but whose mind?
5: Registered replication of Vohs and Schooler (2008), experi- PLOS ONE, 7, Article e29081.
ment 1. Advances in Methods and Practices in Psychological Dubljević, V. (2013). Autonomy in neuroethics: Political and not
Science, 3(3), 429–438. metaphysical. AJOB Neuroscience, 4, 44–51.
Carey, J. M., & Paulhus, D. L. (2013). Worldview implications of Eben, C., Zhang, C., Cracco, E., Brass, M., Billieux, J., &
believing in free will and/or determinism: Politics, morality, Verbruggen, F. (2020). Are post-error adjustments influenced
and punitiveness. Journal of Personality, 81, 130–141. by beliefs in free will? A failure to directly replicate Rigoni,
Carpenter, C. J. (2020). Meta-analyzing apples and oranges: How to Wilquin, Brass and Burle, 2013. Royal Society Open Science,
make applesauce instead of fruit salad. Human Communication 7, 200664.
Research, 46, 322–333. *Evans, J. (2013). The moral psychology of determinism.
Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J. Philosophical Psychology, 26, 639–661.
(2019). Correcting for bias in psychology: A comparison of Ewusi-Boisvert, E., & Racine, E. (2018). A critical review of meth-
meta-analytic methods. Advances in Methods and Practices in odologies and results in recent research on belief in free will.
Psychological Science, 2, 115–144. Neuroethics, 11, 1–14.
Caruso, G. D. (2014). (Un) just deserts: The dark side of moral Feldman, G., Farh, J.-L., & Wong, K. F. E. (2018). Agency beliefs
responsibility. Southwest Philosophy Review, 30, 27–38. over time and across cultures: Free will beliefs predict higher
*Caspar, E. A., & Vuillaume, L. (2017). The influence of (dis) belief job satisfaction. Personality and Social Psychology Bulletin,
in free will on immoral behavior. Frontiers in Psychology, 8, 44, 304–317.
Article 20. Field, A. P., & Gillett, R. (2010). How to do a meta-analysis. British
Cheung, M. W. L. (2014). Modeling dependent effect sizes with Journal of Mathematical and Statistical Psychology, 63, 665–
three-level meta-analyses: a structural equation modeling 694.
approach. Psychological Methods, 19, 211–229. Fisher, Z., & Tipton, E. (2015). Robumeta: An R-package for robust
*Ching, B. H.-H., & Xu, J. T. (2018). The effects of gender neu- variance estimation in meta-analysis. arXiv:1503.02220.
roessentialism on transprejudice: An experimental study. Sex Frith, C. D. (2012). The role of metacognition in human social
Roles, 78, 228–241. interactions. Philosophical Transactions of the Royal Society
Chivers, T. (2010, October 12). Neuroscience, free will and deter- B, 367, 2213–2223.
minism: “I’m just a machine.” The Telegraph. http://www. Furnham, A., Kirkcaldy, B. D., & Lynn, R. (1994). National atti-
telegraph.co.uk/news/science/8058541/Neuroscience-free- tudes to competitiveness, money, and work among young
will-and-determinism-Im-just-a-machine.html people: First, second, and third world differences. Human
*Clark, C. J. (2019). Untitled data [Data set]. Relations, 47, 119–132.
*Clark, C. J., Baumeister, R. F., & Ditto, P. H. (2017). Making *Genschow, O. (2019). Untitled data [Data set].
punishment palatable: Belief in free will alleviates punitive dis- Genschow, O., Hawickhorst, H., Rigoni, D., Aschermann, E., &
tress. Consciousness and Cognition, 51, 193–211. Brass, M. (2020). Professional judges’ disbelief in free will and
Cracco, E., Bardi, L., Desmet, C., Genschow, O., Rigoni, D., De punishment. Social Psychological and Personality Science, 12,
Coster, L., Radkova, I., Deschrijver, E., & Brass, M.. (2018). 357–362.
Automatic imitation: A meta-analysis. Psychological Bulletin, Genschow, O., & Lange, J. (in press). Belief in free will is related to
5, 453–500. internal attribution in self-perception. Social Psychological and
Cracco, E., González-García, C., Hussey, I., Braem, S., & Personality Science. https://doi.org/10.1177/19485506211057711
Wisniewski, D. (2020). Cultural pressure and biased responding *Genschow, O., Rigoni, D., & Brass, M. (2014). Untitled data
in free will attitudes. Royal Society Open Science, 7, 191824. [Data set].
Genschow et al. 29
*Genschow, O., Rigoni, D., & Brass, M. (2016a). Untitled data *Highhouse, S., & Rada, T. B. (2015). Different worldviews
[Data set]. explain perceived effectiveness of different employment tests.
*Genschow, O., Rigoni, D., & Brass, M. (2016b). Untitled data International Journal of Selection and Assessment, 23, 109–119.
[Data set]. *Jettinghoff, W. (2017). Untitled data [Data set].
*Genschow, O., Rigoni, D., & Brass, M. (2017a). Belief in free Keller, J. (2005). In genes we trust: The biological component of
will affects causal attributions when judging others’ behavior. psychological essentialism and its relationship to mechanisms
Proceedings of the National Academy of Sciences, 114, 10071– of motivated social cognition. Journal of Personality and
10076. Social Psychology, 88, 686–702.
*Genschow, O., Rigoni, D., & Brass, M. (2017b). Untitled data *Lantian, A., & Badea, C. (2019). Untitled data [Data set].
[Data set]. Libet, B., Gleason, C. A., Wright, E. W., & Pearl, D. K. (1983).
Genschow, O., Rigoni, D., & Brass, M. (2019a). The hand of god Time of conscious intention to act in relation to onset of cere-
or the hand of Maradona? Believing in free will increases per- bral activity (readiness-potential) the unconscious initiation of
ceived intentionality of others’ behavior. Consciousness and a freely voluntary act. Brain, 106, 623–642.
Cognition, 70, 80–87. Libet, B., Wright, E. W., & Gleason, C. A. (1993). Readiness-
*Genschow, O., Rigoni, D., & Brass, M. (2019b). Untitled data potentials preceding unrestricted’ spontaneous’ vs. pre-
[Data set]. planned voluntary acts. In B. Libet (Ed.), Neurophysiology of
Genschow, O., & Vehlow, B. (2021). Free to blame? The role of free consciousness (pp. 229–242). Birkhäuser.
will belief in victim blaming. Consciousness and Cognition, Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis.
88, 103074. SAGE.
Genschow, O., Westfal, M., Crusius, J., Bartosch, L., Feikes, K. *Lynn, M., Van Dessel, P., & Brass, M. (2013). The influence of
I., Pallasch, N., & Wozniak, M. (2021). Does social psychol- high-level beliefs on self-regulatory engagement: Evidence
ogy persist over half a century? A direct replication of Cialdini from thermal pain stimulation. Frontiers in Psychology, 4,
et al.’s (1975) classic door-in-the-face technique. Journal of Article 614.
Personality and Social Psychology, 120, e1–e7. *MacKenzie, M. J. (2015). How does free will belief influence
Giner-Sorolla, R., Embley, J., & Johnson, L. (2016). Replication guilt? [Unpublished manuscript].
of Vohs & Schooler (2008, PS, Study 1). https://osf.io/i29mh/ *MacKenzie, M. J., Vohs, K. D., & Baumeister, R. F. (2014). You
*Goodyear, K., Lee, M. R., O’Hara, M., Chernyak, S., Walter, H., didn’t have to do that: Belief in free will promotes gratitude.
Parasuraman, R., & Krueger, F. (2016). Oxytocin influences Personality and Social Psychology Bulletin, 40, 1423–1434.
intuitions about the relationship between belief in free will and Markus, H. R., & Kitayama, S. (2003). Models of agency:
moral responsibility. Social Neuroscience, 11, 88–96. Sociocultural diversity in the construction of action. In V.
Greene, J., & Cohen, J. (2004). For the law, neuroscience changes Murphy-Berman & J. J. Berman (Eds.), Cross-cultural dif-
nothing and everything. Philosophical Transactions of the ferences in perspectives on the self (pp. 18–74). University of
Royal Society B: Biological Sciences, 359, 1775–1785. Nebraska Press.
Griffin, A. (2016, April 30). Free will could all be an illusion, sci- Martin, N. D., Rigoni, D., & Vohs, K. D. (2017). Free will beliefs
entists suggest after study shows choice may just be brain trick- predict attitudes toward unethical behavior and criminal pun-
ing itself. Independent. http://www.independent.co.uk/news/ ishment: A global analysis. Proceedings of the National
science/free-will-could-all-be-an-illusion-scientists-suggest- Academy of Sciences, 114, 7325–7330.
after-study-that-shows-choice-could-just-be-a7008181.html *Melcher, W. (2016). Untitled data [Data set].
Hallett, M. (2007). Volitional control of movement: The physiol- *Melcher, W. (2019). Free will in psychological research:
ogy of free will. Clinical Neurophysiology, 118, 1179–1192. Considerations on methodic procedure and reproducibility of
*Hanisch, A. (2015). Frei handeln oder frei wollen? Experimentelle results. Leuphana University Lüneburg.
Manipulation des Glaubens an Willens- und Handlungsfreiheit *Mercier, B. (2017). Untitled data [Data set].
[Acting freely or wanting to act freely? Experimental manipu- *Mercier, B., Wiwad, D., Piff, P. K., Aknin, L. B., Robinson,
lation of the belief in free will and action.] [Unpublished bach- A. R., Shariff, A., & Vazire, S. (2020). Does Belief in Free
elor thesis]. Will Increase Support for Economic Inequality? Collabra:
*Hannikainen, I., Cabral, G., Machery, E., & Struchiner, N. (2017). Psychology, 6, 25.
A deterministic worldview promotes approval of state paternal- *Monroe, A. E. (2018). Untitled data [Data set].
ism. Journal of Experimental Social Psychology, 70, 251–259. *Monroe, A. E., Brady, G., & Malle, B. F. (2017). This isn’t the
*Harms, J., Liket, K., Protzko, J., & Schölmerich, V. (2017). Free free will worth looking for general free will beliefs do not
to help? An experiment on free will belief and altruism. PLOS influence moral judgments, agent-specific choice ascriptions
ONE, 12, Article e0173193. do. Social Psychological and Personality Science, 8, 191–199.
*Harpster, R. (2019). Untitled data [Data set]. *Monroe, A. E., & Malle, B. F. (2010a). From uncaused will to
Harris, S. (2012). Free will. Simon and Schuster. conscious choice: The need to study, not speculate about
Hedges, L. V., Tipton, E., & Johnson, M. C. (2010). Robust vari- people’s folk concept of free will. Review of Philosophy and
ance estimation in meta-regression with dependent effect size Psychology, 1, 211–224.
estimates. Research Synthesis Methods, 1, 39–65. *Monroe, A. E., & Malle, B. F. (2010b). Untitled data [Data set].
Hempel, S., Miles, J. N., Booth, M. J., Wang, Z., Morton, S. C., *Moynihan, A. B., & Igou, E. R. (2019). Untitled data [Data set].
& Shekelle, P. G. (2013). Risk of bias: A simulation study of *Moynihan, A. B., Igou, E. R., & van Tilburg, W. A. (2019). Lost
power to detect study-level moderator effects in meta-analysis. in the crowd: Conformity as escape following disbelief in free
Systematic Reviews, 2, 107. will. European Journal of Social Psychology, 49, 503–520.
Murray, D., & Nahmias, E. (2014). Explaining away incompatibil- Rigoni, D., & Brass, M. (2014). From intentions to neurons: Social
ist intuitions. Philosophy and Phenomenological Research, 88, and neural consequences of disbelieving in free will. Topoi,
434–467. 33, 5–12.
Nadelhoffer, T. (2011). The threat of shrinking agency and free will *Rigoni, D., Kühn, S., Gaudino, G., Sartori, G., & Brass, M.
disillusionism. Conscious Will and Responsibility: A Tribute to (2012). Reducing self-control by weakening belief in free will.
Benjamin Libet, 173–188. Consciousness and Cognition, 21, 1482–1490.
*Nadelhoffer, T. (2012). Untitled data [Data set]. *Rigoni, D., Kühn, S., Sartori, G., & Brass, M. (2011). Inducing
*Nadelhoffer, T., Shepard, J., Crone, D. L., Everett, J. A., Earp, B. disbelief in free will alters brain correlates of preconscious
D., & Levy, N. (2020). Does encouraging a belief in determin- motor preparation the brain minds whether we believe in free
ism increase cheating? Reconsidering the value of believing in will or not. Psychological Science, 22, 613–618.
free will. Cognition, 203, 104342. *Rigoni, D., Pourtois, G., & Brass, M. (2015). “Why should I care?”
Nadelhoffer, T., Shepard, J., Nahmias, E., Sripada, C., & Ross, L. Challenging free will attenuates neural reaction to errors. Social
T. (2014). The free will inventory: Measuring beliefs about Cognitive and Affective Neuroscience, 10, 262–268.
agency and responsibility. Consciousness and Cognition, 25, *Rigoni, D., Trager, B. M., Rys, W., Brass, M., Vallacher, R., &
27–41. Vohs, K. D. (2016). The importance of being earnest: Valuing
Nahmias, E., Morris, S. G., Nadelhoffer, T., & Turner, J. (2005). honesty attenuates anti-social consequences of disbelieving in
Surveying freedom: Folk intuitions about free will and moral free will [Unpublished manuscript].
responsibility. Philosophical Psychology, 18, 561–584. *Rigoni, D., Wilquin, H., Brass, M., & Burle, B. (2013). When
Nahmias, E., Morris, S. G., Nadelhoffer, T., & Turner, J. (2006). Is errors do not matter: Weakening belief in intentional control
incompatibilism intuitive? Philosophy and Phenomenological impairs cognitive reaction to errors. Cognition, 127, 264–269.
Research, 73, 28–53. Rodgers, M. A., & Pustejovsky, J. E. (2020). Evaluating meta-ana-
Nichols, S. (2004). The folk psychology of free will: Fits and starts. lytic methods to detect selective reporting in the presence of
Mind & Language, 19, 473–502. dependent effect sizes. Psychological Methods, 26, 141–160.
Nichols, S. (2006). Folk intuitions on free will. Journal of Cognition Rose, D., & Nichols, S. (2013). The lesson of bypassing. Review of
and Culture, 6, 57–86. Philosophy and Psychology, 4, 599–619.
Nichols, S., & Knobe, J. (2007). Moral responsibility and determin- Rotter, J. B. (1966). Generalized expectancies for internal versus
ism: The cognitive science of folk intuitions. Nous, 41, 663–685. external control of reinforcement. Psychological Monographs:
*Open Science Collaboration. (2015). Estimating the reproducibil- General and Applied, 80, 1–28.
ity of psychological science. Science, 349, aac4716. Saigle, V., Dubljević, V., & Racine, E. (2018). The impact of a
Paulhus, D. L., & Carey, J. M. (2011). The FAD–Plus: Measuring landmark neuroscience study on free will: A qualitative analy-
lay beliefs regarding free will and related constructs. Journal of sis of articles using Libet and colleagues’ methods. AJOB
Personality Assessment, 93, 96–104. Neuroscience, 9, 29–41.
Pereboom, D. (2006). Living without free will. Cambridge Sarkissian, H., Chatterjee, A., De Brigard, F., Knobe, J., Nichols,
University Press. S., & Sirker, S. (2010). Is belief in free will a cultural univer-
Petty, R. E., & Krosnick, J. A. (2014). Attitude strength: Antecedents sal? Mind & Language, 25, 346–358.
and consequences. Psychology Press. Schooler, J. W. (2010). What science tells us about free will. In R.
*Plaks, J. E., & Robinson, J. S. (2015). Construal level and free will F. Baumeister, A. R. Mele, & K. D. Vohs (Eds.), Free will and
beliefs shape perceptions of actors' proximal and distal intent. consciousness: How might they work? (pp. 191–218). Oxford
Frontiers in Psychology, 6, Article 777. University Press.
Polanin, J. R., Tanner-Smith, E. E., & Hennessy, E. A. (2016). *Schooler, J. W. (2014). Untitled data [Data set].
Estimating the difference between published and unpublished Schooler, J. W., Nadelhoffer, T., Nahmias, E., & Vohs, K. D.
effect sizes: A meta-review. Review of Educational Research, (2014). Measuring and manipulating beliefs and behaviors
86, 207–236. associated with free will. In A. R. Mele (Ed.), Surrounding
*Protzko, J., & Genschow, O. (2018). Untitled data [Data set]. free will: Philosophy, psychology, neuroscience (pp. 72–94).
Protzko, J., Ouimette, B., & Schooler, J. (2015). Believing there is Oxford University Press.
no free will corrupts intuitive cooperation. https://papers.ssrn. *Schrag, Y. (n.d.). Untitled data [Data set].
com/sol3/papers.cfm?abstract_id=2490855 *Schrag, Y., Tremea, A., Lagger, C., Ohana, N., & Mohr, C.
*Protzko, J., Ouimette, B., & Schooler, J. (2016). Believing there is (2016). Pro free will priming enhances “risk-taking” behav-
no free will corrupts intuitive cooperation. Cognition, 151, 6–9. ior in the Iowa gambling task, but not in the balloon analogue
Racine, E., Nguyen, V., Saigle, V., & Dubljevic, V. (2017). Media risk task: Two independent priming studies. PLOS ONE, 11,
portrayal of a landmark neuroscience experiment on free will. Article e0152297.
Science and Engineering Ethics, 23, 989–1007. *Seminck, N. (2019). The influence of belief in free will on imita-
Rakos, R. F., Laurene, K. R., Skala, S., & Slane, S. (2008). Belief tion of inappropriate behavior [Unpublished master thesis].
in free will: Measurement and conceptualization innovations. *Seto, E. (n.d.). Untitled data [Data set].
Behavior and Social Issues, 17, 20–40. *Seto, E. (2017). The interplay of true self-knowledge on belief in
*Rangel, U., & Keller, J. (2011). Essentialism goes social: Belief free will and moral outcomes [Unpublished dissertation].
in social determinism as a component of psychological essen- *Seto, E., & Hicks, J. A. (2016). Disassociating the agent from
tialism. Journal of Personality and Social Psychology, 100, the self: Undermining belief in free will diminishes true self-
1056–1078. knowledge. Social Psychological and Personality Science, 7,
*Rigoni, D. (n.d.). Untitled data [Data set]. 726–734.
Genschow et al. 31
*Seto, E., Kim, J., & Hicks, J. A. (2020). The illusion of time: including a software tutorial in Stata and SPSS. Research
Testing the bidirectional relationship between belief in free will Synthesis Methods, 5, 13–30.
and temporal horizons. Social Psychological and Personality Tanner-Smith, E. E., Tipton, E., & Polanin, J. R. (2016). Handling
Science, 12, 176–185. complex meta-analytic data structures using robust variance
Shariff, A. F., Greene, J. D., Karremans, J. C., Luguri, J. B., Clark, estimates: A tutorial in R. Journal of Developmental and Life-
C. J., Schooler, J. W., Baumeister, R. F., & Vohs, K. D. (2014). Course Criminology, 2, 85–112.
Free will and punishment: A mechanistic view of human nature *Thelen, A. (2018). Der Einfluss des Glaubens an einen freien
reduces retribution. Psychological Science, 25, 1563–1570. Willen auf selbstwertdienliche Verzerrungen [The influence of
Shariff, A. F., & Vohs, K. D. (2014). The world without free will. belief in free will on self-serving biases] [Unpublished master
Scientific American, 310, 76–79. thesis].
Shepard, J., & Reuter, S. (2012). Neuroscience, choice, and the free Tipton, E. (2015). Small sample adjustments for robust variance
will debate. AJOB Neuroscience, 3, 7–11. estimation with meta-regression. Psychological Methods, 20,
Shepherd, J. (2012). Free will and consciousness: Experimental 375–393.
studies. Consciousness and Cognition, 21, 915–927. Tipton, E., & Pustejovsky, J. E. (2015). Small-sample adjustments
*Siller, L. (2019). Glaube an den freien Willen: Halten sich die for tests of moderators and model fit using robust variance
Effekte einer experimentellen Manipulation über die Zeit? estimation in meta-regression. Journal of Educational and
[Belief in free will: Do the effects of experimental manipula- Behavioral Statistics, 40, 604–634.
tion hold over time?] [Unpublished master thesis]. *Trager, B. (2016). Untitled data [Data set].
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014). P-curve: Van Bavel, J. J., Mende-Siedlecki, P., Brady, W. J., & Reinero,
A key to the file-drawer. Journal of Experimental Psychology: D. A. (2016). Contextual sensitivity in scientific reproducibil-
General, 143, 534–547. ity. Proceedings of the National Academy of Sciences, 113,
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2015). Official 6454–6459.
user-guide to the P-curve. https://www.p-curve.com/guide.pdf *Van Dessel, P. (2012). Untitled data [Data set].
Slamecka, N. J., & Graf, P. (1978). The generation effect: Van Inwagen, P. (1983). An essay on free will. MIT Press.
Delineation of a phenomenon. Journal of Experimental *Vassena, E. (2019). Untitled data [Data set].
Psychology: Human Learning and Memory, 4, 592–604. Velten, E. (1968). A laboratory task for induction of mood states.
Smilansky, S. (2000). Free will and illusion. Oxford University Behaviour Research and Therapy, 6, 473–482.
Press. Viechtbauer, W. (2010). Conducting meta-analyses in R with the
Smilansky, S. (2002). Free will, fundamental dualism, and the cen- Metafor package. Journal of Statistical Software, 36, 1–48.
trality of illusion. In R. Kane (Ed.), The Oxford handbook of Viney, W., Waldman, D. A., & Barchilon, J. (1982). Attitudes
free will (pp. 489–505). Oxford University Press. toward punishment in relation to beliefs in free will and deter-
*Snoeck, A. (2019). The effect of disbelief in free will on prejudice minism. Human Relations, 35, 939–949.
toward Arabic people [Unpublished master thesis]. *Vohs, K. D., & Schooler, J. W. (2008). The value of believing in
Soon, C. S., Brass, M., Heinze, H.-J., & Haynes, J.-D. (2008). free will encouraging a belief in determinism increases cheat-
Unconscious determinants of free decisions in the human ing. Psychological Science, 19, 49–54.
brain. Nature Neuroscience, 11, 543–545. *Vonasch, A. J., Clark, C. J., Lau, S., Vohs, K. D., & Baumeister,
Stanley, T. D. (2017). Limitations of PET-PEESE and other meta- R. F. (2017). Ordinary people associate addiction with loss of
analysis methods. Social Psychological and Personality free will. Addictive Behaviors Reports, 5, 56–66.
Science, 8, 581–591. Waldman, D. A., Viney, W., Bell, P. A., Bennett, J. B., & Hess,
Stanley, T. D., & Doucouliagos, H. (2014). Meta-regression S. (1983). Internal and external locus of control in relation to
approximations to reduce publication selection bias. Research beliefs in free will and determinism. Psychological Reports,
Synthesis Methods, 5, 60–78. 53, 631–634.
Sterne, J. A. C., & Egger, M. (2005). Regression methods to detect Webb, T. L., & Sheeran, P. (2006). Does changing behavioral
publication and other bias in meta-analysis. In H. R. Rothstein, intentions engender behavior change? A meta-analysis of the
A. J. Sutton, & M. Borenstein (Eds.), Publication bias in experimental evidence. Psychological Bulletin, 132, 249–268.
meta-analysis: Prevention, assessment and adjustments (pp. Wegner, D. (2002). The illusion of free will. MIT Press.
99–110). Wiley. Wisniewski, D., Deutschländer, R., & Haynes, J.-D. (2019). Free will
*Stillman, T. F., & Baumeister, R. F. (2010). Guilty, free, and wise: beliefs are better predicted by dualism than determinism beliefs
Determinism and psychopathy diminish learning from nega- across different cultures. PLOS ONE, 14, Article e0221617.
tive emotions. Journal of Experimental Social Psychology, 46, Wolfe, T. (1997). Sorry, but your soul just died. Independent.
951–960. https://www.independent.co.uk/arts-entertainment/sorry-but-
Stillman, T. F., Baumeister, R. F., Vohs, K. D., Lambert, N. M., your-soul-just-died-1276509.html
Fincham, F. D., & Brewer, L. E. (2010). Personal philosophy *Zhao, X., Liu, L., Zhang, X.-x., Shi, J.-x., & Huang, Z.-w. (2014).
and personnel achievement: Belief in free will predicts bet- The effect of belief in free will on prejudice. PLOS ONE, 9,
ter job performance. Social Psychological and Personality Article e91572.
Science, 1, 43–50. *Zwaan, R. A. (n.d.). Untitled data [Data set].
Stroebe, W., & Strack, F. (2014). The alleged crisis and the illusion *Zwaan, R. A. (2014). The value of believing in free will: A rep-
of exact replication. Perspectives on Psychological Science, 9, lication Attempt. Zeistgeist: Psychological Experimentation,
59–71. Cognition, Language, and Academia. https://rolfzwaan.
Tanner-Smith, E. E., & Tipton, E. (2014). Robust variance esti- blogspot.com/2013/03/the-value-of-believing-in-free-will.
mation with dependent effect sizes: Practical considerations html

Manipulating Belief in Free Will and Its Downstream Consequences: A Meta-Analysis

Uploaded by

Copyright:

Available Formats

Manipulating Belief in Free Will and Its Downstream Consequences: A Meta-Analysis

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Manipulating Belief in Free Will and Its Downstream Consequences: A Meta-Analysis

Uploaded by

Copyright:

Available Formats

1087527

Personality and Social Psychology Review

Manipulating Belief in Free Will and

Meta-Analysis DOI: 10.1177/10888683221087527

Oliver Genschow1* , Emiel Cracco2* , Jana Schneider1 ,

Secondary moderator analyses. In addition to the above-men-

Included in antisocial Included in p-curve

Figure 1. Flowchart of the screening process.

Research Question 2: Do free will belief manipulations have

Publication bias. For both research questions, we tested for

Table 2. Free Will Moderator Statistics.

Moderator g/β SE 95% CI m k F/t (df) p

Table 3. Determinism Moderator Statistics.

Moderator g/β SE 95% CI m k F/t (df) p

Table 4. Weighted Correlations Among Moderator Variables.

Table 5. FWI and FAD Statistics.

Moderator g/β SE 95% CI m k F/t (df) p

Figure 5. Forest plot of the effect of the manipulation on cheating.

Figure 6. Forest plot of the effect of the manipulation on conformity.

Figure 7. Forest plot of the effect of the manipulation on punishment.

You might also like