FWB MetaAnalysis

Manipulating belief in free will and its downstream consequences:
A meta-analysis
Oliver Genschow* (University of Cologne, Germany)
Emiel Cracco* (Ghent University, Belgium)
Jana Schneider (University of Cologne, Germany)
John Protzko (Central Connecticut State University, USA)
David Wisniewski (Ghent University, Belgium)
Marcel Brass (Humboldt University Berlin, Germany)
Jonathan W. Schooler (University of California Santa Barbary, USA)
*shared first-authorship
accepted manuscript at Personality and Social Psychology Review before copyediting
Author Note: Correspondence concerning this article should be addressed to Oliver Genschow,
University of Cologne, Social Cognition Center Cologne, Richard-Strauss Str. 2, 50931 Cologne,
Germany, E-Mail: oliver.genschow@uni-koeln.de
Coding manual, data, and analysis scripts are open accessible at the Open Science Framework (OSF;
https://bit.ly/2L69prl)
Author contributions: Oliver Genschow developed the research idea; all co-authors gave feedback on
the basic research idea; Oliver Genschow conducted the literature research; Oliver Genschow, Emiel
Cracco, and John Protzko collected unpublished data; Oliver Genschow, Emiel Cracco, and Jana
Schneider screened and coded the data; Emiel Cracco conducted the analyses; Oliver Genschow &
Emiel Cracco drafted the paper, all authors gave feedback on the drafted manuscript.
META-ANALYSIS FREE WILL BELIEF 2
Abstract
After some scientists and popular media put forward the idea that free will is an illusion, the
question arose as to what would happen if people stopped believing in free will.
Psychological research has investigated this question by testing the consequences of
experimentally weakening people’s free will belief. The results of these investigations have
been mixed, with successful experiments and unsuccessful replications. This raises two
fundamental questions: Can free will beliefs be manipulated, and do such manipulations have
downstream consequences? In a meta-analysis including 145 experiments (95 unpublished),
we show that exposing individuals to anti-free will manipulations decreases belief in free will
and increases belief in determinism. However, we could not find evidence for downstream
consequences. Our findings have important theoretical implications for research on free will
beliefs and contribute to the discussion of whether reducing people’s belief in free will has
societal consequences.
Keywords: free will; determinism; belief; meta-analysis; morality; cheating; social behavior;
punishment
From morality to politics, public policy, intimate relationships, and punishing behavior—
“most of what is distinctly human about our life depends upon our viewing one another as
autonomous persons, capable of free choice” (Harris, 2012; p. 1). Thus, unsurprisingly, most
people believe that they have free will (Baumeister et al., 2009; Nahmias et al., 2005).
Whether free will actually exists, however, has been a longstanding philosophical debate
(e.g., Dennett, 2015; Van Inwagen, 1983). This debate has reached an extremely high level of
sophistication outlining different theoretical positions that span free will skepticism to
complete libertarianism (for an overview see Dennett, 2015). However, these philosophical
arguments have rarely left academic circles and therefore have had limited impact outside
academia.
In the last decades, cognitive neuroscientists and psychologists entered the debate by
claiming humans’ perception of free will is nothing more than an illusion (e.g., Crick, 1994;
Harris, 2012; Wegner, 2002) and simply arises from unconscious brain activity (Hallett, 2007;
Libet et al., 1983; Soon et al., 2008). A seminal study supporting this view is the study by
Libet and colleagues (1983), who measured neural activity while participants made voluntary
finger movements. After each movement, participants indicated on a clock the time at which
they perceived their first urge to initiate a movement. Libet and colleagues found an increase
in neural activity several hundred milliseconds before participants reported being aware of
this urge. These and similar findings (Libet et al., 1983; Libet et al., 1993) have often been
used as an argument for the claim that free will does not exist.
Despite criticisms of this argument (Brass et al., 2019; Saigle et al., 2018), anti-free
will viewpoints have become in vogue not only in academia (e.g., Greene & Cohen, 2004),
but also, and perhaps even more so, in popular media (e.g., Chivers 2010; Griffin, 2016;
Racine et al., 2017; Wolfe, 1997). This raises fundamental questions of whether reading anti-
free will viewpoints pushes people towards a deterministic world view and lowers their belief
in free will. Moreover, the question arises as to what impact this has on society.
Some philosophers argue that undermining people’s belief in free will saps the basis
for moral behavior and would therefore have catastrophic consequences (e.g., Smilansky,
2000, 2002). In contrast, other philosophers argue that disbelief in free will might instead
have positive effects, as it would cause people to abandon retribution-based morality and
illusory beliefs in a just world (e.g., Caruso, 2014; Greene & Cohen, 2004; Nadelhoffer, 2011;
Pereboom, 2006). Research in social and cognitive psychology as well as in neuroscience has
empirically tested these theories by experimentally manipulating belief in free will (for an
overview, see Ewusi-Boisvert & Racine, 2018). These studies provided evidence for the idea
that free will beliefs indeed have societal consequences. There have also been a number of
failures to replicate some of these findings, however. As a result, it remains unclear whether
exposing people to anti-free will viewpoints has behavioral and societal consequences. In this
article, we address this controversy. First, we review the literature on belief in free will, its
experimental manipulations, and its downstream consequences. Then, we test meta-
analytically (1) whether beliefs related to free will can be manipulated and (2) whether these
manipulations have downstream consequences.
Belief in Free Will and Determinism
Philosophical definitions regarding the concept of free will are rather diverse and
complex (Carey & Paulhus, 2013). Because of this, researchers often assess laypeople’s
concepts of free will and to which degree laypeople believe in these concepts (e.g., Nichols,
2006). These analyses indicate that laypeople’s free will beliefs are metacognitive judgments
about the extent to which individuals intentionally guide their thoughts and actions (Frith,
2012). Specifically, belief in free will reflects the belief that people are responsible for their
actions (Carey & Paulhus, 2013), because they can decide and control their own behavior
(Paulhus & Carey, 2011).
Although conceptually related to internal locus of control (Rotter, 1966) and self-
efficacy (Bandura, 1977), there are important differences between these constructs that set
free will beliefs apart. Internal locus of control shares the emphasis on internal causal
attribution, but represents a personality dimension, whereas belief in free will reflects an
attitudinal orientation (Waldman et al., 1983). Self-efficacy differs from belief in free will in
that it reflects metacognitive judgments about one’s specific skill or ability (e.g., “Can I
execute this successfully?”). In contrast, belief in free will reflects a much broader belief
about choice and freedom (e.g., “Do I have a choice? Can I freely choose to do otherwise?”).
Related to the belief in free will is the belief in determinism. Previous research has
defined belief in determinism in different ways. On a general level, determinism is typically
interpreted as: given the past and the laws of nature, there is only one possible future at any
moment in time (e.g., Van Inwagen, 1983). Some scholars differentiate between fatalistic
determinism, which captures people’s belief in ‘destiny’ or ‘fate’, and scientific determinism,
which measures belief in genetic determinism (e.g., Paulhus & Carey, 2011) and reflects the
belief that genes are the sole or predominant basis of person characteristics (Keller, 2005).
Irrespective of how determinism is defined, psychologists and philosophers debate
whether free will and determinism are the endpoints of the same continuum or separate
constructs. For instance, incompatibilists see free will and determinism as mutually exclusive.
Such a view suggests that the more a person believes in free will, the less they believe in
determinism (Rakos et al., 2008; Viney et al., 1982). In contrast, compatibilists see free will
and determinism as independent constructs. Based on a compatibilistic view, people believe
that if determinism were true, people could nevertheless be free. Previous research has shown
that belief in free will is not correlated with belief in genetic determinism (Nadelhoffer et al.,
2014) and that a compatibilistic view is more widespread in the general public than
philosophers and psychologists may have traditionally assumed (e.g., Monroe & Malle,
2010a; Murray & Nahmias, 2014; Nadelhoffer et al., 2014; Nahmias et al., 2006; Nichols,
2004, 2006; Nichols & Knobe, 2007; Rose & Nichols, 2013; Shepard & Reuter, 2012;
Shepherd, 2012).
Research on Manipulating Belief in Free Will
To test the consequences of (dis)believing in free will, researchers have developed
various approaches to experimentally manipulate people’s belief in free will. The first
investigations in this respect were carried out by Vohs and Schooler (2008). In one
experiment, participants either read a passage from Francis Crick’s (1994) book “the
Astonishing Hypothesis”, arguing against the plausibility of free will or a passage from the
same book that did not mention free will. In another experiment, the authors followed a
Velten-like technique (Velten, 1968), in which participants either read and pondered anti-free
will or pro-free will statements. In both experiments, participants presented with anti-free will
viewpoints reported lower beliefs in free will and were more willing to cheat on a test than
control participants.
These findings inspired researchers all around the world to start investigating the
consequences of experimentally reducing belief in free will by applying and adopting the
manipulations introduced by Vohs and Schooler (2008). This research suggests that
weakening belief in free will increases anti-social behavior, such as prejudice (Zhao et al.,
2014) or aggressiveness towards others (Baumeister et al., 2009), and decreases pro-social
behavior, such as helping (Baumeister et al., 2009) or cooperation (Protzko et al., 2015). At
the same time, however, exposing people to anti-free will viewpoints can also lead to reduced
retributive punishment (Shariff et al., 2014).
In addition, anti-free will manipulations have been found to increase conformity
(Alquist et al., 2013) and feelings of alienation (Seto & Hicks, 2016), and to decrease causal
attributions of other people’s actions (Genschow et al., 2017a), perceived meaningfulness of
life (Crescioni et al., 2016; Moynihan et al., 2019), perceived gratitude (MacKenzie et al.,
2014), counterfactual thinking (Alquist et al., 2015), and risk taking behavior (Schrag et al.,
2016). Finally, a last line of research suggests that experimentally reducing people’s belief in
free will influences neurocognitive processes such as intentional action preparation (Rigoni et
al., 2011), deliberate motor inhibition (Lynn et al., 2013; Rigoni et al., 2012), and the
processing of performance errors (Rigoni et al., 2015; Rigoni et al., 2013).
In sum, there is a large body of research suggesting that manipulating belief in free
will affects societally relevant behaviors such as cheating (Vohs & Schooler, 2008),
retributive punishment (Shariff & Vohs, 2014), and anti-social behavior (Baumeister et al.,
2009; Zhao et al., 2014), as well as basal neurocognitive mechanisms (Rigoni & Brass, 2014).
Based on this research, it has been argued that care should be used in the manner in which
anti-free will and deterministic viewpoints are presented to society, because it may change the
way people interact with each other. For example, some scholars suggested that encountering
anti-free will viewpoints in the popular press may “move judges and jurors toward being less
punitive and less retributive in general” (p. 1569, Shariff et al., 2014) or “provide the ultimate
excuse to behave as one likes” (p. 54, Vohs & Schooler, 2008).
Failed Replications
Despite the mounting evidence that manipulating belief in free will influences
behavior, a number of studies have reported difficulties in replicating some key results (Crone
& Levy, 2019; Eben et al., 2020; Genschow et al., 2020; Giner-Sorolla et al., 2016; Monroe et
al., 2017; Nadelhoffer et al., 2020; Open Science Collaboration, 2015; Schooler et al., 2014;
Shariff & Vohs, 2014; Zwaan, 2014). For example, Monroe et al. (2017) found no effect of
diminishing participants’ belief in free will on moral behavior, judgments of blame, or
punishment decisions. Similarly, Nadelhoffer et al. (2020) found that manipulating free will
beliefs in a robust way is more difficult than implied by previous work, and that the proposed
link with immoral behavior, such as cheating, for instance, might be similarly tenuous (for
similar findings, see Crone & Levy, 2019; Giner-Sorolla et al., 2016; Zwaan, 2014).
Although these failed replications call into question the societal relevance of belief in
free will, it is not yet clear what caused them. Before we can draw conclusions about the role
of free will beliefs in society, it is imperative to understand why some findings failed to
replicate as well as which mechanisms underlie free will belief manipulations. In principle,
three explanations could account for the replication failures reported in the literature. First, it
could be that the failed replications are false negatives. That is, they were not able to detect an
effect that is actually real. Second, it might be that free will beliefs cannot be manipulated,
and that successful studies in the literature are therefore false positives. Third, it could be that
manipulations of belief in free will successfully affect free will beliefs, but that these
manipulations are not causally related to other behaviors, and thus have no downstream
consequences. In the present article, we investigate these explanations by analyzing all
available evidence in a meta-analysis.
Previous Reviews and Meta-Analyses
To the best of our knowledge, this is the first comprehensive meta-analysis on the
effectiveness of manipulations related to free will beliefs. There are, however, two articles
related to our analysis. First, Ewusi-Boisvert and Racine (2018) published a qualitative review
of the literature on free will belief manipulations. Overall, the authors report a substantial
amount of methodological diversity and a lack of replication studies in the published
literature. Moreover, the review suggests that the studied samples are heavily constituted of
women, students, and younger participants, and contain little information about the
representation of ethnic minorities. This review did not allow the accurate estimation of the
effect size of free will belief manipulations and its downstream consequences, however, as the
researchers provided a descriptive review of the literature and did not include unpublished
data.
Second, Genschow and colleagues (2017a) tested the effectiveness of one specific
manipulation (i.e., the Crick manipulation) in a mini meta-analysis involving 9 published and
unpublished experiments that were conducted in their own research group (N = 625
participants). Overall, the authors found that participants who read the anti-free will text had a
significantly lower belief in free will than participants who read the control text. However,
this analysis did not allow strong conclusions regarding the general effectiveness of free will
belief manipulations, because only one manipulation type, a small number of experiments,
one measure, and data from only one research team were investigated. In addition, this
analysis did not explore potential reasons for why the manipulation sometimes works and
sometimes fails, nor did it investigate whether the manipulation has any downstream
consequences.
The Present Meta-Analysis
The present meta-analysis aims to build on and considerably extend previous work by
including both published and unpublished evidence and addressing two main research
questions: (1) Can belief in free will be experimentally manipulated, and (2) does this have
any downstream consequences?
Research Question 1: Can Belief in Free Will Be Manipulated?
In the first part of the meta-analysis, we investigate whether it is possible to
experimentally manipulate beliefs related to free will and explore the conditions under which
the manipulations are effective. To this end, we investigated different moderators.
Beliefs
The two most often used measures to test the effectiveness of free will belief
manipulations are belief in free will scales and belief in determinism scales (Ewusi-Boisvert
& Racine, 2018). It is often assumed (without giving explicit explanations) that anti-free will
manipulations should not only decrease free will beliefs but should also increase beliefs in
determinism (for an overview, see Ewusi-Boisvert & Racine, 2018). This assumption,
however, has never been systematically tested. In the present meta-analysis, we tested
whether the applied manipulations decrease the belief in free will, increase the belief in
determinism, or both.
Scales
Several validated scales have been developed to measure belief in free will and
determinism. The most frequently used scales are the Free Will and Determinism Scale
(FWD; Rakos et al., 2008), the Free Will and Determinism scale (FAD; Paulhus & Carey,
2011), and the Free Will Inventory (FWI; Nadelhoffer et al., 2014). In addition, researchers
have sometimes also used self-made rating scales (e.g., Baumeister et al., 2009; Moynihan et
al., 2019). Less frequently used scales measuring beliefs related to free will and determinism
include the Belief in Genetic Determinism scale (BGD; Keller, 2005), the Belief in Social
Determinism scale (BSD; Rangel & Keller, 2011), and the Wif scale (Melcher, 2019).
However, while a wide range of scales has been used, it remains unknown whether some
scales are more sensitive in picking up effects of the manipulation than others. To address this
question, the present meta-analysis tests whether the effects of free will belief manipulations
differ across scales. We focus on the FWI and the FAD, as these are the only two scales that
tease apart belief in free will and belief in determinism. Moreover, they are also the two most
commonly used scales and the only scales for which sufficient data is available to reliably
compare them to each other.
In addition to the sensitivity of different scales, little is known about whether free will
belief manipulations specifically affect beliefs in free will and determinism or also influence
other, related beliefs. Therefore, we investigated if the influence of free will belief
manipulations extends to other beliefs that are related to free will and are part of free will
questionnaires, such as belief in dualism, fatalistic determinism, and unpredictability.
Type of Manipulation
Not only the scale but also the type of manipulation differs across studies.
Specifically, four types of manipulations can be distinguished. The first method is to let
participants read a text—either a control text or a text arguing against the plausibility of free
will. Frequently used anti-free will texts include a passage of Francis Crick’s (1994) book
“The Astonishing Hypothesis” (e.g., MacKenzie et al., 2014; Rigoni et al., 2011; Shariff et
al., 2014; Vohs & Schooler, 2008), or articles featuring neuroscientific experiments (e.g.,
Harms et al., 2017; Protzko et al., 2016; Shariff et al., 2014), among others. The second
method is to present participants with several statements using a Velten-like technique
(Velten, 1968). That is, participants are either presented with a variety of anti-free will or
control statements (e.g., Rigoni et al., 2012; Stillman et al., 2010; Vohs & Schooler, 2008).
The third method uses a combination of text and statements (e.g., Seto & Hicks, 2016).
Finally, the fourth method is to let participants watch videos related to (anti-)free will
viewpoints (e.g., Highhouse & Rada, 2015). To reliably manipulate belief in free will, it is
important to know which of these manipulations are most effective. Therefore, we directly
compared them in the present meta-analysis.
Participant Involvement
Whether the manipulation is effective may, however, not only depend on the type of
manipulation but also on participants’ involvement in the manipulation. While some
researchers merely presented participants with anti-free will viewpoints (e.g., Baumeister et
al., 2009; Goodyear et al., 2016; Rigoni et al., 2012; Shariff et al., 2014; Stillman et al., 2010),
others engaged participants more strongly by letting them, for instance, summarize or rewrite
the presented messages (Harms et al., 2017; Highhouse & Rada, 2015; Moynihan et al., 2019;
Rigoni et al., 2011; Seto & Hicks, 2016; Vonasch et al., 2017). Yet other researchers told
participants they had to attend the presented messages carefully as they had to summarize
them at the end of the experiment (e.g., Genschow et al., 2017a). An interesting question is
how deeply participants have to process the provided information for the manipulation to
succeed. To answer this question, we investigated if the effectiveness of the manipulation
depends on whether participants have to engage with the task (e.g., by summarizing or
rewriting presented messages) before free will beliefs are measured.

Baseline Condition
Another factor that may influence the manipulations’ effectiveness is the baseline
condition. Past research has shown that most people believe in free will (Baumeister et al.,
2009; Nahmias et al., 2005). As a result, a tacit assumption in the literature is that beliefs in
free will can only be diminished, but not increased. Yet, some experiments have nevertheless
used not only anti-free will and neutral messages, but also pro-free will messages (e.g.,
Baumeister et al., 2009; Clark et al., 2017; Highhouse & Rada, 2015; Moynihan et al., 2019;
Schrag et al., 2016; Seto & Hicks, 2016; Vohs & Schooler, 2008). This allowed us to test the
effectiveness of anti-free will messages both with respect to control messages and with
respect to pro-free will messages and thereby allowed us to investigate whether belief in free
will can also be experimentally increased, in addition to decreased.
Measurement Moment
A final moderator that may influence the effectiveness of the manipulation is the
moment at which free will beliefs are measured. While some experiments measured
participants’ beliefs directly after the manipulation (Baumeister et al., 2009; Monroe et al.,
2017; Seto & Hicks, 2016; Shariff et al., 2014; Vohs & Schooler, 2008; Vonasch et al., 2017),
others measured them after participants performed a secondary task, often included to explore
potential downstream consequences (Clark et al., 2017; Genschow et al., 2017a; Harms et al.,
2017; Highhouse & Rada, 2015; Protzko et al., 2016; Rigoni et al., 2012; Schrag et al., 2016).
Measuring beliefs at the end of the experiment assumes the manipulation has a relatively
long-lasting effect. This may, however, not necessarily be the case. For instance, it could also
be that free will belief manipulations do not fully change people’s beliefs but rather prime an
anti-free will mindset. Such priming effects may not be as long-lasting, and measuring free
will beliefs at the end of the experiment may therefore make it more difficult to confirm that
the manipulation worked. To test this hypothesis, we investigated if the effect of the
manipulation differs depending on whether beliefs were measured immediately after the
manipulation or at the end of the experiment (i.e., after completing another task).
Secondary Moderator Analyses
In addition to the above-mentioned moderators, we also considered the influence of
participant age, sex, continent in which the study was conducted (US vs. Europe), test
location (i.e., online vs. laboratory), and sample type (students vs. non-students) as secondary
moderators.
Research Question 2: Do Free Will Belief Manipulations Have Downstream
Consequences?
In the second part of the meta-analysis, we investigate whether experimentally
reducing free will beliefs influences attitudes, behavior, and cognition. An interesting
characteristic of the literature is that a large variety of dependent variables have been studied
(for an overview, see Table 1). While this illustrates the breadth of the field, it also makes it
difficult to quantify the evidence for downstream consequences, because for some dependent
variables only one or a few experiments exist. In this respect, one solution could be to group
the variables into broad categories such as “behavior” or “attitudes”. However, this would
involve pooling together studies with vastly different dependent variables under the same
denominator and would therefore run the risk of making the meta-analysis uninterpretable, a
problem that is well-known as the “apples and oranges” critique of meta-analysis (Carpenter,
2020). To deal with this issue, we proceeded in three steps.
In a first step, we ran a p-curve analysis across all dependent variables. Whereas the
aim of estimating a population effect size makes a meta-analysis unsuited to evaluate diverse
sets of dependent variables, this is not the case for p-curve. Rather than estimating a
population effect size, p-curve investigates if a set of statistically significant findings contains
evidential value by testing if the distribution of p-values is consistent with the existence of a
true effect (Simonsohn et al., 2014). Importantly, if confirmed, this does not mean that all
included studies show a true effect. Instead, it merely implies that at least one study does
(Simonsohn et al., 2014). As such, p-curve can be applied to diverse findings, as long as they
form a meaningful whole (Simonsohn et al., 2015).
In a second step, we ran meta-analyses on internally coherent sets of dependent
variables. Upon reviewing the literature, one clear set arose—namely anti-social vs. pro-social
behavior (for an overview, see Table 1). Hence, we pooled together the studies in this set and
subjected them to a meta-analysis testing whether manipulating belief in free will influences
social behavior. However, pro- and anti-social behavior is still a relatively broad and
unspecific dependent variable. Therefore, in a third and final step, we also ran meta-analyses
on three specific dependent variables that have been used in at least five experiments:
conformity, punishment, and cheating.

Table 1
Overview of All Published Experiments Included in the P-Curve Analysis and/or the Meta-Analyses Assessing
the Influence of Free Will Belief Manipulations on Downstream Consequences
Included in
Included in p-
Experiment DV anti-social
curve analysis
meta-analysis
Vohs & Schooler (2008); Exp. 1 cheating yes yes
Vohs & Schooler (2008); Exp. 2 cheating yes yes
Baumeister et al. (2009); Exp. 1 helping yes yes
Baumeister et al. (2009); Exp. 3 aggression yes yes
Stillman & Baumeister (2010); Exp. 1 learning no yes
Stillman & Baumeister (2010); Exp. 2 learning no yes
Rangel & Keller (2011); Exp. 6 in-group preference no yes
Rigoni et al. (2011); Exp. 1 preconscious motor preparation no yes
Alquist et al. (2013); Exp. 2 conformity no yes
Alquist et al. (2013); Exp. 3 conformity no yes
Evans (2013); Exp. 1 willingness to help yes yes
Rigoni et al. (2013); Exp. 1 action monitoring no yes
MacKenzie et al. (2014); Exp. 2 gratitude no yes
Shariff et al. (2014); Exp. 2 punishment no yes
Shariff et al. (2014); Exp. 3 punishment no yes
Zhao et al. (2014); Exp. 2 prejudice yes yes
Zhao et al. (2014); Exp. 3 prejudice yes yes
Open Science Collaboration (2015); Exp. 1 cheating yes no
Alquist et al. (2015); Exp. 1 counterfactual thinking no yes
Alquist et al. (2015); Exp. 2 counterfactual thinking and intention
no yes
perception
Plaks and Robinson (2015); Exp. 4 moral judgments no yes
Rigoni et al. (2015); Exp. 1 error detection no yes
Crescioni et al. (2016); Exp. 3 meaningfulness of life no yes
Crescioni et al. (2016); Exp. 4 meaningfulness of life no yes
Goodyear et al. (2016); Exp. 1 assigned moral responsibility no yes
Protzko et al. (2016); Exp. 1 cooperation yes yes
Schrag et al. (2016); Exp. 1 risk behavior no yes
Seto and Hicks (2016); Exp. 1 self-alienation no yes
Seto and Hicks (2016); Exp. 2 reported authenticity no yes
Caspar et al. (2017); Exp. 1 immoral behavior yes yes
Clark et al. (2017); Exp. 4 punitive distress no yes
Genschow et al. (2017a); Exp. 3a correspondence bias no yes
Genschow et al. (2017a); Exp. 3b correspondence bias no yes
Hannikainen et al. (2017); Exp. 4 preference for autonomy vs. welfare no yes
Hannikainen et al. (2017); Exp. 5 approval of paternalistic policies no yes
Monroe et al. (2017); Exp. 1 punishment, cheating, moral
yes no
judgments
Monroe et al. (2017); Exp. 3 punishment and moral judgments no no
Vonasch et al. (2017); Exp. 6 addiction and self-control no yes
Ching & Xu (2018); Exp. 1 prejudice yes yes
Moynihan et al. (2019); Exp. 1 meaninglessness no yes
Nadelhoffer et al. (2020); Exp. 3 cheating yes no
Nadelhoffer et al. (2020); Exp. 4 cheating yes no
Genschow & Vehlow (2019); Exp. 1 victim blaming yes yes
Note. Nadelhoffer er al. (2019) and Genschow & Vehlow (2019) were included in the meta-analysis as
unpublished studies but were published later as Nadelhoffer et al. (2020) and Genschow and Vehlow (2021)
after data-analysis had already been completed. Experiments were only included in the p-curve analysis if the
effect of the manipulation on the DV was significant. All experiments that were not included in the p-curve
analysis (included: “no”) are experiments without a significant effect on the DV. Articles were included in the
meta-analyses on anti-social behavior, conformity, punishment, and cheating irrespective of whether the effect
was significant or not.
Method
Search Strategy
The literature search for published articles was initiated in July 2018 and includes
studies published between January 2008—based on the publication date of the first study that
included ab experimental belief in free will manipulation (Vohs & Schooler, 2008)— and July
2019. To collect published studies, we entered the following search terms in Web of Science,
PubMed, and PsycINFO: (“Free will” AND “belie*”) OR (“Free will” AND “manipulat*”)
OR (“Free will” AND “experiment*”) OR (“Free will” AND “group”) OR (“Free will” AND
“induc*”) OR (“Free will” AND “reduc*”) OR (“Free will” AND “threat*”) OR (“Free will”
AND “undermin*”) OR (“Free will” AND “weak*”) OR (“Determinis*” AND “belie*”) OR
(“Determinis*” AND “manipulat*”) OR (“Determinis*” AND “experiment*”) OR
(“Determinis*” AND “group”) OR (“Determinis*” AND “induc*”) OR (“Determinis*” AND
“encourag*”) OR (“Determinis*” AND “increas*”) OR (“Determinis*” AND “enhanc*”).
In addition to this Boolean search, we also looked for studies that cited the Vohs and
Schooler (2008) paper. Furthermore, we included studies that were cited in the recent review
paper on belief in free will by Ewusi-Boisvert and Racine (2018). Third, unpublished data
were collected by sending requests to authors who had previously published articles using
free will belief manipulations. Fourth, we sent around requests for unpublished data via
different national and international mailing lists, including the mailing list of the German
Psychology Association (DGPs), the mailing list of the Belgian Association for Psychological
Science (BAPS), and the mailing list of the European Society for Cognitive and Affective
Neuroscience (ESCAN). Finally, we asked for unpublished data via Twitter and different
open fora of the Society of Personality and Social Psychology (SPSP). The search for
unpublished data was terminated on August 15, 2019.

Screening Process
We screened titles and abstracts from 3739 records obtained from the literature search.
In addition, 110 unpublished experiments were included. Studies that were clearly not eligible
based on the criteria described below were excluded (i.e., n = 3570). The remaining studies
were then evaluated in more detail by screening the full-text articles. 279 full-text articles
were assessed for eligibility. This procedure resulted in a database of 84 eligible studies
containing 145 experiments (50 published; 95 unpublished). The criteria used to screen the
search results are described below. The numbers of excluded and included studies can be seen
in the PRISMA flow chart in Figure 1.
1. The studied population had to involve healthy adults.
2. The study had to include an experimental manipulation with the aim of reducing or
increasing belief in free will or belief in determinism.
3. For the research question investigating the influence of free will belief manipulations
on free will beliefs, but not for the research question investigating downstream
consequences, the study had to contain a manipulation check administered after the
manipulation—that is a measure of belief in free will, determinism, or both. We
focused on belief in free will and belief in determinism because these are the two most
commonly measured beliefs. Importantly, we only included measures that considered
free will and determinism as separate constructs and not measures that considered
them as two opposite endpoints of the same scale because the latter approach does not
allow us to distinguish between both beliefs1.
4. Sufficient statistical information had to be available to calculate the necessary effect
sizes. The required information had to be either reported in the paper or obtained from
the authors by e-mail.
1
Based on this approach we had to exclude only two published and two unpublished experiments.
Figure 1
Flowchart of the Screening Process
Note. The number of included or excluded studies (n) and experiments (m) are denoted. See
text for additional detail.
Coding and Reliability
Research Question 1: Can Belief in Free Will be Manipulated?
The eligible studies were first coded by the first author. To evaluate the reliability of
the coding, the third author was trained to use the coding manual and subsequently recoded 17
randomly selected published studies containing 27 experiments in total. To further identify
errors, the initial coding was also checked by the third author. Disagreements and
inconsistencies that arose during coding were resolved by the first three authors via
discussion. Coding reliability was assessed with intraclass correlation coefficients (ICC) for
continuous variables and with kappa coefficients for categorical variables. The reliability for
continuous variables was high (all ICC ≥ 0.99). Likewise, the average reliability for
categorical variables was high (κ = 0.96) and varied from 0.74 to 1.00. The coding manual is
openly accessible at the Open Science Framework (OSF; https://bit.ly/2L69prl). The
following variables were coded:
Publication Status. We coded whether an experiment was published (m = 51) or
unpublished (m = 95). An experiment was considered published when it appeared in an
academic journal or book. Dissertations, preprints (if not published elsewhere), conference
posters, and raw data were all considered unpublished. Experiments that were initially
unpublished but then later published before the meta-analysis was completed remained coded
as unpublished experiments to account for potential differences between the original and
published results (e.g., additional data collection, different exclusion criteria, different
analytical choices).
Beliefs. We coded whether belief in free will or belief in determinism was measured.
Effect sizes obtained with instruments measuring belief in free will and belief in determinism
as opposite ends of a scale (e.g., the FWD) were not included. Effect sizes were coded as
measuring belief in free will when they were obtained with the free will subscales of the FWI,
the FAD-Plus, or the FAD. In addition, we also included self-made scales and individual
items designed to measure belief or disbelief in free will. The disbelief scales were reverse
coded.
Effect sizes were coded as measuring belief in determinism when they were obtained
with the following scales: belief in determinism of the FWI (Nadelhoffer et al., 2014),
biological determinism of the WiF (Melcher, 2019), genetic determinism of the BGD (Keller,
2005), social determinism of the BSD (Rangel & Keller, 2011), and scientific determinism of
the FAD-Plus or the FAD scale (Paulhus & Carey, 2011). In addition, as for free will beliefs,
we also included self-made scales and individual items designed to measure either belief or
disbelief in determinism. The disbelief items were reverse coded. We did not include the
FAD/FAD-Plus’ fatalistic determinism subscale, as it does not capture determinism as
construed by the other scales (i.e., the belief that future events are completely determined by
prior events) but rather measures belief in ‘destiny’ or ‘fate’.
Scales. To compare the sensitivity of different scales, we coded which of the
following scales was used to measure belief in free will and/or determinism: the FAD/FAD-
Plus (Paulhus & Carey, 2011), the WiF (Melcher, 2019), the BGD (Keller, 2005), the BSD
(Rangel & Keller, 2011), or self-made scales. The coding indicated that not enough
experiments (i.e., m < 3) had used the WiF, BGD, and BSD to draw reliable conclusions.
Furthermore, the self-made scales did not form an internally coherent cluster. Therefore, we
restricted the scales analysis to the FWI and FAD/FAD-Plus.
For the FWI and FAD/FAD-Plus, we also coded not only the free will and
determinism subscales but also all other subscales to investigate whether the influence of free
will belief manipulations is specific to belief in free will and belief in determinism or also
extends to other, related beliefs. More specifically, we coded the dualism subscale of the FWI
and the fatalistic determinism and unpredictability subscales of the FAD/FAD-Plus.
Type of Manipulation. To compare different types of manipulations, we coded
whether a text, statements, combination of text and statements, or a video was used to
manipulate belief in free will.
Participant Involvement. To test whether the effect of the manipulation depends on
how deeply participant had to process the stimulus material, we coded participant
involvement. For some studies, participants were asked to attend the provided information
carefully so they could summarize it at the end of the experiment. In other studies,
participants had to process the provided information directly after the manipulation by writing
an essay or rewriting the provided messages. Yet in other studies, participants merely
attended the provided information and did not have to reproduce any information. For
participant involvement, we coded whether participants (1) had to summarize or rewrite the
provided information directly after the manipulation, (2) had to summarize or rewrite the
information at the end of the experiment, or (3) simply had to attend the provided
information, such as by merely reading a text or watching a video.
Baseline Condition. To test whether beliefs in free will and determinism can also be
increased, in addition to decreased, we coded whether the baseline condition consisted of
neutral messages or pro-free will messages.
Measurement Moment. To investigate how long-lasting the free will belief
manipulations are, we coded whether belief in free will or determinism was assessed directly
after the manipulation or after the primary dependent variable was assessed.
Secondary Moderators. We also coded several secondary moderators, namely age,
sex, continent, test location, and sample type. Age was coded as the mean age of participants
in the sample. Sex was coded as the proportion of female participants. Continent refers to
whether the sample was collected in the US or on US-based platforms such as MTurk versus
in Europe or on Europe-based platforms such as Prolific (not enough experiments were
conducted on other continents to draw reliable conclusions). Test location refers to whether
the experiment was conducted in the laboratory or online. Finally, sample type refers to
whether the participants were students or panel participants (e.g., MTurk or Prolific).
Consequences?
In a first step, the first author coded all studies measuring downstream consequences
of manipulating belief in free will. The following variables were coded: the dependent
variable, whether the study included a manipulation check, whether the manipulation check
was significant, and whether the experiment was published. A manipulation check was
considered significant when at least one of the measured beliefs (i.e., free will or
determinism) reached a p-value of p < .05. In a second step, the second author checked the
first author’s coding for errors. Disagreements and inconsistencies that arose during coding
were resolved through discussion between both authors. The pooling of dependent variables
into the clusters described below was done through mutual discussion.
P-Curve. Rather than estimating a population effect size, p-curve investigates whether
there is at least one effect in a set of significant effects that is not zero, by testing if the p-
values corresponding to these effects follow a flat or skewed distribution (Simonsohn et al.,
2014). The p-curve analysis assessing the overall evidence for downstream consequences of
free will manipulations included 35 studies with a total of 49 experiments (see OSF;
https://bit.ly/2L69prl). 39 of these experiments were published in academic journals or books.
The remaining experiments were unpublished datasets. P-values used in the p-curve analysis
were coded according to the guidelines laid out in Simonsohn et al. (2014). If a study reported
multiple relevant tests, we coded only the first test. We also report robustness analyses
checking (1) whether the results changed when we used the second instead of the first test and
(2) whether significant effects still remained after removing the most significant test
(Simonsohn et al., 2015). Note that for this latter robustness analysis, we determined the p-
value using a simulated null distribution to account for the fact that removing the lowest p-
value affects the distribution of the test statistic.
Anti-Social Behavior Meta-Analysis. Upon reviewing the literature on downstream
consequences, one clear set of variables arose—namely anti-social versus pro-social behavior.
Hence, we pooled together the 15 studies (m = 21) in this set and subjected them to a meta-
analysis testing whether manipulating belief in free will influences social behavior. The pro-
social dependent variables were helping and positive attitudes towards minorities. The anti-
social dependent variables were immoral behavior, cheating, prejudice, malicious envy, and
victim blaming. Studies measuring pro-social behavior or pro-social attitudes were reverse
coded.
Conformity, Punishment, and Cheating Meta-Analyses. Three dependent variables
were measured in at least five experiments: cheating (n = 5; m = 8), conformity (n = 4, m = 6),
and punishment (n = 3; m = 9).
Meta-Analytic Procedures
Robust Variance Estimation
Analyses were performed in R (v3.5.1) using the metafor (Viechtbauer, 2010) and
robumeta (Fisher & Tipton, 2015) packages. The data were analyzed using random effects
models because we considered heterogeneity to be likely and because random effects models
converge on fixed effects models in the absence of heterogeneity (Field & Gillett, 2010). A
critical assumption of such random effects models is that the included effect sizes are
statistically independent (Lipsey & Wilson, 2001) and violating this assumption is known to
inflate the false-positive rate (Tanner-Smith et al., 2016). In the current meta-analysis,
however, many studies measured multiple dependent variables or included multiple baseline
conditions and therefore yielded more than one relevant effect size. As a result, the included
effect sizes were not independent. To control for this dependency, we decided to empirically
estimate standard errors that do not assume independent effect sizes using robust variance
estimation (RVE; Fisher & Tipton, 2015; Hedges et al., 2010; Tanner-Smith et al., 2016;
Tanner-Smith & Tipton, 2014).
In meta-analyses, effect sizes are typically weighted by their standard error. Within the
RVE framework, two ways to calculate such weights have been proposed: hierarchical effects
weights and correlated effects weights. The former is most appropriate when dependency
originates mainly from effect sizes being nested in larger units (e.g., research groups),
whereas the latter is more appropriate when dependency originates mostly from a single study
providing multiple effect sizes. While RVE provides asymptotically unbiased standard errors
regardless of how the weights are calculated, the choice of weights does influence statistical
efficiency (Hedges et al., 2010). As we expected dependency to arise mainly from studies
providing multiple effect sizes, we used correlated effects weights (Fisher & Tipton, 2015).
These weights are inverse variance weights, where the denominator of the weight assigned to
each effect size in an experiment is determined by the average variance in the experiment
multiplied by the number of effect sizes provided by that experiment. Importantly, this
ensures that the total weight of each experiment does not depend on the number of effect sizes
it provides (e.g., Cracco et al., 2018). Of note, to estimate variability, a parameter ρ
representing the correlation among the effect sizes has to be specified (Fisher & Tipton,
2015). This parameter is assumed to be the same for all experiments and typically has a
negligible influence on the results. In the current meta-analysis, we used the default value of
the robumeta package (i.e., ρ = 0.80), but also report sensitivity analyses where we vary this
value.
Importantly, one problem with RVE is that it has an inflated false positive rate when
the number of studies is moderate to small or when skewed or unbalanced moderators are
included (Tipton, 2015; Tipton & Pustejovsky, 2015). As a solution, small-sample corrections
(i.e., small number of studies within a meta-analysis) have been proposed for both t tests
(Tipton, 2015) and F tests (Tipton & Pustejovsky, 2015). As it is difficult to know when these
corrections should be implemented and when they should not, it has been recommended to
implement them for all RVE analyses, regardless of the meta-analytical sample size (Tipton,
2015). In the current meta-analysis, we follow this recommendation. Importantly, corrections
to the t test are only valid when df ≥ 4 (Tipton, 2015). As a result, we only report t tests with
more than 4 degrees of freedom. In addition, for continuous moderators (e.g., age), we
removed outlier values exceeding the weighted mean by more than 3 SDs because such
outliers strongly reduce the available degrees of freedom and hence statistical power (Tanner-
Smith et al., 2016). Weighted means were calculated by dividing a weight of 1 equally among
the different values provided by the same experiment.
Effect Size Estimation

All included studies used a between-subjects design to test the effect of the
manipulation. Hedge’s g was used as a measure of effect size. Effects were coded so that
positive effect sizes corresponded to stronger beliefs in free will/determinism (research
question 1) and larger values on the outcome measures (research question 2) in the anti-free
will condition than in the reference condition. Hedge’s g was calculated by first calculating
Cohen’s d and then correcting these values using the escalc function of the metafor package
in R (Viechtbauer, 2010). When means and standard deviations were reported, Cohen’s d was
calculated as:
%&'() * %,-.
"# = 6 6
0'
/ &'()
12345&'() 78',-. 12945,-. (1)
'&'() 7',-. 16
With anti referring to the anti-free will condition and ref referring to the reference (i.e.,
baseline) condition. When standard errors were provided instead of standard deviations, these
were transformed to standard deviations and Cohen’s d was calculated using (1). When
insufficient information was reported to use (1), we instead calculated Cohen’s d from the t or
F-value as:
< < < <

"# = :;= += or √@ ;= += (2)
&'() ,-. &'() ,-.
When we could not retrieve sufficient information to calculate an effect size, we contacted the
corresponding author for the necessary information. Nevertheless, despite our efforts, we
could not calculate the effect size for all coded studies. Specifically, for the first research
question, we could not calculate any of the effect sizes for 4 experiments from 3 studies and
only part of the effect sizes for 3 experiments from 3 studies. For the second research
question, we were not able to code 5 experiments from 5 studies. Effects for which we could
not calculate an effect size were not included in the meta-analysis.
Outliers
Outliers were defined as effect sizes exceeding the weighted mean effect size,
calculated as before, by more than 3 SDs. For the first research question, this procedure
identified 1 outlier for free will beliefs (g = -2.89) and 1 outlier for determinism beliefs (g =
4.00). These outliers were replaced by the effect size 3 SDs above the weighted mean effect
size for free will beliefs (g = -1.63) and determinism beliefs (g = 2.09). There were no outliers
for the second research question. In addition to replacing outliers, we also tested the influence
of each individual effect size on the average effect size using a leave-one-out cross-validation
procedure. This indicated that leaving out individual effect sizes did not strongly influence the
average effect size and mostly influenced it similarly in both directions for both the first (free
will beliefs: Δgmin = -0.01, Δgmax = 0.01; determinism beliefs: Δgmin = -0.02, Δgmax = 0.01)
and the second research question (anti-social behavior: Δgmin = -0.03, Δgmax = 0.04; cheating:
Δgmin = -0.11, Δgmax = 0.12; conformity: Δgmin = -0.04, Δgmax = 0.11; punishment: Δgmin = -
0.05, Δgmax = 0.05).
Moderator Correlations
To control for confounded moderators (Field & Gillett, 2010; Lipsey & Wilson, 2001),
we computed the weighted associations, calculated as before, between the different
moderators included in the analysis of the first research question with r for continuous–
continuous and continuous–dichotomous pairs, with multiple R for continuous–polytomous
pairs, and with Cramér’s V for dichotomous–dichotomous, dichotomous–polytomous, and
polytomous–polytomous pairs (Cracco et al., 2018). For all these measures, 0 means no
relationship and 1 means a perfect relationship. When two moderators correlated > 0.50, we
tested whether the moderator effects (if any) remained after controlling for the confounded
moderator. When the contingency table of two confounded moderators did not contain empty
cells, we controlled for their confounding influences by including both moderators in the
same meta-regression model. Using sum coding, this allowed us to test the average effect of
each moderator across the levels of the other moderator (for categorical moderators). When
the contingency table did contain empty cells, we controlled for confounding by restricting
the analysis of moderator A to the level of moderator B where the levels of moderator A were
most balanced, to maximize statistical power (Tipton, 2015; Tipton & Pustejovsky, 2015).
Table 2
Weighted Correlations Among Moderator Variables
Published Scale Type Involvement Moment Baseline Age Sex Continent Location Sample
Published 1.00 0.43 0.32 0.25 0.06 0.00 0.16 0.16 0.25 0.08 0.03
Scale 0.28* 1.00 0.40 0.31 0.52* 0.15 0.27 0.38 0.29 0.28 0.26
Type 0.20 0.36*** 1.00 0.59*** 0.55** 0.75*** 0.23 0.27 0.51*** 0.10 0.27
Involvement 0.19 0.37*** 0.42*** 1.00 0.39* 0.69*** 0.14 0.25 0.50** 0.21 0.36*
Moment 0.08 0.42*** 0.42*** 0.43*** 1.00 0.01 0.11 0.04 0.38** 0.06 0.05
Baseline 0.13 0.25 0.66*** 0.59*** 0.18 1.00 0.05 0.20 0.24 0.07 0.09
Age 0.04 0.30 0.34* 0.10 0.17 0.01 1.00 0.42** 0.25 0.84*** 0.75***
Sex 0.09 0.18 0.39** 0.23 0.10 0.08 0.55*** 1.00 0.36** 0.43** 0.66***
Continent 0.26** 0.21 0.29* 0.26* 0.02 0.11 0.24* 0.43*** 1.00 0.42** 0.69***
Location 0.02 0.19 0.18 0.18 0.08 0.07 0.82*** 0.44*** 0.42*** 1.00 0.86***
Sample 0.15 0.23 0.33* 0.45*** 0.05 0.13 0.81*** 0.63*** 0.50*** 0.72*** 1.00
Note. The lower half of the matrix shows correlations for free will beliefs and the upper half of the matrix shows correlations for determinism
beliefs. Relations between continuous–continuous variables and between continuous–dichotomous variables were assessed with Pearson’s r.
Relations between continuous–polytomous variables were assessed with multiple R. Relations between dichotomous–dichotomous, dichotomous–
polytomous, and polytomous–polytomous variables were assessed with Cramér’s V. Correlations > 0.50 are indicated in bold font. These
correlations were controlled for whenever one of the two involved moderators produced a significant effect. * p < .05, ** p < .01, *** p < .001.
Publication Bias
For both research questions, we tested for publication bias and other small study
effects using a hierarchical Egger’s regression test (Sterne & Egger, 2005). More specifically,
we ran an RVE meta-regression predicting effect sizes from their standard error (Rodgers &
Pustejovsky, 2020). Publication bias leads to a positive relationship between effect sizes and
standard errors because only large effect sizes are statistically significant in studies with a
large standard error (small N). To correct for potential publication bias, we used the PET-
PEESE approach (Stanley & Doucouliagos, 2014) because unlike other popular approaches,
such as three-parameter selection models (3PSM), it is easily implemented within the RVE
framework (Rodgers & Pustejovsky, 2020). Moreover, simulation studies have shown that
PET-PEESE performs reasonably well under most circumstances (Carter et al., 2019). In
PET-PEESE, a bias-corrected effect is calculated by taking the intercept of two RVE meta-
regressions that respectively regress the effect sizes onto their standard error (PET) and
variance (PEESE). PET and PEESE have complementary strengths and weaknesses: whereas
PET tends to be overly conservative when there is a true effect, PEESE tends to be overly
liberal when there is no true effect (Stanley & Doucouliagos, 2014). Therefore, we report both
measures, as has recently been recommended (Carter et al., 2019). Importantly, in some cases,
PET and PEESE overcorrect effect sizes, reversing their sign. Because these overcorrections
are not meaningful (Carter et al., 2019), we report the corrected effect size as 0 when this
occurs. Finally, in addition to using bias correction methods, we also compared published
with unpublished studies and tested if the effects remained significant when only unpublished
studies were considered.
Sensitivity Analyses
To explore the robustness of our main results, we conducted four sensitivity analyses
investigating how variations to our analysis procedure influenced (1) the effect size of the
manipulation and (2) the estimated publication bias. First, we repeated the analyses while
varying the ρ parameter of the RVE models from 0 to 1 in steps of 0.2. Second, we repeated
the analyses after computing effect sizes preferentially from the reported statistics instead of
from the means and standard deviations, as sometimes studies reported both. Although these
two types of information should in principle lead to the same effect size, this is not always the
case. Therefore, we decided to test how changing the type of data used to calculate the effect
sizes influenced the results. Third, we repeated the analyses without replacing outlier values.
Finally, we used a multilevel meta-analytical approach (Cheung, 2014) instead of RVE, to
test whether different methods of dealing with dependent effect sizes yielded similar effect
sizes. A sensitivity analysis was considered to produce results identical to the main analysis as
long as the difference between the corresponding estimates was ≤ 0.01.
Results
Research Question 1: Can the Belief in Free Will Be Manipulated?
Effect on Belief in Free Will and Belief in Determinism
We first tested the overall effectiveness of the manipulation for both free will and
determinism beliefs. This indicated that exposing participants to anti-free will manipulations
decreased belief in free will (g = -0.29, t(111) = -8.74, p < .001, 95% CI = [-0.35, -0.22], m =
119, k = 148), and increased belief in determinism (g = 0.17, t(46.5) = 4.33, p < .001, 95% CI
= [0.09, 0.24], m = 53, k = 61). Next, we compared the influence of the manipulation on these
two beliefs by reverse-coding the determinism belief effect sizes and comparing them with
the free will belief effect sizes. This revealed a borderline non-significant difference, with the
manipulation having a slightly stronger effect on free will beliefs than on determinism beliefs
(t(66) = -1.87, p = .066, m = 124, k = 209). Finally, we looked at the I² coefficients, which
indicated that a substantial portion of the variance for both belief in free will (I² = 79%), and
belief in determinism (I² = 67%), was due to heterogeneity in the effect sizes2. In sum, these
2
Note that RVE uses a simplistic method to estimate I² and that this should therefore be seen as a rough
indicator of heterogeneity rather than as a precise estimate (Tanner-Smith et al., 2016).
analyses indicate that anti-free will manipulations reduce the belief in free will and increase
the belief in determinism.
Publication Bias and Small-Study Effects
A visual inspection of the funnel plots (see Figure 2) revealed a largely symmetrical
effect size distribution for both free will and determinism beliefs. In line with this visual
inspection, a hierarchical Egger test found no relationship between the standard error and the
effect size for either free will (t(41.3) = 0.58, p = .565) or determinism beliefs (t(19.4) = 0.76,
p = .457). Importantly, however, this does not necessarily mean that there is no publication
bias in the literature. Indeed, our analysis included a large number of unpublished studies both
for the free will (m = 93, k = 116) and for the determinism analysis (m = 39, k = 44),
potentially masking the presence of publication bias. In support of this hypothesis, an analysis
including publication status as a moderator showed that the effect of the manipulation was
larger in published than in unpublished studies for both belief in free will (t(36.8) = -3.43, p =
.002; see Table 3), and belief in determinism (t(18.9) = 2.14, p = 0.046; see Table 4).
Importantly, however, the effect size of both free will beliefs (t(86.4) = -6.58, p < .001), and
determinism beliefs (t(34.6) = 2.73, p = .010), remained significant even when only
unpublished studies were included in the analysis. Similarly, correcting the free will belief
effect size with PET (gz = -0.33, p < .001) or PEESE (gz = -0.31, p < .001) revealed that the
effect size was still significant. Correcting the determinism belief effect size with PET made
the effect disappear (gz = 0.06, p = .617), but correcting it with PEESE did reveal a significant
effect (gz = 0.13, p = .040).
Taken together, we find evidence for publication bias, but also find that publication
bias is unlikely to explain the effect of the manipulation on free will beliefs. Although the
effect on determinism beliefs disappeared after applying PET correction, it was still
significant after applying PEESE correction, and an analysis including only unpublished
studies likewise revealed a significant determinism effect. Given that PET is known to be
overly conservative when there is a true effect (Stanley & Doucouliagos, 2014), the combined
evidence suggests that not just belief in free will but also belief in determinism is affected by
the manipulation, even after accounting for publication bias.
Figure 2
Funnel Plots for Belief in Free Will (Left) and Belief in Determinism (Right)
Note. Effect sizes from the same study are plotted in the same color with the same symbol.
Primary Moderators
Scales. We first investigated if the effectiveness of the manipulation depends on
whether free-will and determinism were measured using the FWI scale or using the FAD
scale. This revealed no significant difference between both scales for free-will beliefs (t(74.9)
= 1.53, p = 0.130; see Table 3), but a stronger effect when belief in determinism was
measured with the FAD than with the FWI (t(21.8) = 2.29, p = 0.031; see Table 4). However,
scale correlated highly with measurement moment for determinism beliefs (see Table 2). In
particular, studies that used the FWI tended to measure beliefs at the end of the experiment
rather than right after the manipulation, whereas studies using the FAD were more balanced
with respect to measurement moment. To control for this confound, we fitted a model
including both scale and measurement moment. This revealed that the difference between the
FWI and FAD for determinism beliefs was no longer significant (t(15.8) = 1.43, p = .172),
when measurement moment was controlled, even though the numerical pattern went in the
same direction as before.
Next, we investigated whether free will manipulations also influenced the other
subscales of the FWI and FAD. The FWI analysis indicated that the manipulation influenced
not only belief in free will (t(25.9) = -3.73, p < 0.001), and belief in determinism (t(26.9) =
2.27, p = 0.031), but also belief in dualism (t(25.8) = -6.56, p < 0.001). More specifically, it
indicated that belief in free will and belief in dualism decreased, whereas belief in
determinism increased following anti-free will manipulations. A direct comparison of the size
of these three effects revealed a significant main effect of subscale (F(2, 31.6) = 4.43, p =
.020), with significantly larger effect sizes for dualism than for determinism (t(27.7) = 2.87, p
= .008), but no significant difference between free will and determinism (t(34.4) = 1.86, p =
0.072), or between free will and dualism (t(33.9) = 0.57, p = 0.571; see Table 5).
The FAD analysis likewise showed that the manipulation changed not only belief in
free will (t(33) = -2.88, p = 0.007), and belief in scientific determinism (t(7.8) = 5.26, p <
0.001), but also belief in fatalistic determinism (t(5.3) = 3.10, p = 0.025). More specifically,
belief in free will decreased and belief in scientific and fatalistic determinism increased
following anti-free will manipulations. No effect was found on belief in unpredictability
(t(8.3) = 0.67, p = 0.522). Comparing the effectiveness of the manipulation across subscales
again revealed a significant main effect (F(3, 9.8) = 6.51, p = .011). Post-hoc tests showed
that the effect on scientific determinism was stronger than on all other subscales (all ps ≤
.007), and that the effect on unpredictability was significantly weaker than the effect on the
other subscales (all ps ≤ .045; see Table 5).
In sum, the analysis comparing the different scales indicated that anti-free will
manipulations have similar effects irrespective of whether belief in free will and belief in
determinism are measured with the FWI or with the FAD. Interestingly, the results also
showed that the effects of the manipulation are not specific to belief in free will and belief in
scientific determinism, but also extend to belief in dualism and belief in fatalistic
determinism, albeit not to belief in unpredictability.
Type of Manipulation. We first compared the effectiveness of the different
manipulations (i.e., text, statements, text and statements combined, or video). This revealed a
significant effect of manipulation type on free will beliefs (F(3, 26.3) = 6.70, p = .002; see
Table 3), but not on determinism beliefs (F(2, 15.7) = 1.83, p = .193; see Table 4).
A further analysis of the free will belief effect showed that manipulations combining a
text with statements were more effective than manipulations only presenting statements
(t(11.4) = -3.26, p = 0.007), only presenting a text (t(9.5) = -4.17, p = 0.002), or only showing
a video (t(14.6) = -2.14, p = 0.050). None of the other conditions differed significantly from
each other (all ps ≥ .057; see Table 3). Next, we also compared the effectiveness of the Crick
text, which has been used most often in the literature, with the effectiveness of other texts that
have been used. This revealed no difference for free will beliefs (t(13.2) = -0.71, p = 0.493;
see Table 3). The determinism analysis did not have enough degrees of freedom (i.e., df =
2.15) to interpret.
Importantly, however, manipulation type correlated strongly with baseline condition
for free will beliefs (see Table 3). In particular, a closer look at the coding revealed that
studies combining a text with statements and studies using videos always had a pro-free will
baseline. Therefore, to test if this could explain the effect of manipulation type, we did a
control analysis restricting the analysis to those studies using a pro-free will baseline. While
this changed the main effect of manipulation type from significant to marginally significant
(F(3, 21.9) = 2.86, p = .060), it did not substantially change the pattern of results. In
particular, manipulations combining a text with statements were still significantly more
effective than manipulations only using statements (p = 0.016), and manipulations only using
a text (p = 0.016), and were marginally more effective than manipulations using a video (p =
0.061).
Taken together, attempts to manipulate beliefs in free will are most effective when
combining texts arguing against free will with statements that help strengthen the message.
They are more effective than only using statements, texts, or videos.
Participant Involvement. For participant involvement, there was a significant main
effect on free will beliefs (F(2, 69.5) = 10.9, p < .001; Table 3), but not on determinism
beliefs (F(2, 29.3) = 0.61, p = .549; see Table 4). A further analysis of the free will effect
showed that the manipulation was more effective when participants had to report on the
content of the manipulation immediately after the manipulation compared with when they had
to report on the content at the end of the experiment (t(63.8) = -4.67, p < 0.001), or not at all,
(t(61.9) = -2.79, p = .007). The latter two conditions, however, did not differ, (t(58.5) = 1.19,
p = .240). Importantly, like manipulation type, participant involvement also correlated highly
with baseline condition for free will beliefs. Visual inspection showed that most studies
requiring participants to report on the content of the manipulation immediately after the
manipulation also used a pro-free will baseline, whereas other studies tended to use a neutral
baseline. To control for this confound, we fitted a model including both participant
involvement and baseline condition, which revealed that participant involvement remained
significant (F(2, 50) = 4.71, p = .013).
In conclusion, anti-free will manipulations have the strongest effects when
experimental procedures are rehearsed or verified directly after the manipulation, suggesting
that participant involvement increases its effectiveness.
Measurement Moment. For measurement moment, anti-free will manipulations had a
larger influence on free will beliefs (t(92.3) = -2.32, p = .022; see Table 3) when the belief
was assessed before compared with after the primary dependent variable was measured
(i.e.,downstream consequences). The same statistical test was not significant for determinism
beliefs (t(33.2) = 0.95, p = .349; see Table 4).
In sum, anti-free will manipulations have the strongest effects when free will beliefs
are measured directly after the experimental manipulation instead of at the end of the study.
This suggests the effects of the manipulation fade over the course of the experimental session.
Baseline Condition. Anti-free will manipulations had a larger effect on free will
beliefs when the baseline was a pro-free will condition than when it was a neutral condition,
(t(106.7) = -4.11, p < .001; Table 3). A similar effect was also apparent for determinism
beliefs, but did not reach statistical significance (t(25) = 1.05, p = .304; see Table 4). As
discussed above, however, baseline condition correlated highly with manipulation type and
manipulation task. Since there were no studies in some cells of the baseline condition x
manipulation type cross-table, we decided to control for manipulation type by looking at the
effect of baseline condition for the manipulation type where baseline condition was most
balanced (i.e., statements) to optimize power (Tanner-Smith et al., 2016). This revealed that
even when restricting the analysis to those studies using statements, anti-free will messages
were more effective when the baseline condition was a pro-free will condition than when it
was a neutral condition (p = .032). To control for participant involvement, we fitted a model
including both baseline condition and participant involvement, which did not change the
results (p = .024).
Taken together, comparing anti-free will messages with pro-free will messages results
in larger effect sizes than comparing anti-free will messages with neutral messages. This
suggests that belief in free will cannot only be reduced but can also be increased by
experimental manipulations.
Secondary Moderators
Age. The mean age of the sample did not influence the effectiveness of the
manipulation for either free will beliefs (t(28.2) = 0.27, p = .793; see Table 3), or determinism
beliefs (t(19.1) = 0.21, p = .838; see Table 4).
Sex. The proportion of female participants in the sample did not influence the effect of
the manipulation on either free will beliefs (t(31.5) = 0.53, p = .598; see Table 3), or
determinism beliefs (t(17.3) = -0.14, p = .893; see Table 4).
Continent. The manipulation had a stronger effect on samples collected in the US or
on US-based platforms (e.g., MTurk) than on samples collected in Europe or on Europe-based
platforms (e.g., Prolific) for free will beliefs (t(101) = -2.50, p = .014; see Table 3), but not for
determinism beliefs (t(45.2) = 0.99, p = .329; see Table 4).
Test Location. Whether the experiment was conducted in the lab or online did not
influence the effect of the manipulation on either free will beliefs (t(77.2) = 0.45, p = .655;
see Table 3), or determinism beliefs (t(42.6) = -0.55, p = .586; see Table 4).
Sample Type. The effectiveness of the manipulation did not differ between student
samples and samples collected via online platforms for either free will beliefs (t(62.6) = 0.81,
p = .422; see Table 3), or determinism beliefs (t(28.7) = 0.62, p = .543; see Table 4).
To test the robustness of our main results, we ran four sensitivity analyses
investigating how changing the analysis procedure influenced (1) the effect size of the
manipulation and (2) the evidence for publication bias. First, we varied the ρ parameter
between 0.1 and 1.0 in steps of 0.2. This resulted in identical effect sizes for both free will
beliefs and determinism beliefs. Second, we calculated effect sizes preferentially from the test
statistics instead of from the means and standard deviations. This again resulted in identical
estimates for both free will beliefs and determinism beliefs. Third, we repeated the analysis
without replacing outlier values. This led to slightly larger effect sizes for determinism beliefs
(gz = 0.19) but identical effect sizes for free will beliefs. Finally, we conducted a multilevel
meta-analysis instead of using RVE. This resulted in identical effect sizes for both free will
beliefs and determinism beliefs. Thus, none of the four changes to our analysis procedure had
a notable influence on the manipulation’s effect size. Similarly, the same four changes also
did not have a notable influence on the publication bias results, except that the PEESE-
corrected effect size for determinism beliefs was no longer significant when outliers were not
replaced (gz = 0.12, p = .201). For a full overview of the results of the different sensitivity
analyses, see OSF: https://bit.ly/2L69prl.
Summary of the Effectiveness of Free Will Belief Manipulations
In sum, our analyses indicate that anti-free will manipulations are able to successfully
influence belief in free will and related beliefs. The strongest effects occur when anti-free will
texts (e.g., the Crick text) are paired with anti-free will statements. Manipulation effects are
strongest when measured directly after the manipulation, and group differences are largest
when the anti-free will condition is compared to a pro-free will condition. However, we found
no moderation by the demographics investigated here.

Table 3
Free Will Moderator Statistics
Moderator g/β SE 95% CI m k F/t (df) p

Publication Status 3.43 (36.8) .002
Unpublished -0.23a 0.04 [-0.30, -0.16] 93 116
Published -0.51b 0.07 [-0.66, -0.36] 26 32
Questionnaire 1.53 (74.9) .130
FWI -0.23a 0.04 [-0.31, -0.14] 39 40
FAD -0.13a 0.04 [-0.22, -0.05] 46 59
Manipulation Type 6.70 (3, 26.3) .002
Text -0.18a 0.03 [-0.24, -0.12] 58 63
Statements -0.29a 0.06 [-0.41, -0.17] 35 54
Text and Statements -0.79b 0.14 [-1.12, -0.45] 8 9
Video -0.41a 0.11 [-0.64, -0.17] 12 12
Text Type 0.71 (13.2) .493
Crick -0.17a 0.03 [-0.23, -0.11] 48 51
Other -0.22a 0.06 [-0.35, -0.08] 10 12
Involvement 10.90 (2, 69.5) <.001
None -0.22a 0.06 [-0.34, -0.10] 33 43
Report Before -0.46b 0.06 [-0.59, -0.34] 44 53
Report After -0.14a 0.03 [-0.20, -0.09] 32 39
Measurement Moment 2.32 (92.3) .022
Before -0.38a 0.07 [-0.52, -0.25] 51 68
After -0.21b 0.03 [-0.27, -0.15] 49 58
Baseline 4.11 (106.7) <.001
Control -0.18a 0.03 [-0.23, -0.12] 72 80
Pro-Free Will -0.42b 0.06 [-0.53, -0.31] 64 67
Age 0.04 0.14 [-0.25, 0.32] 64 80 0.27 (28.2) .793
Sex (% Female) 0.09 0.16 [-0.24, 0.42] 66 82 0.53 (31.5) .598
Continent 2.50 (101.0) .014
Europe -0.19a 0.05 [-0.29, -0.09] 54 62
USA -0.36b 0.05 [-0.45, -0.27] 62 83
Test Location 0.45 (77.2) 0.655
Lab -0.29a 0.05 [-0.40, -0.18] 42 49
Online -0.26a 0.05 [-0.36, -0.15] 57 69
Sample 0.81 (62.6) 0.422
Students -0.30a 0.05 [-0.39, -0.21] 50 63
Panel -0.37a 0.08 [-0.53, -0.21] 29 32
Note. Different subscripts indicate p < .05. g/β = Hedge’s g/beta coefficient; SE = standard
error; 95% CI = 95% confidence interval; m = number of experiments; k = number of effect
sizes. All statistical tests are unsigned.
Table 4
Determinism Moderator Statistics

Publication Status 2.14 (18.9) .046
Unpublished 0.12a 0.05 [0.03, 0.21] 39 44
Published 0.30b 0.07 [0.15, 0.44] 14 17
Questionnaire 2.29 (21.8) .032
FWI 0.11a 0.04 [0.02, 0.19] 30 31
FAD 0.30b 0.07 [0.14, 0.47] 16 18
Manipulation Type 1.83 (2, 15.7) .193
Text 0.09a 0.04 [0.02, 0.17] 30 32
Statements 0.26a 0.08 [0.08, 0.43] 15 20
Video 0.26a 0.15 [-0.12, 0.64] 7 7
Involvement 0.61 (2, 29.3) .549
None 0.23a 0.07 [0.08, 0.37] 16 19
Report Before 0.18a 0.11 [-0.05, 0.41] 14 17
Report After 0.14a 0.04 [0.06, 0.22] 21 22
Measurement Moment 0.95 (33.2) .349
Before 0.22a 0.08 [0.05, 0.39] 18 22
After 0.13a 0.05 [0.03, 0.23] 28 29
Baseline 1.05 (25.0) .304
Control 0.13a 0.03 [0.06, 0.20] 40 43
Pro-Free Will 0.25a 0.10 [0.02, 0.47] 18 18
Age 0.03 0.13 [-0.25, 0.30] 42 49 0.21 (19.1) .838
Sex (% Female) -0.03 0.19 [-0.43, 0.38] 45 52 0.14 (17.3) .893
Continent 0.99 (45.2) .329
Europe 0.13a 0.06 [0.01, 0.25] 30 35
USA 0.21a 0.05 [0.10, 0.31] 23 26
Test Location 0.55 (42.6) .586
Lab 0.19a 0.06 [0.07, 0.32] 27 33
Online 0.15a 0.05 [0.04, 0.26] 22 23
Sample 0.62 (28.7) .543
Students 0.19a 0.06 [0.07, 0.31] 31 37
Panel 0.15a 0.05 [0.03, 0.26] 14 15
sizes. All statistical tests are unsigned.
Table 5
FWI and FAD Statistics

FWI 4.43 (2, 31.6) .020
Free Will 0.25a,b 0.07 [0.11, 0.38] 39 40
Determinism 0.11a 0.05 [0.01, 0.21] 30 31
Dualism 0.29b 0.05 [0.20, 0.39] 28 29
FAD 6.52 (3, 9.8) .011
Free Will 0.12a 0.04 [0.04, 0.21] 46 59
Scientific Determinism 0.40b 0.08 [0.22, 0.57] 16 18
Fatalistic Determinism 0.13a 0.04 [0.02, 0.23] 13 14
Unpredictability -0.03c 0.05 [-0.13, 0.07] 12 13
sizes. Effect sizes are coded such that larger effects in the expected direction lead to more
positive effect sizes. All statistical tests are unsigned.
Consequences?
P-Curve Analysis
A p-curve is considered to contain evidential value if the null-hypothesis that a zero
true effect size underlies all effect sizes can be rejected. This null-hypothesis can be rejected
if the half p-curve, considering only p-values < .025, is significantly right skewed at p < .05,
or if the full p-curve, considering all p-values < .05, and half p-curve are both right-skewed at
p < .10. Similarly, a p-curve is considered to signal the absence of evidential value if the half
p-curve is significantly flatter than the curve expected with 33% power at p < .05 or if both
the full and half p-curve are flatter than 33% power at p < .10 (Simonsohn et al., 2015).
A first p-curve analysis with 49 experiments testing the influence of anti-free will
manipulations on behavior, attitudes, and cognition revealed that both the full (zfull = -3.54,
pfull < .001), and half p-curve (zhalf = -1.38, phalf = .084), had a p-value < .10, and hence that
the set of studies contained evidential value. However, a visual inspection of the p-curve
revealed that it was almost entirely flat (see Figure 3). Although the flatness test did not find
evidence for the absence of evidential value (zhalf = 5.86, phalf > .999), a robustness check, as
recommended by Simonsohn et al. (2015), indicated that removing the single most significant
p-value across all included studies (i.e., Ching & Xu, 2018) was sufficient to render the test of
evidential value non-significant (zhalf = -0.70, phalf = .128). A second robustness check
indicated that evidential value was stronger (zfull = -4.57, pfull < .001, zhalf = -2.49, phalf = .006)
when the second rather than the first result was used from studies reporting multiple relevant
results, and this was still true even after removing the most significant test.
A potential explanation for why the main analysis did not find clear evidence for
downstream effects could be that some of the included studies were not able to confirm that
their manipulation changed participants’ belief in free will. That is, a study cannot be
expected to have downstream consequences if it did not successfully manipulate the belief in
free will in the first place. To test this hypothesis, we ran a second p-curve analysis including
only those studies with a statistically significant manipulation check (i.e., p < .05). Across 23
tests, this revealed no evidence for either the presence (zhalf = -1.09, phalf = .138) or absence
(zhalf = 3.66, phalf = .999) of evidential value (Figure 3). A robustness check using the second
instead of the first reported test in studies with multiple relevant tests indicated the presence
of evidential value (zfull = -1.87, pfull = .031, zhalf = -1.48, phalf = .069), but removing the single
most significant test was again sufficient to make this evidence disappear (zhalf = -0.71, phalf =
.103).
Taken together, the p-curve analysis finds only little evidence for the hypothesis that
manipulating belief in free will has downstream consequences on behavior, attitudes, and
cognition. That is, although some analyses provided weak evidence for downstream
consequences, this evidence disappeared in all but one case when the single lowest p-value
was excluded (Simonsohn et al., 2015). At the same time, there was also no conclusive
evidence for the absence of an effect. Hence, the p-curve found no clear evidence for, but also
not against, the hypothesis that free will belief manipulations have downstream consequences.
Figure 3
P-curve on the Entire Dataset (Left) and on the Studies with a Significant Manipulation
Check (Right)
Note. See the OSF folder (https://bit.ly/2L69prl) for the evidence tables.
Anti-Social Behavior
A random effects meta-analysis with robust variance estimation indicated an effect
size of g = 0.33 for the influence of anti-free will manipulations on anti-social behavior,
(t(21.1) = 3.10, p = .005, 95% CI = [0.11, 0.55], m = 23, k = 33)3. However, a visual
inspection of the forest plot (see Figure 4) suggested that this significant effect was mostly
driven by studies with smaller sample sizes, whereas studies with larger sample sizes tended
to report null results. This was confirmed by a hierarchical Egger test, indicating a significant
positive relationship between the effect size and the standard error (β = 3.01, t(12.8) = 3.52, p
= .004). In other words, while the meta-analysis suggested an effect of the manipulation on
anti-social behavior, additional analyses indicated that this effect may have been driven by
3
Note that after data collection had already been completed, Many Labs 5 was published, containing 5
replications of Vohs and Schooler’s (2008) Experiment 1 (Buttrick et al., 2020). While these 5 experiments are
not included in the meta-analysis, we did check whether adding them changed the results. For anti-social
behavior, the estimated effect size with these 5 experiments included was g = 0.29 (m = 28, k = 38, p = .004).
For cheating, it was g = 0.27 (m = 13, k = 14, p = .082). In both cases, a hierarchical Egger test found evidence
for small study bias (both p = .004) and both PET and PEESE indicated that the corrected effect size was g = 0.
In other words, the anti-social behavior and cheating meta-analyses yielded very similar results with and without
these 5 experiments.
small-study bias. Correcting the effect size for this bias using PET (g = 0.00) or PEESE (g =
0.00) resulted in substantially reduced and non-significant effect sizes. Likewise, even though
there was no significant difference between published (g = 0.50) and unpublished (g = 0.15)
studies (t(19.6) = 1.76, p = .095), the effect was descriptively larger for published studies and
was no longer significant if only unpublished studies were included in the analysis (t(8.6) =
1.16, p = .279). Taken together, this indicates that there is insufficient evidence that anti-free
will manipulations influence anti-social behavior.
However, including all studies in a meta-analysis of this sort may be problematic
because studies that did not change belief in free will (e.g., failed at the manipulation check)
could not hope to have downstream consequences on anti-social behavior. That said, a
moderator analysis comparing effect sizes in studies with a significant manipulation check (g
= 0.30, m = 10, k = 13) to effect sizes in studies with no manipulation check or a non-
significant manipulation check (g = 0.37, m = 13, k = 20) revealed no significant difference
(t(19.4) = -0.34, p = .736). If anything, the effect size was even numerically smaller in the set
of studies reporting a successful manipulation check, and this effect size was not significant
(t(8.7) = 2.11, p = .065, 95% CI = [-0.02, 0.61]).
In sum, the analysis showed that the effect of anti-free will manipulations on anti-
social behavior was no longer significant after controlling for publication and small sample
biases, which was true even when we only included studies that found a significant effect of
the manipulation on belief in free will. This indicates that there is insufficient evidence for the
idea that manipulating belief in free will influences anti-social behavior.

Figure 4
Forest Plot of the Effect of the Manipulation on Anti-Social Behavior.
Note. Pro-social effect sizes are reverse coded. Published studies are annotated with “+” and
unpublished studies with “-”. Nadelhoffer er al. (2019) and Genschow & Vehlow (2019) were
included in the meta-analysis as unpublished studies but were published later as Nadelhoffer
et al. (2020) and Genschow and Vehlow (2021) after data-analysis had already been
completed.
Cheating, Conformity, and Punishment
No effects of anti-free will manipulations were found on either cheating (g = 0.39,
t(6.8) = 1.81, p = .114, 95% CI = [-0.12, 0.91], m = 8, k = 9), conformity (g = 0.26, t(4.7) =
1.63, p = .168, 95% CI = [-0.16, 0.68], m = 6, k = 9), or punishment (g = -0.15, t(7.5) = -1.60,
p = .151, 95% CI = [-0.36, 0.07], m = 9, k = 9). In line with the anti-social behavior meta-
analysis, visual inspection of the forest plots (see Figures 5-7) indicated that studies with
smaller sample sizes tended to report significant effects, whereas studies with larger sample
sizes tended to report null effects. This was confirmed by a hierarchical Egger test, showing
significant relationships between the effect size and the standard error for all three dependent
variables (all p ≤ .014). Correcting the effect sizes for these small-study biases using PET (all
g = 0.00) or PEESE (all g = 0.00) eliminated all possible evidence. However, the Egger and
PET/PEESE results should be interpreted with care, as the limited dfs (all ≤ 4.1) likely led to
an inflated false positive rate (Tipton, 2015). Similarly, for cheating and conformity, there
were not enough studies to reliably test if the effect remained significant when including only
the unpublished studies (both df ≤ 3.9), but in both cases, effect sizes were numerically
smaller for unpublished (gcheating = 0.23, gconformity = 0.04) than for published studies (gcheating =
0.69, gconformity = 0.90). For punishment, effect sizes were also numerically smaller for
unpublished studies (g = -0.10) than for published studies (g = -0.23), but this difference did
not reach significance (t(6.1) = -0.61, p = .563).
Finally, we investigated if the effect was modulated by whether a significant
manipulation check was reported and if it was significant when including only those studies
reporting a significant manipulation check. However, for cheating and conformity, there were
insufficient studies to test either of these two hypotheses (all df ≤ 3.77). Although the cheating
effect size was descriptively larger for studies reporting a significant manipulation check (g =
0.94, m = 3, k = 4) than for studies not reporting a significant manipulation check (g = 0.02, m
= 5, k = 5), this difference should not be taken at face value, given the limited number of
studies and given that 3 of the 4 included effect sizes with a significant manipulation check
came from the same paper (i.e., Vohs & Schooler, 2008). The conformity effect size was
descriptively smaller for studies reporting a significant manipulation check (g = 0.10, m = 3, k
= 4) than for studies not reporting a significant manipulation check (g = 0.49, m = 3, k = 5).
For punishment, the difference between studies that did or did not report a significant
manipulation check was not significant (t(6.0) = 0.24, p = .821). There were not enough
studies to test if the effect remained significant when only including studies reporting a
significant manipulation check (df = 3.9). However, if anything, the effect was numerically
smaller in studies that reported a significant manipulation check (g = -0.13, m = 5, k = 5) than
in studies that did not report a significant manipulation check (g = -0.18, m = 4, k = 4).
In conclusion, after controlling for small-study biases, the effect sizes of free will
belief manipulations on cheating, conformity, and punishment became statistically
indistinguishable from zero.
Figure 5
Forest Plot of the Effect of the Manipulation on Cheating
Note. Published studies are annotated with “+” and unpublished studies with “-”. Nadelhoffer
er al. (2019) and Genschow & Vehlow (2019) were included in the meta-analysis as
unpublished studies but were published later as Nadelhoffer et al. (2020) and Genschow and
Vehlow (2021) after data-analysis had already been completed.
Figure 6
Forest Plot of the Effect of the Manipulation on Conformity
Note. Published studies are annotated with “+” and unpublished studies with “-”.
Figure 7
Forest Plot of the Effect of the Manipulation on Punishment
Note. Published studies are annotated with “+” and unpublished studies with “-”.
To test the robustness of our main results, we used the same approach as for the first
research question. That is, for all four dependent measures, we investigated how changing the
analysis procedure influenced (1) the effect size of the manipulation and (2) the evidence for
publication bias. In contrast to the first research question, however, we now ran only three
instead of four sensitivity analyses, because there were no outlier values in the data of the
second research question.
First, we varied the ρ parameter from 0.1 to 1.0 in steps of 0.2. This led to identical
effect sizes for all four dependent measures. Second, we calculated effect sizes preferentially
from the test statistics instead of from the means and standard deviations. This led to a
slightly larger effect size for conformity (gz = 0.28, p = .138), but did not change the effect
sizes of the other dependent variables. Finally, we conducted a multilevel meta-analysis
instead of using RVE. This resulted in slightly higher effect sizes for anti-social behavior (gz
= 0.35, p = .004), cheating (gz = 0.43, p = .093), and conformity (gz = 0.33, p = .128), but did
not change the effect size for punishment. Thus, in line with the first research question,
varying the analysis strategy did not have a notable influence on the results. The same was
also true for the publication bias results, both in the sense that all sensitivity analyses
indicated evidence for such a bias and in the sense that correcting the bias made the effects
disappear regardless of the analysis approach. For a full overview of the sensitivity analysis
results, see OSF: https://bit.ly/2L69prl.
Discussion
Past research has shown that reducing individuals’ belief in free will affects societally
relevant behaviors such as cheating (Vohs & Schooler, 2008) and other anti-social behaviors
(Baumeister et al., 2009; Zhao et al., 2014). These and similar results have been used by some
scholars as an argument that anti-free will and deterministic viewpoints should be kept away
from society, because they may change the way people interact with each other. However, a
number of studies reported difficulties in replicating some of the key results in the field
(Genschow et al., 2020; Giner-Sorolla et al., 2016; Monroe et al., 2017; Open Science
Collaboration, 2015; Schooler et al., 2014; Shariff & Vohs, 2014). While these failed
replications call into question the societal relevance of belief in free will, it is not yet clear
what caused them. In this article, we tested three possible explanations that may account for
the failed replications. First, it could be that the failed replications are false negatives. That is,
they were not able to detect an effect that is actually real. Second, it could be that the
manipulations commonly used in the literature do not alter individuals’ belief in free will and
thus have no impact on other behaviors. Third, it could be that free will belief manipulations
reduce belief in free will, but that this does not have any downstream consequences.
Our meta-analysis favors the third explanation. That is, we found that beliefs related to
free will can effectively be manipulated by commonly used experimental manipulations,
although the effects were rather small, with g = -0.29 for free will beliefs and g = 0.17 for
determinism beliefs. However, we did not find evidence for downstream consequences on
attitudes, behavior, or cognition. In the remainder of this article, we discuss the implications
of our results for the processes underlying free will belief manipulations, the potential reasons
for why these manipulations did not have downstream consequences, the societal implications
of the meta-analysis, possible steps for future research, and the limitations of the present
meta-analysis.
Processes Underlying Free Will Belief Manipulations
The analysis of our first research question on the effectiveness of free will belief
manipulations indicated that beliefs related to free will can be influenced by experimental
manipulations. Although we found evidence for publication bias, we also found that this
cannot explain the effect. To investigate the conditions under which these manipulations were
effective, we conducted several moderator analyses. The results of these analyses further the
understanding of free will belief manipulations manifold.
First, previous research indicated that most people believe that they have free will
(Baumeister et al., 2009; Nahmias et al., 2005). Based on this finding, it has been assumed
that free will beliefs can only be decreased (for an overview, see Ewusi-Boisvert & Racine,
2018). While this assumption has never been systematically investigated, the present meta-
analysis allowed us to test whether experimental manipulations can also increase the belief in
free will. The results demonstrate that belief in free will can be reduced as well as increased
by experimental means.
Second, the changes in the beliefs were relatively weak. An open question is whether
the conceptualization of free will underlying these manipulations influences the magnitude of
the effects. Indeed, definitions regarding free will are rather diverse and complex (Carey &
Paulhus, 2013) and may thus be impalpable for laypeople. If true, free will belief
manipulations relying on these definitions are unlikely to have a strong impact on laypeople’s
beliefs. To mitigate this problem, future research could investigate whether manipulations that
focus more specifically on laypeople’s concepts of free will produce stronger effects.
Third, from a methodological point of view, it is important to know which
manipulations are most effective to reliably alter belief in free will. By comparing all the
different manipulations used in previous research, our analysis suggests that an approach in
which participants have to both read a text and reproduce statements produces the largest
effects.
Fourth, an interesting and open question was how deeply participants have to process
the stimulus information for the manipulation to succeed. To answer this question, we
investigated whether the effectiveness of the manipulation depended on the degree to which
participants had to engage with the task (e.g., by summarizing or rewriting presented
messages). Manipulations worked best when participants had to summarize the stimulus
information directly after the manipulation. In other words, actively processing the provided
information (e.g., by summarizing it) increased the effects of the manipulation. This indicates
that participants’ involvement in the task strengthens the effect of free will belief
manipulations—a finding that is in line with previous research on the self-generation effect
(Slamecka & Graf, 1978) where self-generated information increases memory performance of
that information.
Fifth, past research left unanswered the question of how long the effects of free will
belief manipulations last. Measuring beliefs at the end of the experiment assumes the
manipulation lasts through the duration of the experiment; this is not necessarily the case,
however. For instance, it could also be that free will belief manipulations do not fully change
people’s beliefs but merely activate an anti-free will mindset for a brief moment. To test how
long-lasting the effects of free will belief manipulations are within the span of an experiment,
we investigated if the effect of the manipulation differed depending on whether beliefs were
measured immediately after the manipulation or at the end of the experiment (i.e., after
completing another task). At least for belief in free will, the effect reduced over time. While
the same was not true for belief in determinism, it is important to note that deterministic
beliefs were generally less influenced by moderators. This could mean that determinism
beliefs are more robust than beliefs in free will, but there are also several alternative
explanations. For example, the meta-analysis included fewer studies measuring belief in
determinism than belief in free will and the determinism moderator analyses may therefore
have had less statistical power to detect such effects (Hempel et al., 2013). Alternatively, it
could be that because belief in determinism was less influenced by the manipulation as such,
there was a floor effect, making it difficult to further reduce the effect of the manipulation.
Sixth, the meta-analysis yielded a borderline non-significant trend indicating that
belief in free will manipulations influenced beliefs in free will slightly more strongly than
beliefs in determinism. Given that this effect was not very strong and not significant based on
conventional levels of significance, it should be interpreted with care. Nevertheless, if
confirmed by future research, a potential explanation could be that common free will belief
manipulations influence beliefs in free will more strongly than beliefs in determinism. As
beliefs in free will are typically not correlated with belief in determinism (Nadelhoffer et al.,
2014), it is not surprising that manipulations have different effects on both beliefs. The
stronger effect of anti-free will manipulations on free will beliefs could thus indicate that they
specifically influence the belief in free will and only influence other beliefs to a lesser degree.
However, in contrast to this view, the results of our meta-analysis suggest that the influence
of anti-free will manipulations tends to be rather unspecific. For example, we found that the
effect of anti-free will manipulations is at least equally strong, if not stronger, on belief in
dualism as on belief in free will. Similarly, we found that belief in free will manipulations
influence not only belief in free will and belief in scientific determinism but also belief in
fatalistic determinism. There are different possible explanations for these unspecific effects.
Some of the beliefs are correlated with each other and are thus quite unspecific themselves.
For example, belief in free will positively correlates with belief in dualism (Nadelhoffer et al.,
2014; Wisniewski et al., 2019). In addition, it is likely that the manipulations affect not only
beliefs related to free will, but also other psychological and cognitive factors. For, instance, it
is conceivable that challenging the fundamental belief that free will exists leads to a general
feeling of confusion and uncertainty about the world.
Insufficient Evidence for Downstream Consequences and its Potential Reasons
While we found clear evidence that beliefs related to free will can be experimentally
manipulated, there was insufficient evidence these free will manipulations have downstream
consequences. That is, p-curve analyses across all dependent variables and meta-analyses on
internally coherent sets of dependent variables found either no evidence for downstream
consequences or weak evidence that, in all but one instance, disappeared when correcting for
extreme values or small-sample effects. This remained true even if only studies with
significant manipulation checks (i.e., significant effects on belief in free will and/or
determinism) were used in the analyses. This indicates that there is currently insufficient
evidence for downstream consequences.
How can this be reconciled with the fact that some primary studies did find
downstream consequences? The results of our meta-analysis suggest that one likely candidate
is publication bias. Indeed, the results of the Egger test indicated that studies with larger effect
sizes tended to be studies with higher standard errors. This is a typical sign of publication bias
because studies with small samples or noisy measurements only reach statistical significance
if the effect size is large (Sterne & Egger, 2005). Hence, a potential reason for why we could
not confirm previous studies finding downstream consequences is that these studies are false
positives.
Nevertheless, it is important to note that absence of evidence does not equal evidence
of absence. Indeed, the p-curve analysis lumped together a variety of different variables and
the meta-analysis focused only on a limited subsample of outcomes for which there were
enough data points. Thus, we cannot rule out that there are other specific single variables that
are influenced by free will belief manipulations. For example, we did not have enough data to
investigate the influences of free will belief manipulations on neurocognitive processes
(Rigoni et al., 2012; Rigoni et al., 2011; Rigoni et al., 2015), feelings of alienation (Seto &
Hicks, 2016), attributions of other people’s actions (Genschow et al., 2017a), perceived
meaningfulness of life (Crescioni et al., 2016; Moynihan et al., 2019), perceived gratitude
(MacKenzie et al., 2014), counterfactual thinking (Alquist et al., 2015), or risk taking
behavior (Schrag et al., 2016). For these and potential other variables that have not been
studied yet, it remains open to what degree they are influenced by free will belief
manipulations, although one of the effects on neurocognitive processes recently failed to
replicate (Eben et al., 2020).
It is also possible that the failure to find robust evidence for previously reported
downstream effects of free will belief manipulations has to do with subtle differences in the
manner in which the experimental protocols were implemented by the research teams that
were versus were not successful in finding such effects. Differences in experimental outcomes
between labs might reflect variations in the implementation of effective experimental
procedures (e.g., ensuring that participants were sufficiently motivated, believed the
experimental ruse, and followed the protocol) could be due to the inclusion of artefacts that
produced false positive effects (e.g. introducing experimenter demands characteristics), or
driven by the belief of the research teams to find an effect (Doyen et al., 2012). Future
studies, perhaps by research teams that have previously been effective in finding downstream
effects, might profitably examine whether procedural differences can be identified that
reliably differentiate between when such effects are versus are not observed.
Even if currently employed manipulations do turn out, as the present findings suggest,
to not produce robust downstream effects, this does not necessarily mean that such effects
cannot happen. It might be that existing manipulations are not suited for this purpose. In other
words, it might well be that experimentally reducing beliefs in free will has downstream
consequences, but that established free will belief manipulations are not able to produce them.
There are several reasons why this might be the case. As mentioned above, the manipulations
are rather unspecific and do not only affect belief in free will and determinism, but also other
beliefs. This is problematic for at least two reasons. On the one hand, an unspecific
manipulation may be detrimental in finding downstream consequences, because the different
factors influenced by the manipulation may counteract the influence of free will beliefs on the
dependent variable. On the other hand, an unspecific manipulation opens the question of to
which degree downstream consequences (if any) are actually driven by free will beliefs or
rather by other beliefs and psychological variables.
In addition, existing manipulations of free will beliefs have only weak effects on free
will (g = -0.29) and determinism (g = 0.17) beliefs. Given that any effect of anti-free will
manipulations on attitudes and behavior is likely smaller than their effect on the beliefs they
purport to change, this makes finding evidence for downstream consequences particularly
challenging. Indeed, research on the relationship between behavioral intentions and actual
behavioral change shows that the influence of behavioral interventions on behavior is roughly
twice as small as their influence on behavioral intentions (Webb & Sheeran, 2006). The same
problem is likely true for belief in free will and its downstream consequences. In this sense, it
may be helpful to relate our findings to the idea of “attitude strength” (Petty & Krosnick,
2014). Attitude strength is defined “as the extent to which attitudes manifest the qualities of
durability and impactfulness” (Petty & Krosnick, 2014; p. 3). Durability refers to the degree
to which an attitude can be changed over time and how strongly it resists attacks (e.g.,
persuasion attempts). Impactfulness relates to how strongly an attitude influences information
processing, judgments, and behaviors. Applying the idea of attitude strength to beliefs in free
will may help to explain why we did not find evidence for downstream consequences. That is,
free will beliefs are most likely to influence attitudes, behavior, and cognition when they are
strong. However, people with strong beliefs are least likely to be influenced by the
manipulation. Moreover, even if people were influenced by the manipulation, our meta-
analysis indicates that these new beliefs are likely to be weak. Given that weak beliefs are less
likely to influence behavior, this might explain why we did not find evidence for downstream
consequences in the current meta-analysis.
Related to this idea, another potential reason for the difficulty to find downstream
effects could be that that the effect of the manipulation on free will beliefs fades over time, as
our results indicate. As a result, it is possible that in some experiments, the effect of the
manipulation had already disappeared when the dependent variable was measured. From this
perspective, downstream consequences may be visible only when they are measured directly
after the manipulation. Future research should test this hypothesis more directly.
Finally, it is possible that some participants are not committed to a particular belief
about free will, but nevertheless have strong moral convictions about behaviors like cheating
(see e.g., Dubljević, 2013), for instance. In this case, manipulating belief in free will may
influence beliefs in free will without having downstream consequences, because moral beliefs
are held separate from their other beliefs. Future research could try to test this hypothesis in
more detail.
Taken together, many possible explanations exist for why we could not find evidence
for downstream consequences in this meta-analysis. An important task for future research will
be to investigate these explanations and to determine whether and under which circumstances
free will belief manipulations influence attitudes, behavior, and cognition.

Societal Implications
Whether free will exists is part of a longstanding philosophical debate (e.g., Dennett,
2015; Van Inwagen, 1983). However, after cognitive neuroscientists and psychologists started
claiming that humans’ perception of free will is nothing more than an illusion (e.g., Crick,
1994; Harris, 2012; Wegner, 2002), anti-free will viewpoints have become in vogue not only
in academia (e.g., Greene & Cohen, 2004), but also in popular media (e.g., Chivers 2010;
Griffin, 2016; Racine et al., 2017; Wolfe, 1997). When psychological research found that
presenting individuals with such anti-free will viewpoints influences fundamental behavior,
cognition, and attitudes, the question arose of whether the public press should publish anti-
free will viewpoints. While some philosophers argue that undermining people’s belief in free
will would have catastrophic consequences, as free will forms the basis for moral behavior
(e.g., Smilansky, 2000, 2002), other philosophers argue that disbelieving in free will might
also have positive effects, because it could lead to abandoning retribution-based morality and
illusory beliefs in a just world (Caruso, 2014; Greene & Cohen, 2004; Nadelhoffer, 2011).
The present research adds to this debate by suggesting that confronting individuals with anti-
free will viewpoints might not have as strong consequences as this has been previously
assumed.
Although these manipulations affect people’s beliefs in free will, we did not find
evidence for effects on behavior, cognition, or attitudes. This is in line with recent findings
indicating that although professional judges’ beliefs in free will are influenced by reading
anti-free will texts, reading these viewpoints does not influence their judgments (Genschow et
al., 2020). However, it is important to keep in mind that all these findings only speak to the
effect of presenting individuals with a single anti-free will viewpoint. While such a short
exposure may not have downstream consequences, it remains an open question as to whether
more concentrated and repeated presentations of anti-free will messages, as may happen in
real-life, could nevertheless have important consequences. Indeed, preliminary support for
this possibility comes from the results of the present meta-analysis, demonstrating that a
combined manipulation of presenting participants with a text and statements has the strongest
impact on individuals’ belief in free will. It would be interesting to test to what extent
repeated exposure to anti-free will messages may have stronger effects on individuals’ belief
in free will and thereby also lead to downstream consequences.
In the same vein, although we did not find sufficient support for the idea that
conventional belief in free will manipulations have downstream consequences, it is important
to note that we do not necessarily argue that belief in free will itself has no impact. That is, on
an interindividual level, belief in free will may well contribute to societally relevant behaviors
such as retributive punishment and anti- or pro-social behavior—to name just a few examples.
Indeed, several studies could robustly replicate the link between belief in free will and
retributive punishment on a correlational level (Genschow et al., 2017a; Martin et al., 2017).
Similar strong correlations were also found between belief in free will and job satisfaction
(Feldman et al., 2018), intentional attributions (Genschow & Lange, in press; Genschow et
al., 2017a; 2019a) as well as between free will beliefs and just world beliefs, religious
worldviews, and a conservative world view (Carey & Paulhus, 2013; Genschow & Vehlow,
2021). This suggests that on a correlational level, belief in free will may well be connected to
societally relevant behaviors.
Taken together, there is a debate about whether anti-free will viewpoints should be
discussed in the public media. Our findings suggest that the influence on society may be
weaker than previously assumed. In this respect, we would like to argue that discussions
about the implications of believing in free will should distinguish between scientific facts and
philosophical speculations (Schooler, 2010) as well as acknowledge methodological
limitations of the cited research (Racine et al., 2017).

Limitations of the Present Meta-Analysis
There are a few limitations to this meta-analysis that call for a careful discussion.
First, all publication bias correction methods have downsides (Carter et al., 2019; Stanley,
2017) and should hence be interpreted with care. We used PET and PEESE because they are
easily incorporated within the RVE framework (Rodgers & Pustejovsky, 2020) and have been
shown to retain reasonable false positive rates across a wide range of scenarios (Carter et al.,
2019). However, PET-PEESE can suffer from low power, especially when sample sizes are
small, heterogeneity high, or when there is either very little or very heavy use of questionable
research practices (Carter et al., 2019; Rodgers & Pustejovsky, 2020). As a result, absence of
evidence for publication bias cannot be taken as evidence of absence. Therefore, in addition to
using bias correction methods, we also compared published with unpublished studies and
tested if the effects remained significant if only unpublished studies were considered. While
no single method is perfect by itself, we believe that by combining these different methods,
we were able to get a clearer overview of the underlying true effects and the degree to which
they were inflated by publication bias.
Second, the current meta-analysis included a large number of unpublished studies.
This approach is consistent with recent calls to include unpublished research in meta-analyses
as a means to counter publication bias (Polanin et al., 2016). However, a potential downside is
that the unpublished studies may have been of inferior quality to the published research. One
way to rule this out is to code study quality. Existing tools focus strongly on clinical
intervention studies, however, and are therefore not easily applied to the research synthesized
here (Armijo-Olivo et al., 2012; Higgins et al., 2019). Instead, the main indicators of study
quality in research on free will belief manipulations (and other experimental social
psychological research) are blinded and random assignment, sample size, and the use of
validated manipulations and scales. With respect to blinded and random assignment, we had
initially planned to include a moderator coding whether the experimenter was blind to the
manipulation. However, after completing the coding, it was clear that this variable was almost
completely confounded with study location, such that online studies were blinded, whereas
papers on studies conducted in the lab did not provide information about this variable. Indeed,
experimenter blindness could only be coded for 7 studies conducted in the lab, of which 3
were our own unpublished studies. A similar problem exists for random assignment. While
random assignment is standard in the field, whether or not group assignment was indeed
random is often not explicitly reported in the paper, making it difficult to code.
The three other indicators of study quality (i.e., sample size, manipulation, and scale)
were explicitly addressed in the meta-analysis. With respect to sample size, the mean sample
per group was slightly larger in unpublished (N = 96) than in published studies (N = 85). With
respect to the manipulation, the best validated method to manipulate belief in free will is the
Crick text. Of the 55% published and 45% unpublished studies using a text to manipulate
beliefs in free will, 59% of the published and 91% of the unpublished studies used the Crick
text. Finally, with respect to the scale, 53% of the published and 74% of the unpublished
studies used a validated scale. Hence, a deeper analysis of the study quality indicates that, if
anything, unpublished studies were of higher quality than published studies. More generally,
methodological differences between published and unpublished studies in the current meta-
analysis were small. This was also confirmed by our analysis of moderator correlations (see
Table 2), which found no evidence for substantial correlations between publication status and
any of the coded variables.
Third, although no correlations were observed with publication status, large
correlations were observed for other moderators. Such confounding of moderators is
inevitable in meta-analytic research (Lipsey & Wilson, 2001) and was addressed by
performing control analyses analyzing confounded moderators together. Whereas most effects
remained even after controlling for these confounds, truly controlling for moderator overlap is
difficult and these findings should hence be tested more directly in future empirical work.
Finally, meta-analytical moderator analyses are known to often be underpowered
(Hempel et al., 2013). It is therefore possible that some of the non-significant moderator
effects found here could be explained by a lack of power. However, it is worth noting that at
least for the meta-analysis on beliefs in free will and determinism, most analyses included a
relatively large number of studies (see Tables 3 and 4). While we are not aware of research
that has systematically assessed the statistical power of RVE meta-analyses under different
conditions, previous simulation work using regular meta-analytical approaches suggests that
given the parameters of the current meta-analysis, many of the moderator analyses were, in
fact, well powered to detect even fairly small effect sizes (Hempel et al., 2013).
How to Move Forward? Possible Steps and Recommendations for Future Research
Our results offer several promising routes for future research. First, future research
should continue investigating the exact underlying mechanisms of free will belief
manipulations to further increase the understanding of these manipulations. For example,
recent research suggests that it is important to consider not just beliefs but also attitudes
towards free will (Cracco et al., 2020). From this perspective, a potential avenue for future
research could be to investigate whether attitudes towards free will alter the effect of anti-free
will manipulations.
Second, to better test the downstream consequences of free will belief manipulations,
researchers should aim to develop manipulations that (1) more specifically manipulate belief
in free will and determinism, but not other factors and (2) lead to larger effect sizes, for
example, by using dual-approach manipulations where participants read and repeat the
presented messages. Implementing these changes may allow testing whether belief in free will
manipulations have a meaningful societal relevance.
Third, future research could also investigate the somewhat surprising results obtained
in this meta-analysis. For example, an interesting finding is that anti-free will manipulations
appeared to have stronger effects in the US than in Europe. A potential explanation for this
effect might be that individual responsibility and personal agency is more central to US
culture than to European culture. Indeed, even though previous investigations suggest that
irrespective of culture, most people belief in free will (Sarkissian et al., 2010; Wisniewski et
al., 2019), there is an extensive literature showing cross-cultural differences on how
individuals construe agency and choice (e.g., Furnham et al., 1994; Markus & Kitayama,
2003). In line with a recent call for cross-cultural replications in social psychology
(Genschow et al., 2021), an interesting avenue for future research could therefore be to
explore the influence of the manipulation in different continents and cultures. Such research
could test not only if there is a different influence of free will belief manipulations on the
beliefs themselves but also whether cultural differences moderate the degree to which these
manipulations have downstream consequences.
Fourth, besides cultural differences and the moderators investigated in our meta-
analysis, there might be other factors that moderate the influence of free will belief
manipulations on beliefs, attitudes, behavior, and cognition. Indeed, there is a discussion
regarding the degree to which hidden moderators and high context sensitivity influence
psychological effects and account for the success of replications (e.g., Stroebe & Strack,
2014; Van Bavel et al., 2016). Future research should investigate hidden moderators and test
how context dependent free will belief manipulations are.
Fifth, an interesting question that we did not investigate in this meta-analysis is the
extent to which the dependent measures investigated here can be influenced by other
variables. For example, it could be that psychological variables related to free will such as
locus of control (Rotter, 1966) or self-efficacy (Bandura, 1977), for example, have stronger
downstream consequences. Directly comparing the effects of these different variables would
be interesting, because there are subtle differences between them that could tell us exactly
what determines the studied behavior. For example, whereas belief in free will can be defined
as an attitude, locus of control is a personality dimension (Waldman et al., 1983). Hence, if

locus of control is associated more strongly with cheating than belief in free will, this could
indicate that internal attribution traits are more important for cheating than internal attribution
attitudes. While such an analysis goes beyond the scope of the present meta-analysis, future
research could investigate for selected downstream consequences whether free will belief
manipulations relate to different behaviors, attitudes, and thoughts in comparison to other
psychological variables.
Finally, even though we did not find sufficient support for the hypothesis that free will
belief manipulations have downstream consequences, it might still be that on a correlational
level, free will beliefs relate to individuals’ behavior, cognition, and attitudes. Future research
could, thus, (1) test to which degree previous findings reported in the literature can be
replicated on a correlational level and (2) investigate whether other relevant societal factors
are modulated by individuals’ belief in free will.
Summary
The present meta-analysis finds support for the idea that beliefs related to free will can
be experimentally manipulated. These effects are stronger when participants are presented
with a combination of texts and statements, when they have to report on the content of the
manipulation, and when belief in free will is measured directly after the manipulation.
Moreover, beliefs related to free will can be experimentally increased as well as decreased.
However, the used manipulations produce rather weak effects and are rather unspecific in the
sense that beliefs and concepts other than belief in free will are also affected. Although we
find support for the idea that individuals’ belief in free will can be experimentally
manipulated, the meta-analysis did not find evidence for the idea that these manipulations
have meaningful downstream consequences. These findings call into question prior claims of
a causal relationship between belief in free will and attitudes, behavior, and cognition (a
number of which were made by authors of this article). Further research is warranted to
determine whether free will beliefs are related to attitudes, behavior, and cognition on an
interindividual (correlational) level, or by applying stronger and more specific manipulations.

References
References marked with an asterisk indicate studies included in the meta-analysis.
*Akin, M., Braun, I., Daenhardt, K., Fischer, S., Kollmann, O., Müller, F., Pistillo, E.,
Schmidt, K., Stritzel, C., Ueffing, D., Weber, A., Werkle, A., & Wilke, J. (2016).
Macht der Glaube an Willensfreiheit dankbarer? [Unpublished thesis].
*Alquist, J. L., Ainsworth, S. E., & Baumeister, R. F. (2013). Determined to conform:
Disbelief in free will increases conformity. Journal of Experimental Social
Psychology, 49, 80-86.
*Alquist, J. L., Ainsworth, S. E., Baumeister, R. F., Daly, M., & Stillman, T. F. (2015). The
making of might-have-beens: Effects of free will belief on counterfactual thinking.
Personality and Social Psychology Bulletin, 41, 268-283.
Armijo-Olivo, S., Stiles, C. R., Hagen, N. A., Biondo, P.D., & Cummings, G. G.. Assessment
of study quality for systematic reviews: a comparison of the Cochrane Collaboration
Risk of Bias Tool and the Effective Public Health Practice Project Quality Assessment
Tool: Methodological research. Journal of evaluation in clinical practice, 18, 12-18.
*Bastart, J., Redersdorff, S., & Martinot, D. (2015). Le libre arbitre au service du jugement
émis envers des victimes de sexisme. Psychologie Française, 60, 223-236.
* Ball, E.-C Ball, Degen, D., Fohrer, A., Grunwald, M., Hintze, S., Höhn, V., Kästel, M.,
Koziol, P., Matz, M., Papadakis, A., Porth, T., Renner, M., Seyder, M., Willems, S., &
Ziske, A. (2016). Der Glaube an den freien Willen und die Neigung zur Konformität.
[Unpublished thesis].
*Balodis, K. (2016). Influence of belief in free will on intertemporal decisions. [Unpublished
Bachelor thesis].
Bandura, A. (1977). Self-efficacy: toward a unifying theory of behavioral change.
Psychological Review, 84, 191.

*Baumeister, R. F., Masicampo, E., & DeWall, C. N. (2009). Prosocial benefits of feeling
free: Disbelief in free will increases aggression and reduces helpfulness. Personality
and Social Psychology Bulletin, 35, 260-268.
*Becker, L., Kandt, P., Leupelt, A., & Matthies, I. (2018). Das Ebenbild Gotttes oder ein
Sklave seiner Triebe? [Unpublished thesis].
*Blatz, L., Genschow, O., & Crusius, J. (2017). Untitled. [Unpublished manuscript].
*Bomke, N. Goldapp, M., & Schlack, A. (2017). Der Glaube an den freien Willen, Dualismus
und Ambivalenz. [Unpublished thesis].
*Braem, S., & Vermeylen, L. (2016). Untitled. [Unpublished manuscript].
Brass, M., Furstenberg, A., & Mele, A. R. (2019). Why neuroscience does not disprove free
will. Neuroscience & Biobehavioral Reviews, 102, 251-263.
Buttrick, N. R., Aczel, B., Aeschbach, L. F., Bakos, B. E., Brühlmann, F., Claypool, H. M., . .
. Szecsi, P. (2020). Many Labs 5: Registered Replication of Vohs and Schooler
(2008), Experiment 1. Advances in Methods and Practices in Psychological Science,
2515245920917931.
Carey, J. M., & Paulhus, D. L. (2013). Worldview implications of believing in free will
and/or determinism: Politics, morality, and punitiveness. Journal of Personality, 81,
130-141.
Carpenter, C. J. (2020). Meta-analyzing apples and oranges: How to make applesauce instead
of fruit salad. Human Communication Research, 46, 322-333.
Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J. (2019). Correcting for bias in
psychology: A comparison of meta-analytic methods. Advances in Methods and
Practices in Psychological Science, 2, 115-144.
Caruso, G. D. (2014). (Un) just deserts: The dark side of moral responsibility. Southwest
Philosophy Review, 30, 27-38.

*Caspar, E. A., & Vuillaume, L. (2017). The Influence of (Dis) belief in Free Will on
Immoral Behavior. Frontiers in Psychology, 8.
*Ching, B. H.-H., & Xu, J. T. (2018). The effects of gender neuroessentialism on
transprejudice: An experimental study. Sex Roles, 78, 228-241.
Chivers , T. (2010). Neuroscience, free will and determinism: 'I'm just a machine'. The
Telegraph, Retrieved from
http://www.telegraph.co.uk/news/science/8058541/Neuroscience-free-will-and-
determinism-Im-just-a-machine.html.
* Clark, C. J. (2019). Untitled. [Unpublished manuscript].
*Clark, C. J., Baumeister, R. F., & Ditto, P. H. (2017). Making punishment palatable: Belief
in free will alleviates punitive distress. Consciousness and cognition, 51, 193-211.
Cracco, E., Bardi, L., Desmet, C., Genschow, O., Rigoni, D., De Coster, L., . . . Brass, M.
(2018). Automatic Imitation: A Meta-Analysis. Psychological Bulletin, 5, 453-500.
Cracco, E., González-García, C., Hussey, I., Braem, S., & Wisniewski, D. (2020). Cultural
pressure and biased responding in free will attitudes. Royal Society Open Science, 7,
191824.
*Crescioni, A. W., Baumeister, R. F., Ainsworth, S. E., Ent, M., & Lambert, N. M. (2016).
Subjective correlates and consequences of belief in free will. Philosophical
Crick, F. (1994). The Astonishing Hypothesis: The Science Search for the Soul. New York:
Touchstone.
*Crone, D. L., & Levy, N. L. (2019). Are free will believers nicer people?(Four studies
suggest not). Social Psychological and Personality Science, 10, 612-619.
*Darrall, A. (2017). Freiheit! Für unsere Handlungen, aber nicht für unseren Willen.
Konsequenzen einer philosophischen Theorie zum freien Willen. [Unpublished Master
thesis].
Dennett, D. C. (2015). Elbow room: The varieties of free will worth wanting: MIT Press.
*De Souter, L. (2017). The influence of belief in free will on pseudo-random behavior.
[Unpublished Master thesis].
*Diel, K., & Genschow, O. (2018). Untitled. [Unpublished manuscript].
Doyen, S., Klein, O., Pichon, C.-L., & Cleeremans, A. (2012). Behavioral priming: it's all in
the mind, but whose mind? PloS one, 7, e29081.
Dubljević, V. (2013). Autonomy in neuroethics: Political and not metaphysical. AJOB
Neuroscience, 4, 44-51.
Eben, C., Zhang, C., Cracco, E., Brass, M., Billieux, J., & Verbruggen, F. (2020). Are post-
error adjustments influenced by beliefs in free will? A failure to directly replicate
Rigoni, Wilquin, Brass and Burle, 2013. Royal Society Open Science, 7, 200664.
*Evans, J. (2013). The moral psychology of determinism. Philosophical Psychology, 26, 639-
661.
Ewusi-Boisvert, E., & Racine, E. (2018). A critical review of methodologies and results in
recent research on belief in free will. Neuroethics, 11, 1-14.
Feldman, G., Farh, J.-L., & Wong, K. F. E. (2018). Agency beliefs over time and across
cultures: Free will beliefs predict higher job satisfaction. Personality and Social
Psychology Bulletin, 44, 304-317.
Field, A. P., & Gillett, R. (2010). How to do a meta-analysis. British Journal of Mathematical
and Statistical Psychology, 63, 665-694.
Fisher, Z., & Tipton, E. (2015). Robumeta: An R-package for robust variance estimation in
meta-analysis. retrieved from arXiv preprint arXiv:1503.02220.
Frith, C. D. (2012). The role of metacognition in human social interactions. Phil. Trans. R.
Soc. B, 367, 2213-2223.

Furnham, A., Kirkcaldy, B. D., & Lynn, R. (1994). National attitudes to competitiveness,
money, and work among young people: First, second, and third world differences.
Human Relations, 47, 119-132.
*Genschow, O. (2019). [Unpublished manuscript].
Genschow, O., Hawickhorst, H., Rigoni, D., Aschermann, E., & Brass, M. (2020).
Professional judges’ disbelief in free will and punishment. Social Psychological and
Personality Science, 12, 357-362.
Genschow, O., & Lange, J. (in press). Belief in free will affects internal attribution in self-
perception. Social Psychological and Personality Science.
*Genschow, O., Rigoni, D., & Brass, M. (2014). Untitled. [Unpublished manuscript].
*Genschow, O., Rigoni, D., & Brass, M. (2016a). Untitled. [Unpublished manuscript].
*Genschow, O., Rigoni, D., & Brass, M. (2016b). Untitled. [Unpublished manuscript].
*Genschow, O., Rigoni, D., & Brass, M. (2017a). Belief in free will affects causal attributions
when judging others’ behavior. Proceedings of the National Academy of Sciences,
114, 10071-10076.
*Genschow, O., Rigoni, D., & Brass, M. (2017b). Untitled. [Unpublished manuscript].
Genschow, O., Rigoni, D., & Brass, M. (2019a). The hand of god or the hand of Maradona?
Believing in free will increases perceived intentionality of others’ behavior.
Consciousness and cognition, 70, 80-87.
*Genschow, O., Rigoni, D., & Brass. M. (2019b). [Unpublished manuscript].
*Genschow, O., & Vehlow, B. (2021). Free to blame? The role of free will belief in victim
blaming. Consciousness and cognition, 88, 103074.
Genschow, O., Westfal, M., Crusius, J., Bartosch, L., Feikes, K. I., Pallasch, N., & Wozniak,
M. (2021). Does social psychology persist over half a century? A direct replication of
Cialdini et al.’s (1975) classic door-in-the-face technique. Journal of Personality and
Social Psychology, 120, e1–e7.

Giner-Sorolla, R., Embley, J., & Johnson, L. (2016). Replication of Vohs & Schooler (2008,
PS, Study 1). Retrieved from osf.io/i29mh.
*Goodyear, K., Lee, M. R., O’Hara, M., Chernyak, S., Walter, H., Parasuraman, R., &
Krueger, F. (2016). Oxytocin influences intuitions about the relationship between
belief in free will and moral responsibility. Social Neuroscience, 11, 88-96.
Greene, J., & Cohen, J. (2004). For the law, neuroscience changes nothing and everything.
Philos Trans R Soc Lond B Biol Sci, 359, 1775-1785.
Griffin, A. (2016). Free will could all be an illusion, scientists suggest after study shows
choice may just be brain tricking itself. Independent, Retrieved from
http://www.independent.co.uk/news/science/free-will-could-all-be-an-illusion-
scientists-suggest-after-study-that-shows-choice-could-just-be-a7008181.html.
Hallett, M. (2007). Volitional control of movement: the physiology of free will. Clinical
Neurophysiology, 118, 1179-1192.
*Hanisch, A. (2015). Frei handeln oder frei wollen? Experimentelle Manipulation des
Glaubens an Willens- und Handlungsfreiheit. [Unpublished Bachelor thesis].
*Hannikainen, I., Cabral, G., Machery, E., & Struchiner, N. (2017). A deterministic
worldview promotes approval of state paternalism. Journal of Experimental Social
*Harms, J., Liket, K., Protzko, J., & Schölmerich, V. (2017). Free to help? An experiment on
free will belief and altruism. PloS one, 12.
*Harpster, R. (2019). Untitled. [Unpublished manuscript].
Harris, S. (2012). Free will. New York: Simon and Schuster.
Hedges, L. V., Tipton, E., & Johnson, M. C. (2010). Robust variance estimation in meta-
regression with dependent effect size estimates. Research synthesis methods, 1, 39-65.
Hempel, S., Miles, J. N., Booth, M. J., Wang, Z., Morton, S. C., & Shekelle, P. G. (2013).
Risk of bias: a simulation study of power to detect study-level moderator effects in
meta-analysis. Systematic reviews, 2, 107.
*Highhouse, S., & Rada, T. B. (2015). Different worldviews explain perceived effectiveness
of different employment tests. International Journal of Selection and Assessment, 23,
109-119.
*Jettinghoff, W. (2017). Untitled. [Unpublished manuscript].
Keller, J. (2005). In genes we trust: the biological component of psychological essentialism
and its relationship to mechanisms of motivated social cognition. Journal of
Personality and Social Psychology, 88, 686-702.
*Lantian, A., & Badea, C. (2019). Untitled. [Unpublished manuscript].
Libet, B., Gleason, C. A., Wright, E. W., & Pearl, D. K. (1983). Time of conscious intention
to act in relation to onset of cerebral activity (readiness-potential) the unconscious
initiation of a freely voluntary act. Brain, 106, 623-642.
Libet, B., Wright, E. W., & Gleason, C. A. (1993). Readiness-potentials preceding
unrestricted’spontaneous’ vs. pre-planned voluntary acts Neurophysiology of
Consciousness (pp. 229-242). Boston, MA.: Birkhäuser.
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA:
SAGE.
*Lynn, M., Van Dessel, P., & Brass, M. (2013). The influence of high-level beliefs on self-
regulatory engagement: evidence from thermal pain stimulation. Frontiers in
Psychology, 4.
* MacKenzie, M. (2015). How does free will belief influence guilt? [Unpublished
manuscript].
*MacKenzie, M. J., Vohs, K. D., & Baumeister, R. F. (2014). You didn’t have to do that:
Belief in free will promotes gratitude. Personality and Social Psychology Bulletin, 40,
1423-1434.
Markus, H. R., & Kitayama, S. (2003). Models of agency: Sociocultural diversity in the
construction of action. In V. Murphy-Berman & J. J. Berman (Eds.), Cross-cultural
differences in perspectives on the self (pp. 18–74). Lincoln, NE: University of
Nebraska Press.
Martin, N. D., Rigoni, D., & Vohs, K. D. (2017). Free Will Beliefs Predict Attitudes toward
Unethical Behavior and Criminal Punishment: A Global Analysis. Proceedings of the
National Academy of Sciences, 114, 7325-7330.
*Melcher, W. (2016). Untitled. [Unpublished manuscript].
*Melcher, W. (2019). Free will in psychological research: considerations on methodic
procedure and reproducibility of results. Lüneburg, Germany: Leuphana University
Lüneburg.
*Mercier, B. (2017). Untitled. [Unpublished manuscript].
*Mercier, B., Wiwad, D., Piff, P. K., Aknin, L. B., Robinson, A. R., Shariff, A., & Vazire, S.
(2020). Does Belief in Free Will Increase Support for Economic Inequality?.
Collabra: Psychology, 6, 25.
*Monroe, A. E. (2018). Untitled. [Unpublished manuscript].
*Monroe, A. E., Brady, G., & Malle, B. F. (2017). This Isn’t the Free Will Worth Looking
For General Free Will Beliefs Do Not Influence Moral Judgments, Agent-Specific
Choice Ascriptions Do. Social Psychological and Personality Science, 191-199.
Monroe, A. E., & Malle, B. F. (2010a). From uncaused will to conscious choice: The need to
study, not speculate about people’s folk concept of free will. Review of philosophy and
psychology, 1, 211-224.
*Monroe, A. E., & Malle, B. F. (2010b). Untitled. [Unpublished manuscript].

*Moynihan, A. B., Igou, E. R. (2019). Untitled. [Unpublished manuscript].
*Moynihan, A. B., Igou, E. R., & van Tilburg, W. A. (2019). Lost in the crowd: Conformity
as escape following disbelief in free will. European Journal of Social Psychology, 49,
503-520.
Murray, D., & Nahmias, E. (2014). Explaining away incompatibilist intuitions. Philosophy
and Phenomenological Research, 88, 434-467.
*Nadelhoffer, T. (2012). Untitled. [Unpublished manuscript].
Nadelhoffer, T. (2011). The threat of shrinking agency and free will disillusionism. Conscious
will and responsibility: A tribute to Benjamin Libet, 173-188.
*Nadelhoffer, T., Shepard, J., Crone, D. L., Everett, J. A., Earp, B. D., & Levy, N. (2020).
Does encouraging a belief in determinism increase cheating? Reconsidering the value
of believing in free will. Cognition, 203, 104342.
Nadelhoffer, T., Shepard, J., Nahmias, E., Sripada, C., & Ross, L. T. (2014). The free will
inventory: Measuring beliefs about agency and responsibility. Consciousness and
cognition, 25, 27-41.
Nahmias, E., Morris, S., Nadelhoffer, T., & Turner, J. (2005). Surveying freedom: Folk
intuitions about free will and moral responsibility. Philosophical Psychology, 18, 561-
584.
Nahmias, E., Morris, S. G., Nadelhoffer, T., & Turner, J. (2006). Is incompatibilism intuitive?
Philosophy and Phenomenological Research, 73, 28-53.
Nichols, S. (2004). The folk psychology of free will: Fits and starts. Mind & Language, 19,
473-502.
Nichols, S. (2006). Folk intuitions on free will. Journal of Cognition and Culture, 6, 57-86.
Nichols, S., & Knobe, J. (2007). Moral responsibility and determinism: The cognitive science
of folk intuitions. Nous, 41, 663-685.

*Open Science Collaboration. (2015). Estimating the reproducibility of psychological
science. Science, 349, aac4716.
Paulhus, D. L., & Carey, J. M. (2011). The FAD–Plus: Measuring lay beliefs regarding free
will and related constructs. Journal of Personality Assessment, 93, 96-104.
Pereboom, D. (2006). Living without free will. New York: Cambridge University Press.
Petty, R. E., & Krosnick, J. A. (2014). Attitude strength: Antecedents and consequences. New
York: Psychology Press.
*Plaks, J. E., & Robinson, J. S. (2015). Construal level and free will beliefs shape perceptions
of actors' proximal and distal intent. Frontiers in Psychology, 6, 777.
Polanin, J. R., Tanner-Smith, E. E., & Hennessy, E. A. (2016). Estimating the difference
between published and unpublished effect sizes: A meta-review. Review of
Educational Research, 86, 207-236.
*Protzko, J., & Genschow, O. (2018). Untitled. [Unpublished manuscript].
Protzko, J., Ouimette, B., & Schooler, J. (2015). Believing There Is No Free Will Corrupts
Intuitive Cooperation. Available at SSRN 2490855.
*Protzko, J., Ouimette, B., & Schooler, J. (2016). Believing there is no free will corrupts
intuitive cooperation. Cognition, 151, 6-9.
Racine, E., Nguyen, V., Saigle, V., & Dubljevic, V. (2017). Media portrayal of a landmark
neuroscience experiment on free will. Science and Engineering Ethics, 23, 989-1007.
Rakos, R. F., Laurene, K. R., Skala, S., & Slane, S. (2008). Belief in free will: Measurement
and conceptualization innovations. Behavior and Social Issues, 17, 20-40.
*Rangel, U., & Keller, J. (2011). Essentialism goes social: Belief in social determinism as a
component of psychological essentialism. Journal of Personality and Social
Psychology, 100, 1056-1078.
*Rigoni, D. (n.a.). Untitled. [Unpublished manuscript]

Rigoni, D., & Brass, M. (2014). From intentions to neurons: social and neural consequences
of disbelieving in free will. Topoi, 33, 5-12.
*Rigoni, D., Kühn, S., Gaudino, G., Sartori, G., & Brass, M. (2012). Reducing self-control by
weakening belief in free will. Consciousness and cognition, 21, 1482-1490.
*Rigoni, D., Kühn, S., Sartori, G., & Brass, M. (2011). Inducing disbelief in free will alters
brain correlates of preconscious motor preparation the brain minds whether we believe
in free will or not. Psychological Science, 22, 613-618.
*Rigoni, D., Pourtois, G., & Brass, M. (2015). ‘Why should I care?’Challenging free will
attenuates neural reaction to errors. Social Cognitive and Affective Neuroscience, 10,
262-268.
*Rigoni, D., Trager, B. M., Rys, W., Brass, M., Vallacher, R., & Vohs, K. D. (2016). The
importance of being earnest: valuing honesty attenuates anti-social consequences of
disbelieving in free will. [Unpublished manuscript].
*Rigoni, D., Wilquin, H., Brass, M., & Burle, B. (2013). When errors do not matter:
Weakening belief in intentional control impairs cognitive reaction to errors.
Cognition, 127, 264-269.
Rodgers, M. A., & Pustejovsky, J. E. (2020). Evaluating meta-analytic methods to detect
selective reporting in the presence of dependent effect sizes. Psychological Methods,
26, 141–160.
Rose, D., & Nichols, S. (2013). The lesson of bypassing. Review of philosophy and
psychology, 4, 599-619.
Rotter, J. B. (1966). Generalized expectancies for internal versus external control of
reinforcement. Psychological monographs: General and applied, 80, 1-28.
Saigle, V., Dubljević, V., & Racine, E. (2018). The impact of a landmark neuroscience study
on free will: A qualitative analysis of articles using Libet and colleagues' methods.
AJOB Neuroscience, 9, 29-41.

Sarkissian, H., Chatterjee, A., De Brigard, F., Knobe, J., Nichols, S., & Sirker, S. (2010). Is
belief in free will a cultural universal? Mind & Language, 25, 346-358.
Schooler, J., Nadelhoffer, T., Nahmias, E., & Vohs, K. D. (2014). Measuring and
manipulating beliefs and behaviors associated with free will. In A. R. Mele (Ed.),
Surrounding free will: Philosophy, psychology, neuroscience (pp. 72-94). Oxford:
Oxford University Press.
*Schooler, J.W. (2014). Untitled. [Unpublished manuscript].
Schooler, J. W. (2010). What Science Tells Us about Free Will. In R. F. Baumeister, A. R.
Mele & K. D. Vohs (Eds.), Free will and consciousness: How might they work? (pp.
191–218). Oxford, England: Oxford University Press.
*Schrag, Y. (n.a.). Untitled. [Unpublished manuscript].
*Schrag, Y., Tremea, A., Lagger, C., Ohana, N., & Mohr, C. (2016). Pro Free Will Priming
Enhances “Risk-Taking” Behavior in the Iowa Gambling Task, but Not in the Balloon
Analogue Risk Task: Two Independent Priming Studies. PloS one, 11, e0152297.
*Seminck, N. (2019). The influence of belief in free will on imitation of inappropriate
behavior [Unpublished Master thesis].
*Seto, E. (n.a.). Untitled. [Unpublished manuscript].
*Seto, E. (2017). The interplay of true self-knowledge on belief in free will and moral
outcomes. [Unpublished dissertation].
*Seto, E., & Hicks, J. A. (2016). Disassociating the agent from the self: Undermining belief
in free will diminishes true self-knowledge. Social Psychological and Personality
Science, 7, 726-734.
*Seto, E., Kim, J., & Hicks, J. A. (2020). The Illusion of Time: Testing the Bidirectional
Relationship Between Belief in Free Will and Temporal Horizons. Social
Psychological and Personality Science, 12, 176-185.

*Shariff, A. F., Greene, J. D., Karremans, J. C., Luguri, J. B., Clark, C. J., Schooler, J. W., . . .
Vohs, K. D. (2014). Free will and punishment: A mechanistic view of human nature
reduces retribution. Psychological Science, 25, 1563-1570.
Shariff, A. F., & Vohs, K. D. (2014). The World Without Free Will. Scientific American, 310,
76-79.
Shepard, J., & Reuter, S. (2012). Neuroscience, choice, and the free will debate. AJOB
Neuroscience, 3, 7-11.
Shepherd, J. (2012). Free will and consciousness: Experimental studies. Consciousness and
cognition, 21, 915-927.
*Siller, L. (2019). Glaube an den freien Willen: Halten sich die Effekte einer experimentellen
Manipulation über die Zeit? [Unpublished Master thesis].
Simonsohn, U., Nelson, L., & Simmons, J. (2015). Official user-guide to the P-curve.
Retrieved from www.p-curve.com/guide.pdf. Retrieved 29.05., 2020
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014). P-curve: a key to the file-drawer.
Journal of Experimental Psychology: General, 143, 534-547.
Slamecka, N. J., & Graf, P. (1978). The generation effect: Delineation of a phenomenon.
Journal of experimental Psychology: Human learning and Memory, 4, 592-604.
Smilansky, S. (2000). Free will and illusion. New York: Oxford University Press.
Smilansky, S. (2002). Free will, fundamental dualism, and the centrality of illusion. In R.
Kane (Ed.), The Oxford handbook of free will. New York: Oxford University Press.
*Snoeck, A. (2019). The effect of disbelief in free will on predjudice toward Arabic people.
[Unpublished Master thesis].
Soon, C. S., Brass, M., Heinze, H.-J., & Haynes, J.-D. (2008). Unconscious determinants of
free decisions in the human brain. Nature Neuroscience, 11, 543.
Stanley, T. D. (2017). Limitations of PET-PEESE and other meta-analysis methods. Social
Psychological and Personality Science, 8, 581-591.

Stanley, T. D., & Doucouliagos, H. (2014). Meta-regression approximations to reduce
publication selection bias. Research synthesis methods, 5, 60-78.
Sterne, J. A. C., & Egger, M. (2005). Regression methods to detect publication and other bias
in meta-analysis. In H. R. Rothstein, A. J. Sutton & M. Borenstein (Eds.), Publication
bias in meta-analysis: Prevention, assessment and adjustments (pp. 99-110).
Chichester, England: Wiley.
*Stillman, T. F., & Baumeister, R. F. (2010). Guilty, free, and wise: Determinism and
psychopathy diminish learning from negative emotions. Journal of Experimental
Social Psychology, 46, 951-960.
Stillman, T. F., Baumeister, R. F., Vohs, K. D., Lambert, N. M., Fincham, F. D., & Brewer, L.
E. (2010). Personal philosophy and personnel achievement: Belief in free will predicts
better job performance. Social Psychological and Personality Science, 1, 43-50.
Stroebe, W., & Strack, F. (2014). The alleged crisis and the illusion of exact replication.
Perspectives on Psychological Science, 9, 59-71.
Tanner-Smith, E. E., Tipton, E., & Polanin, J. R. (2016). Handling complex meta-analytic
data structures using robust variance estimates: A tutorial in R. Journal of
Developmental and Life-Course Criminology, 2, 85-112.
Tanner-Smith, E. E., & Tipton, E. (2014). Robust variance estimation with dependent effect
sizes: Practical considerations including a software tutorial in Stata and SPSS.
Research synthesis methods, 5, 13-30.
*Thelen, A. (2018). Der Einfluss des Glaubens an einen freien Willen auf selbstwertdienliche
Verzerrungen. [Unpublished Mater thesis].
Tipton, E. (2015). Small sample adjustments for robust variance estimation with meta-
regression. Psychological Methods, 20, 375.

Tipton, E., & Pustejovsky, J. E. (2015). Small-sample adjustments for tests of moderators and
model fit using robust variance estimation in meta-regression. Journal of Educational
and Behavioral Statistics, 40, 604-634.
*Trager, B. (2016). Untitled. [Unpublished manuscript].
Van Bavel, J. J., Mende-Siedlecki, P., Brady, W. J., & Reinero, D. A. (2016). Contextual
sensitivity in scientific reproducibility. Proceedings of the National Academy of
Sciences, 113, 6454-6459.
*Van Dessel, P. (2012). Untitled. [Unpublished manuscript].
Van Inwagen, P. (1983). An essay on free will. London: MIT Press.
*Vassena, E. (2019). Untitled. [Unpublished manuscript].
Velten, E. (1968). A laboratory task for induction of mood states. Behaviour Research and
Therapy, 6, 473-482.
Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of
statistical software, 36, 1-48.
Viney, W., Waldman, D. A., & Barchilon, J. (1982). Attitudes toward punishment in relation
to beliefs in free will and determinism. Human Relations, 35, 939-949.
*Vohs, K. D., & Schooler, J. W. (2008). The value of believing in free will encouraging a
belief in determinism increases cheating. Psychological Science, 19, 49-54.
*Vonasch, A. J., Clark, C. J., Lau, S., Vohs, K. D., & Baumeister, R. F. (2017). Ordinary
people associate addiction with loss of free will. Addictive behaviors reports, 5, 56-66.
Waldman, D. A., Viney, W., Bell, P. A., Bennett, J. B., & Hess, S. (1983). Internal and
external locus of control in relation to beliefs in free will and determinism.
Psychological Reports, 53, 631-634.

Webb, T. L., & Sheeran, P. (2006). Does changing behavioral intentions engender behavior
change? A meta-analysis of the experimental evidence. Psychological Bulletin, 132,
249-268.
Wegner, D. (2002). The illusion of free will: Cambridge, MA: MIT Press.
Wisniewski, D., Deutschländer, R., & Haynes, J.-D. (2019). Free will beliefs are better
predicted by dualism than determinism beliefs across different cultures. PloS one, 14,
e0221617.
Wolfe, T. (1997). Sorry, but your soul just died. Forbes, Retrieved from
http://90.146.148.118/en/archiv_files/19971/E11997_19236.pdf.
*Zhao, X., Liu, L., Zhang, X.-x., Shi, J.-x., & Huang, Z.-w. (2014). The effect of belief in free
will on prejudice. PloS one, 9, e91572.
*Zwaan, R. A. (n.a.) Untitled. [Unpublished manuscript].
*Zwaan, R. A. (2014). The Value of Believing in Free Will: A Replication Attempt
Zeistgeist: Psychological Experimentation, Cognition, Language, and Academia.
Retrieved 30.11.2019, 2019, from https://rolfzwaan.blogspot.com/2013/03/the-value-
of-believing-in-free-will.html

FWB MetaAnalysis

Uploaded by

Copyright:

Available Formats

FWB MetaAnalysis

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

FWB MetaAnalysis

Uploaded by

Copyright:

Available Formats

Manipulating belief in free will and its downstream consequences:

Oliver Genschow* (University of Cologne, Germany)

Emiel Cracco* (Ghent University, Belgium)

Jana Schneider (University of Cologne, Germany)

John Protzko (Central Connecticut State University, USA)

David Wisniewski (Ghent University, Belgium)

Marcel Brass (Humboldt University Berlin, Germany)

Jonathan W. Schooler (University of California Santa Barbary, USA)

accepted manuscript at Personality and Social Psychology Review before copyediting

Germany, E-Mail: oliver.genschow@uni-koeln.de

Psychological research has investigated this question by testing the consequences of

downstream consequences? In a meta-analysis including 145 experiments (95 unpublished),

experimental manipulations, and its downstream consequences. Then, we test meta-

manipulations have downstream consequences.

Belief in Free Will and Determinism

(Paulhus & Carey, 2011).

defined belief in determinism in different ways. On a general level, determinism is typically

Irrespective of how determinism is defined, psychologists and philosophers debate

and determinism as independent constructs. Based on a compatibilistic view, people believe

Research on Manipulating Belief in Free Will

To test the consequences of (dis)believing in free will, researchers have developed

retributive punishment (Shariff et al., 2014).

In addition, anti-free will manipulations have been found to increase conformity

attributions of other people’s actions (Genschow et al., 2017a), perceived meaningfulness of

processing of performance errors (Rigoni et al., 2015; Rigoni et al., 2013).

diminishing participants’ belief in free will on moral behavior, judgments of blame, or

consequences. In the present article, we investigate these explanations by analyzing all

available evidence in a meta-analysis.

Previous Reviews and Meta-Analyses

amount of methodological diversity and a lack of replication studies in the published

The Present Meta-Analysis

any downstream consequences?

Research Question 1: Can Belief in Free Will Be Manipulated?

In the first part of the meta-analysis, we investigate whether it is possible to

the manipulations are effective. To this end, we investigated different moderators.

compare them to each other.

questionnaires, such as belief in dualism, fatalistic determinism, and unpredictability.

method is to present participants with several statements using a Velten-like technique

compared them in the present meta-analysis.

manipulation but also on participants’ involvement in the manipulation. While some

succeed. To answer this question, we investigated if the effectiveness of the manipulation

rewriting presented messages) before free will beliefs are measured.

will can also be experimentally increased, in addition to decreased.

Secondary Moderator Analyses

In addition to the above-mentioned moderators, we also considered the influence of

Research Question 2: Do Free Will Belief Manipulations Have Downstream

In the second part of the meta-analysis, we investigate whether experimentally

2020). To deal with this issue, we proceeded in three steps.

form a meaningful whole (Simonsohn et al., 2015).

In a second step, we ran meta-analyses on internally coherent sets of dependent

conformity, punishment, and cheating.

AND “undermin*”) OR (“Free will” AND “weak*”) OR (“Determinis*” AND “belie*”) OR

(“Determinis*” AND “manipulat*”) OR (“Determinis*” AND “experiment*”) OR

(“Determinis*” AND “group”) OR (“Determinis*” AND “induc*”) OR (“Determinis*” AND

“encourag*”) OR (“Determinis*” AND “increas*”) OR (“Determinis*” AND “enhanc*”).

unpublished data was terminated on August 15, 2019.

in the PRISMA flow chart in Figure 1.

1. The studied population had to involve healthy adults.

increasing belief in free will or belief in determinism.

AND “undermin”) OR (“Free will” AND “weak”) OR (“Determinis” AND “belie”) OR

(“Determinis” AND “manipulat”) OR (“Determinis” AND “experiment”) OR

(“Determinis” AND “group”) OR (“Determinis” AND “induc”) OR (“Determinis” AND

“encourag”) OR (“Determinis” AND “increas”) OR (“Determinis” AND “enhanc*”).