SLR 2 JP

15206793, 2022, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mar.21657 by Indian Institute Of Management, Wiley Online Library on [24/10/2022].
See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Received: 19 July 2021 | Accepted: 1 March 2022
DOI: 10.1002/mar.21657
REVIEW ARTICLE
Meta‐analysis and traditional systematic literature

reviews—What, why, when, where, and how?
Justin Paul1,2 | Mojtaba Barari3
1
University of Reading, Henley Business
School, United Kingdom Abstract
2
Graduate School of Business, University of Meta‐analysis is a research method for systematically combining and synthesizing
Puerto Rico, San Juan, Puerto Rico, USA
findings from multiple quantitative studies in a research domain. Despite its
3
Department of Marketing, Griffith Business
School, Griffith University, Gold Coast, importance, most literature evaluating meta‐analyses are based on data analysis and
Australia statistical discussions. This paper takes a holistic view, comparing meta‐analyses to
traditional systematic literature reviews. We described steps of the meta‐analytic
Correspondence
Justin Paul, Graduate School of Business, process including question definition, data collection, data analysis, and reporting
University of Puerto Rico, San Juan, PR
results. For each step, we explain the primary purpose, the tasks required of the
00907, USA.
Email: j.paul@reading.ac.uk and meta‐analyst, and recommendations for best practice. Finally, we discuss recent
profjust@gmail.com
developments in meta‐analytic techniques, which increase its effectiveness in
business research.
KEYWORDS
meta‐analysis, quantitative review, research synthesis, systematic literature review
1 | INTRODUCTION domain (Card, 2015). SLRs allow readers to glean a deep under-
standing of literature and also help them to identify research gaps in
Scientific knowledge is based on the accumulated findings from prior the area (Paul & Criado, 2020). In this way, an SLR may be viewed as a
research in a research domain, where individual studies constitute platform for knowledge advancement (Palmatier et al., 2018).
building blocks (Rowley & Paul, 2021). It is expected that a new study In contrast to traditional SLR, a meta‐analysis takes an objective
will build on previous findings to contribute to knowledge formation approach to quantitively synthesizing studies in a research domain,
and development in a research domain (Grewal et al., 2018). To while a traditional SLR is qualitative and subjective in nature (Card,
accomplish this, authors must define their research objectives based 2015). Meta‐analysis statistical assesses the robustness of findings in
on gaps in the relevant literature, and design a study to addresses this an area and identifies and resolves conflicting findings in past
gap (Paul et al., 2021). This requires deep knowledge and under- research to provide more clarity on the topic for scholars and
standing of a research domain, which can be facilitated by systematic practitioners (Grewal et al., 2018). The advantage of meta‐analysis,
literature reviews (SLRs). compared to a single study with small sample size, is its higher power
SLRs can provide authors with an overview of a research domain (i.e., combined sample size of individual studies) (Cooper, 2015),
in a single paper (Rosado‐Serrano et al., 2018; Keupp & Gassmann, which enables the meta‐analyst more conclusively characterize the
2009; Paul et al., 2017). This method is considered to be a scientific relationships between variables in a domain, and variables which
and highly informative method for systematically collecting, review- moderate these relationships (Littell et al., 2008). While meta‐analysis
ing, and synthesizing research findings on a particular topic (Paul was introduced in the 1970s as a method to synthesize prior
et al., 2021) to determine what is known –and what is not known—at research, its acceptance as tool for advancing knowledge
This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium,
provided the original work is properly cited.
© 2022 The Authors. Psychology & Marketing published by Wiley Periodicals LLC.
Psychol Mark. 2022;39:1099–1115. wileyonlinelibrary.com/journal/mar | 1099

15206793, 2022, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mar.21657 by Indian Institute Of Management, Wiley Online Library on [24/10/2022]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
1100 | PAUL AND BARARI
development among researchers in the business and management 2.1 | Traditional SLRs—What and why?
fields has been relatively recent (Aguinis, Dalton, et al., 2011).
Meta‐analysis follows very technical and sophisticated procedures A traditional SLR is a “process for assembling, arranging, and
to collect, combine, and analyze empirical research (Siddaway et al., assessing existing literature in a research domain” (Paul et al.,
2019). This rigorous approach guarantees the validity and reliability of 2021). In this process, assembling involves identification (i.e., defining
the method, while at the same time obfuscating the technique for the the literature review domain, main question, and source type/quality)
researchers and practitioners who could benefit from conducting meta‐ and acquisition (i.e., obtaining papers to be included). The scientific
analyses. Given this, the primary objective of this manuscript is to detail steps for an SLR include organization (i.e., specifying the codes and
current practices and advancements in meta‐analysis research and to framework) and purification (i.e., specifying the inclusion and
contrast the technique with traditional SLRs. We eschew technical exclusion criteria). The final step is setting the future research
jargon to enhance the accessibility of our research among readers who agenda, based on a gap analysis (Littell et al., 2008). Following this
do not possess advanced statistical knowledge. Moreover, instead of process helps researchers to meet two main goals: (1) providing a
focusing exclusively on data collection and analysis, we cover the entire comprehensive picture of what is known in a research domain (i.e.,
meta‐analytic process including question definition, data collection, data defining its scope overview, identifying inconsistencies, and their
analysis, and presentation of results. Finally, we will introduce recent probable explanation, and developing a framework to summarize
advancements in meta‐analytic technique to demonstrate the ongoing previous research); and, (2) providing directions for future research
developments. based on what is not known in that research domain (Paul &
Criado, 2020).
Traditional SLRs include: domain‐based reviews; theory‐based
2 | T R A D I T I O N A L SL R s A N D reviews, and method‐based reviews. Domain‐based reviews synthe-
M E T A‐A NAL YS I S size studies in the same research domain to extend the body of
literature in this domain. Synthesizing diverse perspectives allows
There are several fundamental differences between traditional SLRs authors to describe state‐of‐the‐art knowledge in the research
and meta‐analysis, distinguishing these popular methods for accumu- domain and identify useful paths for research. There are several
lating knowledge in a research domain. Table 1 provides an overview types of domain‐based reviews:
of these differences based on five key questions (i.e., What, Why, Structured reviews focus on the development of a research
When, Where, and How). We believe these key questions provide a domain such as associated theories, models, constructs, contexts, and
better picture of both traditional SLRs and meta‐analysis and their methods in which authors use tables and figures to provide
difference and help researchers choose an appropriate literature structured insight into a research domain (Paul & Criado, 2020).
review method in their work. Therefore, the rest of this study aims to Examples include a structured review of entry mode by Canabal and
extend these differences in more detail. White III (2008) and export barriers by Kahiya (2018).
TABLE 1 Differences between traditional SLRs and meta‐analysis
Differences Traditional SLRs Meta‐analysis
What A qualitative process for assembling, arranging, and assessing A quantitative method that integrates the results of empirical
existing literature in a research domain. studies to provide an aggregate summary of findings in a
research domain.
Why Providing a comprehensive picture of what is known and To achieve statistically precise and accurate conclusions about
proposing directions for future research based on what is not the strength and direction of a relationship between variables
known in that research domain. and moderator role in a research domain.
When The research topic is evolving to allow a researcher to provide a The research topic is mature enough to allow a researcher to
current view of what is known and define the future direction provide an overall picture of relationships and the role of
of the research domain. moderators in a research domain.
Where Include all types of relevant studies from high‐quality journals Include published and unpublished studies that empirically
through subjective selection and interpretation of data to examine the relationships of interest through objective and
synthesize the findings of prior studies in a systematic manner. rigorous statistical procedures to synthesize the findings of
prior studies and test hypotheses that have not been studied
in prior research.
How Through SLRs process include defining a research question, data Through meta‐analysis process include defining the research
collection, data preparation, data analysis, and reporting and question, data collection, data preparation, data analysis, and
using the SLRs approach, that is, domain‐based reviews, reporting and meta‐analysis approach, that is, main model and
theory‐based reviews, and method‐based reviews. moderator analysis.
Abbreviation: SLR, systematic literature review.

PAUL AND BARARI | 1101
Framework‐based reviews are papers developed using a classic 2.2 | Meta‐analysis: What and why?
organizing framework to scientifically synthesize information, as
illustrated by Paul and Benito's (2018) ADO (Antecedents, Decisions, Meta‐analysis is a collection of statistical methods that integrates the
and Outcome) framework. The ADO framework has been used by results of a large number of studies to provide an aggregate summary
Lim et al. (2021) and Södergren (2021). Another approach to of knowledge in a research domain (Littell et al., 2008). The
conducting a framework‐based review, which is utilized by Xie advantage of meta‐analysis over an individual study is in its higher
et al. (2017). Still, another method is Paul and Rosado‐Serrano's power (i.e., sample size; Geyskens et al., 2009). A meta‐analysis
(2019) TCCM framework for writing reviews and others (e.g., Chen combines the findings of single studies for specific relationships, it
et al., 2021). Although these frameworks suggest a different allows authors to achieve statistically precise and accurate conclu-
approach to writing an SLR, they have a common goal: to facilitate sions about the strength and direction of a relationship between
the combining and synthesizing research in a domain. variables (Littell et al., 2008), and to resolve contradictory results in
Bibliometric reviews entail analyzing bibliographic data of prior studies by examining the impact of moderator variables
published literature by using statistical tools to highlight trends and, (Geyskens et al., 2009). Meta‐analysts calculate an “effect size,” which
therefore, provide an overview of the body of knowledge in a indicates the direction and strength of association between two
research domain. Writing bibliometric reviews is facilitated by variables (Card, 2015), and is a standardized metric that is comparable
available software such as Visualization of Similarities (VOS Viewer). across studies (Lipsey & Wilson, 2001). The researcher extracts
However, critics assert that bibliometric reviews do not adequately information from each study comprising the meta‐analytic database
describe theories, methods, and constructs in a particular domain to calculate an effect size from every single study (Geyskens et al.,
(Paul & Criado, 2020). Randhawa et al. (2016) open innovation, Goyal 2009). After calculating an effect size, for each study, the meta‐
and Kumar (2021) financial literacy, and Pattnaik et al. (2020) trade analyst combines all of the effect sizes to determine the strength and
credit are examples of the bibliometric reviews. direction of associations between pairwise relationships at the
Hybrid reviews combine elements of different review types aggregate level (Geyskens et al., 2009). Popular effect sizes in
(Paul & Criado, 2020). For example, some reviewers supplement a business and management are the correlation coefficient (r) and,
bibliometric review with a structured review (Bahoo et al., 2020) and standardized mean difference coefficient (e.g., Hedges's g and
develop a robust hybrid review. For example, Kumar et al. (2020) Cohen's d; Littell et al., 2008).
masstige marketing, Dabić et al. (2020) immigrant entrepreneurship Usually, there are conflicting findings in a literature stream.
and Rebouças and Soares (2021) voluntary simplicity utilized hybrid Therefore, the meta‐analyst identifies appropriate moderators in an
reviews for their domain‐based literature reviews. attempt to explain variations across studies (Card, 2015). Some
Finally, conceptual reviews aim for theory, model, and/or control variables may also be defined to account for other sources of
propositions development in a research domain (Paul et al., 2021). variation in effect sizes. In their meta‐analysis, Rubera and Kirca
For example, Pansari and Kumar (2017) proposed customer engage- (2012) studied the impact of firm innovativeness on firm perform-
ment framework and Paul's (2019) marketing in emerging markets ance. In addition to measuring how firm innovativeness influences
model to enhance the knowledge in the research domain. firm performance, these researchers assess the impact of control
Theory‐based and method‐based SLRs follow a similar process, variables such as product diversification, firm age, intangible factors,
from defining research questions to reporting findings (Cooper, and competitive intensity on their relationships. They also explore the
2015); meta‐analyses feature key differences. While traditional moderating impact of firm size, advertising intensity, industry, and
qualitative literature reviews include all type of relevant studies, country.
meta‐analyses only include studies that empirically examine the
relationships of interest (Geyskens et al., 2009). Moreover, in
qualitative literature reviews, researchers typically focus on 3 | CONDUCTING A META ‐A N A L Y S I S —
publications from higher‐quality journals to increase the quality W HEN?
of output. However, meta‐analyses feature different kind
of manuscripts—both published and unpublished—from variety of Deciding on the right time to conduct meta‐analysis is quite
sources and account for variations in journal tier/quality analyti- challenging (Paul et al., 2021). The research topic covered in the
cally (Barari et al., 2021). Also, in qualitative SLRs, the subjective meta‐analysis must be mature enough to allow a researcher to
selection and interpretation of authors play an important role in include enough homogeneous empirical research in terms of subjects,
developing an analytical framework to integrate prior studies in a interventions, and outcomes (Haidich, 2010). This allows researchers
research domain. In contrast, a meta‐analysis follows an objective to statistically sensitize findings in a domain to provide a state‐of‐the‐
and rigorous statistical procedure which limits the scope of art view and encourage further development in that domain (Grewal
authors' own interpretations (Siddaway et al., 2019). Finally, et al., 2018). When there is not enough empirical research, it might be
meta‐analyses not only synthesize the findings of prior studies, better not to perform a meta‐analytic review. Instead, researchers
but also enable researchers to test hypotheses that have not been could employ qualitative SLRs to synthesize research in that domain
studied in prior research (Lipsey & Wilson, 2001). (Borenstein et al., 2021).
The number of effect sizes for the relationship of interest in a 4.1 | Defining the research question
meta‐analysis conceptual model provides practical guidance about
the right time for conducting meta‐analysis. There is no agreement The first step in a meta‐analysis is defining the main research
among meta‐analysts about the minimum number of effect sizes question (Paul et al., 2021). It is important for authors to have a clear
required for a relationship to include in a meta‐analysis. For instance, vision of the main research question before conducting the meta‐
Pigott (2012) proposed two as the minimum effect size for analysis (Paul & Criado, 2020), because this will impact the entire
conducting a meta‐analysis on a pair. However, it seems two effect process—especially data collection and data analysis. The research
sizes as a threshold could not fully reflect the complexity of questions in a meta‐analysis typically address a focal concept (e.g.,
relationships (Palmatier et al., 2006). Thus, other researchers offer commitment across cultures; Fischer & Mansell, 2009), examining
a higher required effect size as a threshold. For instance, in the study relationships between two or more focal concepts (e.g., corporate
of the role of customer relational benefits on customer response, social performance and financial performance; Orlitzky et al., 2003;
Gremler et al. (2020) only considered relationships with at least five or investigate antecedents and consequences of a focal concept or
effect sizes in their study. construct (e.g., antecedents and consequences of leader‐member
Suppose an existing meta‐analysis has been published in a high‐ exchange; Dulebohn et al., 2012). The research question for a meta‐
quality journal in recent years. In that case, it is not beneficial to analysis could be formulated around specific theory (e.g., regulatory
conduct further meta‐analysis unless the new meta‐analysis could fit theory; Motyka et al., 2014) or model (e.g., technology acceptance
provide substantially new insight into the research domain. For model; King & He, 2006)). Defining a research question in meta‐
instance, Palmatier et al.'s (2006) meta‐analysis studies the effective- analysis requires a deep understanding of the topic and literature, and
ness of factors influencing relationships marketing, providing an entails specifying a valuable, feasible question that prefaces the
excellent overview of the relationship marketing research area. Later, meta‐analytic framework that will guide authors through data
Samaha et al. (2014) used meta‐analysis to study the role of culture in collection and analysis.
relationship marketing in the international context, which has not
been studied in previous meta‐analyses. Furthermore, Verma et al.
(2016) used meta‐analysis to study relationship marketing in an 4.1.1 | In‐depth understanding of research
online context. topic/literature stream
Meta‐analysis provides an overview of relationships in a research area,

4 | M E TA‐ A N A LY S IS S T R U CT U R E—W H E R E resolves conflicts, and identifies directions for further research in a
AND HOW? mature research domain (Borenstein et al., 2021). The meta‐analyst
must possess an overarching knowledge of the current research domain
Meta‐analysts make critical decisions at each step of the meta‐ (Paul & Criado, 2020) to define an appropriate research question and
analytic process (Geyskens et al., 2009). As shown in Table 2, steps in develop a conceptual framework (Paul et al., 2021). With this in mind,
a meta‐analysis include defining the research question, data collec- authors might begin by reading highly cited papers, to gain an
tion, data preparation, data analysis, and reporting results. understanding of the topic and extant research. Researchers might also
Table 3 details the main goal of each step and its substeps to review qualitative SLRs to enrich their understanding.
provide an overview of the meta‐analysis process. Importantly,
although the steps are invariable, how the researcher performs each
step and make judgment category throughout, is subjective (Grewal 4.1.2 | Specify a valuable and feasible question
et al., 2018). However, we offer suggestions to make guide
researchers' decision‐making. In the sections that follow, we describe The quality of Meta‐analytic output depends on the quality of the
the tasks of the meta‐analyst in each step of the process. research question (Paul & Criado, 2020). Valuable research questions
TABLE 2 Meta‐analysis process
Step Purpose
Defining research question Formulating the main research question to be considered in the meta‐analysis framework development
Data collection Collecting all related and qualified quantitative studies from the literature that are matched with the research question
Data preparation Extracting and checking effect size and required data from individual studies for main and moderator/control analysis
Data analysis Use corrected and combined effect size to test the relationship in the meta‐analysis framework and do moderator and
control variable analysis
Reporting Report all required details in a logical formation to reflect completeness and transparency of meta‐analysis review
TABLE 3 Meta‐analysis process
Step Suggestions
Defining research question
Depth of knowledge of an area • Read related publications in high impact factor journals
• Review current qualitative literature review in a research domain
Specify a valuable and feasible • Define a question that helps to resolve contradictory results and add new insight through the main
question model and moderator analysis
• Needs to be broad enough to include enough quantitative studies and limited enough to define a unique
position in the research domain
Develop meta‐analysis framework • Define focal concepts, their conceptual and operational definitions
• Define the relationships between concepts and variables which moderate them
• Use a logical model or theory to underpin relationships in a meta‐analysis framework
Data collection
Searching literature • Define keyword search in related or unrelated research field

• Define popular databases for search
• Manually review the top journals' issues
• Check the references of high cited papers
• Communicate with top researchers and Post requests on academic list services
Define inclusion/exclusion • Include quantitative research which tested desired relationships

• Include research in a specific time frame
• Specify other criteria based on the meta‐analysis framework
Data coding and final sample • Develop a coding manual to code pairwise relationships, moderator, and control variables
description • Employ at least two coders in studies coding
• Calculate inter‐coder agreement, report discrepancies, and strategies to deal with them
Data preparation
Effect size selection and extraction • Correlation and standard mean differences are common effect sizes in meta‐analysis
• The research design of most original studies determines the effect size metric
• When the effect size value is not reported, use available information in the original study to calculate
the effect size
Effect size extraction issues • In the original study, effect sizes from the same sample need to be combined
• In the original study, effect sizes from the independent samples are independent
Effect size correction • The correct effect size for measurement error, range restriction, and dichotomization
Deal with outliers • Use sample‐adjusted meta‐analytic deviancy statistic (SAMD) for outlier identification
• Conduct sensitivity analysis to see the role of outliers on the result and decide about how to deal with
outliers
Publication bias • Use techniques such as file drawer N, win fail‐safe N, or trim and fill to assess publication bias
Effect size combination • Use weighted average to combine effect sizes from individual studies
Moderator and control extraction • Assign a quantity (categorical or continuous) to these variables based on available data in the original studies
Data analysis
Model selection • Choose fixed‐effects model or random‐effects model, however, the latter is preferable
Homogeneity test • Use the Q and I2 tests to analyze the heterogeneity in the effect sizes
• Use the results of heterogeneity analysis to confirm your model selection, not vice versa
Overall analysis • Use univariate and/or meta‐analytic structural equation modeling analysis to analyze the direction and
indirect relationships and their strength of the relationship
• Include additional tests to compare relationships in the univariate analysis model.
Moderator analysis • Use subgroup analysis, meta‐regression, or multilevel analysis to test the effect of moderators in the
meta‐analytic conceptual model
Software to conduct a meta‐analysis • Comprehensive Meta‐Analysis (CMA), Review Manager (RevMan), and Stata are commercial software
but R packages for meta‐analysis are free and open‐source
• Commercial software is easy to use and does not require programming skill
• R packages are flexible but require a basic knowledge of R software and programming
(Continues)
TABLE 3 (Continued)
Step Suggestions
Reporting
Title • Include main concept/construct and meta‐analysis or meta‐analysis review in the title
Abstract • Overview of the main question in the meta‐analysis

• A brief overview of the method, especially the number of studies and effect size
• The main focus is on the major findings of the analysis and its implications
Introduction • Start with a real example to highlight the topic of interest

• Overview of your main question and its importance
• Describe research streams in your research topic
• Highlight the variability in prior studies in your research topic
• Illustrate the role of meta‐analysis in the synthesis of prior studies and resolve the variabilities
Background and theoretical • Review of history, evaluation, definitions, and main research streams in a research topic
framework • Develop a framework underpinned by theory, model, or research streams
• Justify the hypothesized relationship in the framework, moderator, and control variables.
Method • Include all items related to data collection, then describe data preparation and data analysis procedures
in the meta‐analysis
Data analysis • Descriptive analysis to explain outlier, publication bias, heterogeneity test results
• The overall analysis results
• Moderator analysis and control variables findings
Conclusion • Research theoretical and empirical contributions in a summary table

• Explain research limitations and define further research avenues to extend the research topic
Appendix • The studies included in the research, formulas and their calculations, and extra analysis
allow a researcher to resolve contradictory results about the especially during data gathering and data preparation (Kirca & Yaprak,
relationship between concepts, the direction of these relationships, 2010). The researcher also needs to define the relationships between
and variables that moderate these relationships (Paul et al., 2021). variables and moderators to these relationships (Cooper, 2015).
Moreover, valuable research questions allow a researcher to test Often, there are numerous‐sometimes contradictory—definitions and
relationships that have not been previously studied through both the relationships proposed by different researchers. Thus, authors must
overall analysis and moderator analyses (Geyskens et al., 2009). have clear conceptualizations and strong justification for relation-
The meta‐analytic research question needs to be sufficiently ships in their meta‐analytic framework (Kirca & Yaprak, 2010).
broad so that researcher may find enough quantitative studies (Paul Authors might tap into extant theories or models to specify the
et al., 2021), but bound enough to occupy a unique position in a relationships between focal concepts and how moderator variables
research domain (Paul & Criado, 2020). Consider, for example, Crook moderate key relationships (Grewal et al., 2018). Consider, for
et al. (2011). These researchers study how human capital affects firm example, corporate social responsibility (CSR) initiatives study in
performance—relationships for which there are enough quantitative previous research. Vishwanathan et al. (2020) developed the
studies to permit examination through meta‐analysis, for which there strategic CSR concept to include different types of CSR in their
are controversial findings. meta‐analysis and tested its impact on firm corporate financial
performance. Moreover, variables such as context, industry type, the
potential impact of product type, and culture are explored as
4.1.3 | Developing a meta‐analytic conceptual moderators.
framework
In a meta‐analysis, it is necessary to define the focal concept and 4.2 | Data collection
relationships, as well contextually moderators to these relationships a
priori (Grewal et al., 2018). In defining the main concept in a meta‐ Data collection in a meta‐analysis begins with searching the
analysis, the existing literature may use the same or different terms literature, then applying inclusion/exclusion criteria to finalize the
for the same concept (Card, 2015). Having clear conceptual and mate‐analytic database. Data collection in a meta‐analysis should
operational definitions of focal concepts are critical in a meta‐analysis employ a funnel model, in which various sources are examined so as
to identify as many publications as possible in the search process. reference lists of top papers on the research topic (Steel et al., 2021).
Then, through screening, the researcher can eliminate publications To do this, authors might use Web of Science or Google Scholar to
that do not meet inclusion criteria (Grewal et al., 2018). It is very identify relevant papers and seminal articles with high citations and
important to keep detailed records of the screening process to report review their reference lists, as usual as more recent research citing
in the meta‐analysis to demonstrate transparency and accuracy (Paul the top papers. Some meta‐analysts also contact leading researchers
et al., 2021). Some meta‐analysts use PRISMA (Preferred Reporting in the research domain and request their unpublished, forthcoming,
Items for Systematic reviews and Meta‐Analyses; Moher et al., 2009) and recent work on the topic of interest. For example, Fischer and
or SPAR‐4‐SLR (Paul et al., 2021) to document this process. Mansell (2009) contacted 25 researchers who conducted studies on
employee commitment among different cultures to obtain their
unpublished work. Finally, a meta‐analyst might post a request on
4.2.1 | Literature search academic list‐server websites or email lists to request papers,
especially unpublished work (Jeyaraj & Dwivedi, 2020). To illustrate,
“Data” in a meta‐analysis include prior studies on the topic of interest Zhao et al. (2022) complete their meta‐analysis database by sending
which they must seek out in the search process (Littell et al., 2008). emails to the Academy of Management e‐mail lists, asking research-
Authors should define the search terms based on focal concepts in ers to send their working and unpublished publications.
their conceptual model to search them in different databases later.
For instance, to assess the effectiveness of customer involvement in
the new product development, Chang and Taylor (2016) consider 4.2.2 | Define inclusion/exclusion criteria
search terms such as “customer participation,” “customer involve-
ment,” “cocreation,” “coproduction,” and “crowdsourcing” in combi- After collecting all related publications in the search process, authors
nation with search terms like “new product” and “new service” to must specify inclusion/exclusion criteria to ensure all studies
search all publications related to their meta‐analysis framework. ultimately included in meta‐analysis have similar, desired features
The authors could use a keyword search to search the data from (Grewal et al., 2018). Here, we explain some of inclusion/exclusion
related and unrelated research fields based on the research question. criteria that are common in most meta‐analyses. First, meta‐analyses
This method would allow the authors to synthesize their findings involve only quantitative studies (Jeyaraj & Dwivedi, 2020); hence,
across disciplines and enhance the quality of their work. However, researchers should exclude all review papers, qualitative research,
research requires close attention to the meaning of the keywords in and papers with descriptive analyses from the meta‐analytic
different research fields. For example, Barari et al. (2021) meta‐ database. Moreover, the original studies must contain the required
analysis of antecedent and consequences of customer engagement, statistical information (e.g., correlation coefficients, standardized beta
only includes studies in which engagement is toward for‐profit firms coefficients, and t‐values) to calculate a common effect size (Grewal
to limit their search to the field of marketing research. Engagement is et al., 2018). Most studies in high‐quality journals provide enough
a popular research area in different fields such as education, human information to calculate an effect size (Jeyaraj & Dwivedi, 2020);
resource, and computer science, while its meaning differs across however, when this information is not reported in a paper, the meta‐
different research fields. However, Blut and Wang's (2020) meta‐ analyst may contact the authors to retrieve this information or
analysis of technology readiness includes “innovativeness” as one of remove the study from the meta‐analytic database (Card, 2015).
the technology readiness motivators. Researchers from different Second, all studies included in a meta‐analysis must quantitively test
research fields use different terms for this concept. Therefore, they the association between the variables of interest (Jeyaraj & Dwivedi,
searched for “consumer innovativeness” in the marketing literature 2020). To ensure the “right” relationships are measured, authors must
but “personal innovativeness” in the information system literature. develop a coding manual with clear conceptual and operational
When collecting data, authors should consider both published definitions of all variables in the meta‐analytic framework and include
works (e.g., journal articles, book chapters, and published conference only research that tests at least one pairwise relationship in that
papers) and unpublished works (e.g., theses, working papers, meta‐analytic framework. Third common criteria for papers in a
unreleased papers, and conference papers). Doing so would decrease meta‐analytic would be the time frame (i.e., the period from which
the publication bias (Cooper, 2015), which is a serious issue with studies are drawn), which may be a consideration. For example, in
meta‐analysis, as including unpublished work may change the results their meta‐analysis examining the impact of strategic resources on
(Geyskens et al., 2009). One way to cast a wide net in one's search is performance, Crook et al. (2008) chose 1991 as the starting point
to use comprehensive electronic databases in business and manage- of data collection. Because this is the year resource‐based theory
ment to search out both published and unpublished work (e.g., ABI/ was introduced. Finally, the meta‐analyst might impose various
Inform Global, ProQuest, PsycINFO, SSRN, and EBSCO; Geyskens research design criteria, so long as there is strong justification
et al., 2009). A manual review of premier journals publishing papers in for it. To illustrate, Knoll and Matthes (2017) included only
a research domain can also be helpful in identifying individual papers quantitative studies with experimental research designs in their
(Grewal et al., 2018). This strategy helps ensure the researcher did meta‐analysis of celebrity endorsements effectiveness, because it
not miss any related studies. Meta‐analysts should also check the allows causality inference.
4.2.3 | Data coding and final sample description should report how coders addressed disagreements in coding. For
instance, in Jiang et al.'s (2012) meta‐analysis, the first and third
In this step, researchers code the different characteristics of authors independently coded all studies, and reported their inter‐
individual studies for use in data analysis. Specifically, the meta‐ coder agreement (96%). They also specified that disagreements were
analyst must code: (1) pairwise relationship (s) which have been resolved through discussion.
studied in each individual study; (2) characteristics from prior studies Finally, the meta‐analyst should describe the final sample
which may moderate relationships in the meta‐analytic framework; characteristics of the studies included in the meta‐analysis
and, (3) characteristics to control for the conceptual framework. (Jeyaraj & Dwivedi, 2020). To illustrate, Dulebohn et al. (2012)
Initially, researchers must determine which pairwise relationships in describe their meta‐analysis sample based on the type of organization
the meta‐analytic framework have been tested each original study, (63% private and for‐profit, 16% public sector, 15% education, and
and extract the effect sizes corresponding to these relationships. For 6% health sector), sample location (83% the United States and 17%
example, Jiang et al. (2012) were examined the interrelationships rest of the word), and research design (97% study reported cross‐
between HR practice, human capital, employee motivation, voluntary sectional results and only 3% reported longitudinal data).
turnover, operational, and financial outcomes in their meta‐analysis
of the role of human resource management on organizational
outcomes. Thus, they examined each original study to determine 4.3 | Data preparation
which of these pairwise relations were tested and captured the
corresponding correlation coefficients. In this step, researchers extract the required information from each
Researchers also code different moderators that could explain individual study. For the pairwise relationship, the meta‐analyst
the fluctuation in pairwise relationships (Higgins et al., 2019), extracts or calculates an effect size from each individual study, which
including study design characteristics (Jeyaraj & Dwivedi, 2020). A indicates the direction and strength of the association between two
review of previous empirical studies and extant meta‐analyses on a variables (Card, 2015). While there are different statistics to consider,
research topic often helps authors complete their list of potential in business and management correlation coefficients and standard
moderators. Study design characteristics might include subjects' age, mean difference coefficients are the meta‐analysis effect size
gender, and nationality, as well as year of publication and type of metrics. For moderator and control variable, the meta‐analyst uses
research design. For example, Fischer and Mansell's (2009) meta‐ available information from each individual study to assign value to
analysis of employee commitment, the country in which studies were these variables (e.g., based on type of market, studies conducted in
conducted, sample sizes, percentage male/female, mean age, the business‐to‐business context = 1, and studies conducted in the
whether the sample was blue or white collar, response rate, industry, business to customer context = 0).
organization, and job type.
Finally, meta‐analysts typically code probable control variables to
role them out as a source of heterogeneity in the relationships 4.3.1 | Choose an effect size metric
between the main variables. Including control variables in the meta‐
analytic framework enhance the rigor of work (Grewal et al., 2018). Choosing the right effect size depends on the research design of the
As with moderator selection, authors might follow prior studies or studies included in the meta‐analytic database (Steel et al., 2021). If
previous meta‐analyses on the topic to refine their list of control most studies employ an experimental design with control and
variables. In their meta‐analysis of customer relational benefits, experimental groups, the authors are limited to the mean difference
Gremler et al. (2020) include several control variables (e.g., single vs. metric as the effect size (Card, 2015). If most studies administered as
multiple industries, student vs. nonstudent samples, and published vs. a survey, then authors would likely use correlation coefficients as the
unpublished studies) to ensure that fluctuations in effect sizes are not effect size (Kirca & Yaprak, 2010). In contrast to correlation or mean
because of these variables. difference coefficient, metrics such as regression beta coefficient,
Data coding involves developing manual to minimize subjectivity t‐test, Χ2 test, F‐test values are not an effect size because they
and increase the reliability of the coding process (Jeyaraj & Dwivedi, cannot combine and compare them between individual studies.
2020). A clear coding manual is useful when different terms are used However, these data can be used to calculate the desired effect size.
for the concept of interest. A coding manual typically includes each
concept's conceptual and operational definitions and common aliases Correlation as an effect size
(Higgins et al., 2019). Correlation as an effect size metric is very popular in business and
It is important to involve at least two coders who keep track of management. Some meta‐analyses use Fisher's z‐transformed corre-
discrepancies and calculate inter‐coder agreement to demonstrate lation for data analysis (Card, 2015). In contrast to the correlation
reliability in the coding process. The agreement rate is a percentage coefficient, a z‐transformed correlation has an approximately normal
from 0% to 100% where higher percentage represents higher distribution (Geyskens et al., 2009). However, the z‐transformed
agreement between coders and higher quality in the coding process. correlation is not comparable across studies. Thus, there is
In addition to reporting the inter‐coder agreement, the meta‐analyst an ongoing debate about using the correlation coefficient or
z‐transformed correlation for meta‐analysis (Kirca & Yaprak, 2010). we only focus on the most common artifact correction procedures in
Importantly, if some studies did not report the correlations between business and management, beginning with the measurement error
research variables, it is possible to calculate the correlation coefficient correction.
from different statistics such as regression beta coefficient, t‐value, The measurement error correction is necessary because individ-
f‐value, Χ2 statistic, z‐value, and mean differences. ual studies fail to measure the variables correctly and perfectly
(Geyskens et al., 2009). To correct effect size for this artifact, the
Standard mean differences as an effect size authors could use Hunter and Schmidt's (2004) formula, where
A mean difference as an effect size metric indicates the magnitude of the effect size (rxy) needs to be divided by the square root of two
the difference between the mean of two groups as a function of the variable measurement reliability products √(rxx)*√(ryy). This correction
group standard deviation (Jeyaraj & Dwivedi, 2020). The two main may result in an effect size greater than one, in this case, the meta‐
mean difference metrics are Hedges's g and Cohen's d (Card, 2015). analyst must consider one as the effect size. Second, the meta‐
They differ in their standard deviation calculation in the denominator analyst corrects for range restriction (Jeyaraj & Dwivedi, 2020), in
of the formula. With large sample sizes they are identical, so some which the standard deviation of effect size from the original study is
regard them as similar (Higgins et al., 2019). These two effect size smaller than the population's standard deviation. Both variables in an
metrics are convertible to each other. In business and management, effect size will be corrected. Authors must divide the sample standard
mostly Cohen's d as standard mean difference is used in meta‐ deviation of a study by the reference population standard deviation.
analysis. Similar to the correlation coefficient, it is possible to A third, effect size correction is related to artificial dichotomization
calculate this metric based on available information in an individual (Geyskens et al., 2009). If one or both of the variables believed to be
study (Kirca & Yaprak, 2010). associated are artificially dichotomized, a correction is required. The
meta‐analysts need first to know the dichotomization method. They
could then employ the Hunter and Schmidt (2004) procedure to
4.3.2 | Issues in effect size extraction restore its original value.
There are some issues in the effect size extraction worth noting (Steel
et al., 2021). It is common in business and management studies to 4.3.4 | Dealing with outliers
conduct multiple studies to test the same relationships in a
framework through different research designs in an effort to enhance Authors must deal with outliers, which are extremely small or large
the robustness of findings (Jeyaraj & Dwivedi, 2020). In such cases, effect sizes that may influence the accuracy of data analysis
there are more than one effect size for the same pairwise relation- (Geyskens et al., 2009). Schematic plot analyses or analyzing the
ships in an individual study. In this situation, if these effect sizes are number of standard deviations from the mean are common methods
from the same sample, the preferred procedure is to combine them to for outlier identification (Jeyaraj & Dwivedi, 2020); however, these
calculate a single effect size for a relationship in a study. The meta‐ techniques do not take the sample size of individual studies into
analyst might use the Hunter and Schmidt (2004) method to calculate account, so they are not optimal for use in meta‐analyses (Card, 2015).
a composite correlation. If there is not enough information to One popular and sophisticated method for outlier identification in meta‐
calculate the composite correlation, the best alternative would be the analysis is the Huffcutt and Arthur (1995) sample‐adjusted meta‐
average of the equivalent correlations (Geyskens et al., 2009). analytic deviancy statistic (SAMD). Here, the authors calculate the
However, if authors conduct several independent substudies in a difference between each original study's effect size and the mean
single study with an independent sample to test the same pairwise sample‐weighted coefficient without including the original study's effect
relationships, extracting an effect size from each substudy is size in the coefficient calculation to identify the outliers. When outliers
appropriate (Steel et al., 2021). Regarding, it is important to role that are identified, the meta‐analyst must decide whether to eliminate them
in a meta‐analysis, the number of effect sizes may be higher than the or conduct sensitivity analysis. With a sensitivity analysis, the meta‐
number of individual studies. analyst examines data with and without outliers to see the impact of
outliers on results (Geyskens et al., 2009). If the outlier has an impact on
result, the author might report the result with and without the outlier
4.3.3 | Effect sizes corrections (Geyskens et al., 2009). As the outlier could not make a study
automatically incorrect, the author needs to be careful about removing
After extracting/calculating the effect size, the meta‐analyst corrects outliers from database (Grewal et al., 2018).
it before using it in the data analysis. This correction is due to some
artifacts that lead to biases (Geyskens et al., 2009), such as
measurement error, imperfect validity in variable measurement, or 4.3.5 | Publication bias
an imperfect sample that does not represent the whole population
(Jeyaraj & Dwivedi, 2020). Although there are several artifact Meta‐analysts have easier access to published studies; however,
corrections in the meta‐analysis literature (Borenstein et al., 2021), there may be some unpublished studies on topics that authors do not
include in the meta‐analysis (Jeyaraj & Dwivedi, 2020). It is possible In their meta‐analysis, Gremler et al. (2020) designated type of
that the inclusion of such publications in the data analysis would service and type of market as two main moderators. They defined
change the magnitude, direction, or significance of relationships two levels of type of service (i.e., encounter service = 0 and
between pairwise relationships in the meta‐analysis (Card, 2015). As relationships service = 1). Similarly, for the type of market, they
mentioned previously, comprehensive database development can defined two levels (i.e., business‐to‐customer = 0 and business‐to‐
remedy this issue. There are also several methods to provide business = 1). They also included control variables such as sample
additional evidence of the robustness of the results, and to type (student sample = 1 and nonstudent sample = 0) and publication
demonstrated that publication bias is not a problem (Steel et al., state (published paper = 0 and unpublished paper = 1).
2021). A traditional metric is the file drawer N procedure in which
researchers show how many null effects studies would need to exist
to change a significant relationship in a meta‐analysis to a 4.4 | Data analysis
nonsignificant one (Grewal et al., 2018). A second technique is the
Orwin fail‐safe N procedure, which indicates how many missing Before data analysis, the meta‐analyst must choose between a fixed‐
effect sizes are required to bring an effect size to a nonzero value. effects model or random‐effects model, and assess variation in the
Finally, the meta‐analyst may use Duval and Tweedie's (2000) Trim effect sizes through a heterogeneity test (Steel et al., 2021). Choosing
and Fill method to identify and correct for publication bias. This between a fixed‐effects model or random‐effects model is a critical
method includes removing (i.e., trim) the extreme effect sizes to step and will affect the whole data analysis process (Aguinis, Dalton,
reduce the variance. Then add (i.e., fill) removed studies to correct the et al., 2011). This selection is based on the researcher's assumptions
variance of the adjusted effect size (Grewal et al., 2018). about the population from which studies come. After this, the main
model is analyzed to determine the significance and strength of
relationships in the meta‐analytic. Corrected and combined effect
4.3.6 | Effect sizes combination sizes and univariate analysis are then used to further assess pairwise
relationships in the model. Moreover, to explain the heterogeneity in
In the final step of data preparation, the meta‐analyst would combine the relationships between variables of interest, the authors conduct a
effect size from different studies (Steel et al., 2021). Because each moderator analysis. Subgroup analysis or meta‐regression are used to
study in the meta‐analytic database has a different sample size, the test the role of moderators in the framework. Finally, the probable
point estimation of the true effect size varies (Jeyaraj & Dwivedi, role of control variables in the meta‐analytic framework is tested.
2020). A study with a larger sample size would have a more precise
estimation. Thus, simply averaging effect sizes is inappropriate
because it does not take into account these differences across 4.4.1 | Fixed‐effects model
studies. The first approach is the reciprocals of the estimated
variances of the observed effect sizes (Steel et al., 2021), which In the fixed‐effects model, the researcher assumes all studies are
determine the weight of each study. This approach gives more weight based on the same population and thus share the same underlying
to studies with smaller standard errors than other studies. The meta‐ true effect size. Because of this, the model is used as a singular term
analyst may also calculate the weighted average of effect sizes for in the fixed‐effects model (Steel et al., 2021). As all studies try to
each pairwise relationship based on the sample size of each study. In estimate the same parameter (i.e., population effect size), the only
business and management authors usually use a weighted average to source of variation among the different studies is sampling errors in
combine effect sizes from individual studies (Geyskens et al., 2009). each study (i.e., within‐study error). Therefore, in the fixed‐effects
model, the meta‐analytic findings are generalizable only to studies
included in the meta‐analysis (Grewal et al., 2018).
4.3.7 | Moderator and control variables
To examine the impact of moderator and control variables, the meta‐ 4.4.2 | Random‐effects model
analyst must assign a value to include these variables in analyses
(Geyskens et al., 2009). How such values are assigned depends on the In contrast to the fixed‐effects model, with random‐effect models,
nature of these variables and the information reported in the original studies are not assumed to come from the same population, and each
studies (Jeyaraj & Dwivedi, 2020). These variable values might be study estimates a unique parameter. Thus, the researcher uses
categorical or continuous values, like for demographic moderators models as a plural term in the random‐effects models (Steel et al.,
such as age, gender, education, or income (Steel et al., 2021). For 2021). Sources of variation or heterogeneity in the effect sizes are
other moderators such as culture, authors could use the Hofstede the sampling error of each study population (i.e., within‐study
et al. (2005) cultural dimension index (ranging from 1 to 100)—a variance) and sampling error of the universe of all relevant
continuous variable. Similar to moderator variables, information in populations (i.e., between‐studies variance; Grewal et al., 2018).
the original studies are used to assign a value to the control variables. More importantly, the generalization of meta‐analysis results in the
random‐effect model is not limited to studies included in a meta‐ used as input for SEM to test different relationships between
analysis. Therefore, most of the meta‐analyses in business and variables of interest (Jak, 2015). Because the sample size for
management prefer to choose a random‐effects model. different correlations is not equal, authors must calculate the
harmonic mean for analysis. Compared to the arithmetic mean, the
harmonic mean gives less weight to large sample sizes and thus
4.4.3 | Homogeneity analysis parameter estimation is better. As with SEM, the researcher uses
statistics to assess the model fit with the data. Based on these
Homogeneity analyses help authors to test variation among effect statistics, the optimal model which has the best fit with data is
sizes in prior studies necessary and to define appropriate moderators determined. Jiang et al. (2012) employ meta‐analytic SEM in their
to capture and justify this variation. Q and I2 tests are used to test the study, defining several mediation variables to provide in‐depth
homogeneity between effect sizes (Steel et al., 2021). Q statistics test analysis of how three dimensions of human resource (HR) practices
the null hypothesis of homogeneity versus heterogeneity. If the result affect organizational performance. Their optimal model highlights
exceeds the Χ2 critical value, indicates the heterogeneity of effect that HR practices through mediation variables (i.e., human capital,
sizes (Card, 2015). When there is no heterogeneity among the effect employee motivation voluntary turnover, and operational out-
sizes, the result would be one with a higher number denoting higher come) impact firm financial outcomes.
heterogeneity between effect sizes. In business and management,
this statistic is usually higher than one, indicating heterogeneity in
effect sizes from individual studies. Q‐tests, however, cannot indicate 4.4.5 | Analysis of moderators and control variables
the magnitude of the heterogeneity in effect sizes; therefore, authors
use the I2 statistic which indicates the percentage of variability A moderator analysis allows researchers to capture variations in the
between effect sizes attributable to the total variability among effect relationship(s) of interest and resolve contradictory findings (Steel
sizes. Since I2 is in the form of a percentage, it shows the magnitude et al., 2021). With a meta‐analysis, authors can include new
of heterogeneity among effect sizes. Usually, an effect size of 25% is moderators which have not been tested in the original studies to
considered as small, 50% is medium, and higher than 75% is provide more insight on the topic. The meta‐analyst examines control
considered large heterogeneity (Card, 2015). variables to determine if the variability in the effect size for the
pairwise relationship is because of these variables. Three main
approaches for conducting a moderator analysis are subgroup
4.4.4 | Overall analysis analysis, meta‐regression, and multilevel analysis.
In the overall analysis, the meta‐analyst tests the relationships in the Subgroup analysis
proposed framework. Two main methods for the overall analysis are Subgroup analysis is used to test moderator effects in a meta‐analysis
univariate analyses and meta‐analytic structural equations modeling. (Steel et al., 2021). In a subgroup analysis, the meta‐analyst computes
the mean effect for different subgroup studies (Borenstein & Higgins,
Univariate analyses 2013). Then, the means of two or more sets of studies are compared
In the overall analysis, we test the pairwise relationships in our and analysis of variance or t‐tests are used to analyze the significance
framework through univariate analyses. Univariate analysis involve of differences (Borenstein et al., 2021). Subgroup analysis is limited to
testing the significance of combined and corrected effect sizes for moderators that are categorical; continuous variables require
each pairwise relationship (Steel et al., 2021). Moreover, univariate dichotomization, which degrades information and reduces statistical
analysis allows the researcher to determine the direction and power.
strength of the relationship between two variables, and thus provide
insight into the relationships between concepts in the meta‐analytic Meta‐regression
framework. Palmatier et al. (2006) use univariate analysis to study With meta‐regression, the meta‐analyst uses regression analysis to
relationships between customer, seller, and dyadic antecedents and study whether fluctuations in effect sizes for pairwise relationship(s)
customer‐focused relational mediators (e.g., commitment, trust, and are explained by a moderator variable (Steel et al., 2021). In this
relationship satisfaction) to study both influences of different analysis, moderator variables are predictors and effect sizes are
antecedents on mediators and interrelationships among these dependent variables. Meta‐regression can accommodate both con-
antecedents. tinuous and categorical variables in the analysis (Aguinis, Pierce, et al.,
2011); however, a high correlation between independent variables
Meta‐analytic structural equation modeling (i.e., multicollinearity) can cause problems with fit for model and
Univariate analyses only involve effect sizes of pairwise relationships; interpreting results. The authors could use WLS regression instead of
however, with meta‐analytic structural equations modeling (SEM), ordinary least squares to reduce the multicollinearity in the meta‐
the researcher correlates all variables in the meta‐analytic regression (Steel & Kammeyer‐Mueller, 2002). To illustrate, Blume
framework with the correlation matrix. The correlation matrix is et al. (2010) studied the impact of independent variables (i.e., trainee
characteristics, work environment, and training interventions) on the Stata

transfer of training using meta‐regression analysis. Tasks and Stata allows authors to conduct a wide range of statistical analyses,
context‐based moderators (e.g., employee experience and pretraining including meta‐analyses. Stata enables the meta‐analyst to develop
self‐efficacy) were also assessed. Their results confirm the ability of forest plots to analyze outliers and to conduct a heterogeneity
these moderators to explain the inconsistencies in the impact of analysis. Further, researchers can conduct univariate analysis for the
predictor variables on the transfer of training. overall analysis, and subgroup, meta‐regression, and multilevel
regression for the moderator analysis. Stata also estimates both
Multilevel analysis fixed‐effects and random‐effects models.
In business and management, we extract multiple effect sizes from an
original study. As these effect sizes are not independent, ignoring R packages
these tendencies led to the underestimation of standard error R has several packages developed for meta‐analyses, such as the
(Grewal et al., 2018). In the multilevel meta‐regression, we define psychmeta (Dahlke & Wiernik, 2019) and metafor (Viechtbauer, 2010).
multiple levels to account for this issue, providing a more accurate These packages cover almost all aspects of data analysis in a meta‐
estimation (Gremler et al., 2019). Multilevel models in a meta‐analysis analytic study such as effect size corrections/combination, outlier
usually include two levels: (1) level one indicates information that analysis, publication bias estimation, and both the overall analysis and
varies within studies (i.e., effect size level); and, (2) level two includes moderator analysis. Compared to commercial software/programs,
characteristics that vary between studies (i.e., study level). Similar to such as CMA, RevMaN, and Stata, R is open‐source and free;
meta‐regression, effect sizes are the dependent variable while however, some familiarity with R coding is necessary to install
mediators and outcome variables are predictors in level one. packages, import data, and conduct the meta‐analysis.
Moderators and control variables are in the second level. To test
the model, an iterative generalized least squares procedure is used
which provides maximum likelihood estimates and variables that are 4.5 | Reporting
centered in the model (Gremler et al., 2019). For example, Roschk and
Hosseinpour (2020) used multilevel analysis to assess moderators to One of the challenging parts of meta‐analysis is writing the report.
the relationship between ambient scents and customer responses. The meta‐analyst must comprehensively describe the details of each
They defined a different group of moderators such as scent step of the meta‐analysis in a logical, and easy‐to‐follow sequence
characteristics, scent perceptual properties, environmental factors, (Siddaway et al., 2019). Based on prior meta‐analyses in business and
research operational factors, and individual factors to provide a clear management, we propose a structure for writing up a meta‐analysis.
and detailed overview of the role of in‐store ambient scents on Like manuscript paper, a meta‐analysis includes a title, abstract,
customer wide ranges of responses such as mood, evaluations and, introduction, theoretical background, conceptual framework and
behaviors. hypotheses, methodology, data analysis, and general discussion.
However, with a meta‐analysis, the content of each section is
somewhat different.
4.4.6 | Meta‐analytic software
Comprehensive Meta‐Analysis 4.5.1 | Title

Comprehensive Meta‐Analysis (CMA) software is employed wide in
meta‐analyses in business and management (Rana & Paul, 2020). This A meta‐analysis title should reflect the primary relationships, concept
software enables the meta‐analyst to import effect sizes in different (s), or construct (Cooper, 2015). Moreover, it is better to include the
formats or directly input effect sizes into the program. It also allows term “meta‐analysis” or “meta‐analytic review” in the title, to gain the
authors to conduct different statistics into the desired effect size audience's attention and ensure it can easily be located in database
metric (e.g., a correlation coefficient or standard mean difference). searches (Kepes et al., 2013). An example of an appropriate title for a
Furthermore, CMA helps researchers assess outlier, publication bias, meta‐analysis is “A Meta‐analysis of Customer Engagement Beha-
and heterogeneity in the database and examine both fixed‐ and viour” (Barari et al., 2021), which describes the focal relationships and
random‐effects models. includes “meta‐analysis” in the title.
Review Manager
Review Manager (RevMan) is a web‐based software that manages 4.5.2 | Abstract
the entire literature review process and meta‐analysis. The
meta‐analyst uploads all studies to RevMan library, where they Summarizing a meta‐analytic study in a short paragraph can be a
can be managed and exanimated for inclusion. Like CMA, RevMan challenging task. The meta‐analyst should mention the main relation-
enables authors to conduct overall analysis and moderator ships explored in the meta‐analysis, and explain its importance
analysis. (Cooper, 2015). The number of studies and observations should be
highlighted to demonstrate the power of meta‐analysis (Kepes et al., the data collection, procedure used, and explain the data preparation
2013). The abstract of meta‐analysis should also summarize key and analysis process (Cooper, 2015). In the data collection section,
findings and takeaways. Specifically, the meta‐analyst should provide meta‐analyst should provide details of the search strategy and
an overall view of results from the overall analysis and moderator process employed (i.e., keywords, databases, manual searches, and
analysis, and highlight significant contributions of meta‐analysis to others effort(s)) to identify all related publications and decrease
the research domain (Cooper, 2015). publication bias (Cooper, 2015). Authors must also describe inclusion/
exclusion criteria and data coding processes (i.e., the coding manual,
who coded the papers, and coder agreement rate) and explain these
4.5.3 | Introduction decisions (Kepes et al., 2013). After this, the meta‐analyst must describe
the final sample characteristics to provide an overview of studies
The introduction of a meta‐analysis could begin with a real example to included in the meta‐analysis (Siddaway et al., 2019). In data
highlight the topic of interest and its importance for the meta‐analytic preparation, authors need to explain effect size extraction/calculation,
study (Siddaway et al., 2019). Then, authors should provide a brief history corrections employed, and any issues involved in this process (Cooper,
of the main research topic, explain key concepts, construct, and theories 2015). Moreover, authors need to describe analyses to assess outliers,
in the research domain (Cooper, 2015). The meta‐analyst should also publication bias, and heterogeneity and their meta‐analytic model (i.e., a
describe the relevant literature, highlighting conflicting findings in prior fixed‐effects model or a random‐effects model). Finally, the authors
studies and unresolved research questions (Kepes et al., 2013). Doing so must describe the overall analysis and moderator analysis, regarding
helps to justify the necessity of a meta‐analysis and engage readers. The software that was used (Cooper, 2015).
author should explain how the meta‐analysis will help resolve discrepan-
cies in the literature and provide an overview view of the research
domain (Cooper, 2015). For example, in Iyer et al.'s (2020) meta‐analysis 4.5.6 | Data analysis
of impulse buying behavior, the introduction explains the importance of
this topic from a practical and academic perspective, and describes In the data analysis section, the meta‐analyst describes the
various research streams that examine the triggers of impulse buying descriptive analysis, overall analysis, and moderator and control
from different perspectives. They also explain why a meta‐analysis on variables analysis (Cooper, 2015). The descriptive analysis section
impulse buying is necessary (i.e., to combine and synthesize these diverse details result of the outlier analysis, and how the researcher dealt
studies). Finally, the authors briefly explain their meta‐analytic framework, with them, as well as the publication bias check and heterogeneity
and how their work contributes to this study domain. test. In the overall analysis, researchers statistically characterize the
relationships between variables in the meta‐analytic framework and
their significance. The authors can use different statistical techniques
4.5.4 | Background and theoretical framework to maximize their analysis in the overall analysis (Kepes et al., 2013).
For example, in their meta‐analysis investigating the relationship
A meta‐analysis aims to provide a comprehensive inventory view of a between HR practices and organizational outcomes, Jiang et al.
specific research domain (Grewal et al., 2018). Thus, it is necessary to (2012) Z‐test statistic to study relative impact of HR practice on
discover the history, relevant definitions, and research stream in the employee motivation and human capital. Moreover, meta‐analytic
background section of the manuscript (Siddaway et al., 2019). Tables SEM is employed to test how Human Resources practice through a
may be used to depict different aspects of research domain. The mediation process influences the organizational outcome. The
authors should also develop a Meta‐analytic framework based on moderator analysis entails defining numerous variables that may
prior research streams in the topic, drawing on relevant theory to impact variations in the original studies. Ancillary analyses may also
explain relationships between key variables and formulate hypothe- be conducted to extend findings or explore specific relationships
ses (Cooper, 2015). To illustrate, the background section of Rana and (Cooper, 2015). For example, in their meta‐analysis of customer
Paul's (2020) meta‐analysis of organic food purchase, first illustrates responses to in‐store ambient scents (Roschk & Hosseinpour, 2020)
organic food, then discovers organic food consumers and factors that explored numerous moderators and examined specific interaction
affect consumers' organic food purchases. They then develop a meta‐ effects to enrich their moderator analysis.
analytic framework that depicts the relationships between health
motives and consumers' organic food purchases.
4.5.7 | Discussion
4.5.5 | Methodology This section address how researchers might describe theoretical and
managerial implications of conducting meta‐analysis, limitations
A meta‐analysis should feature a method section that is transparent assigned with this technique, and future research directions (Kepes
and accurate, rationalize methodological choices made by researchers et al., 2013). In discussing implications, the authors should consider
(Siddaway et al., 2019). In this section, the meta‐analyst should detail using a table to summarize their key research findings, organized by
research hypothesis or the meta‐analytic framework. They could then size. The Bayesian analysis allows authors to include a prior expected
discuss the theoretical and managerial implications of these key sampling distribution of a quantity of interest (Higgins et al., 2019).
findings. For example, Palmatier et al.'s (2006) meta‐analysis on Bayesian analysis aids in estimates of an alternative distribution of
factors affecting the effectiveness of relationship marketing presents variability among effect sizes from prior research, which is more
a table with key findings for antecedents, outcomes, and moderators, accurate. Some researchers believe that, compared to traditional
which illustrates theoretical and managerial implications of their meta‐analysis, the Bayesian method is more accurate and less
research. As with often methods, a meta‐analysis has specific unbiased for effect size variance estimation (Steel et al., 2015).
limitations that must be acknowledged (Siddaway et al., 2019). This
also helps authors to identify avenues for future research. For
example, a meta‐analysis is based on and restricted by prior studies, 5.2 | Network analysis
and there might not be enough effect sizes for all relationships of
interest to analyze. Similarly, many potentially interesting moderators Researchers are often interested in determining the relative effective-
may not be examined because of insufficient information because ness of different interventions and treatments in a population (Higgins
insufficient information in the original studies prohibit systematic et al., 2019). This is difficult to do using the traditional meta‐analytic
coding. Moreover, meta‐analysis involves a combination of statistical technique; however, network meta‐analysis allows researchers to
techniques and each of which has limitations. The meta‐analyst must review the comparative effectiveness of competing interventions. For
explain such limitations and explain their method selection (Siddaway studies with more than two interventions, when the direct compression
et al., 2019). Involvement in the meta‐analytic process enables between a network of interventions is available, network analysis allows
researchers to identify the areas where more scholarly attention is indirect comparisons of multiple interventions. The meta‐analyst uses
needed, and topics for extended research. mathematical combinations of direct interventions effect available to
estimate the indirect comparisons between interventions. In a network
meta‐analysis, authors combine these direct and indirect estimates
4.5.8 | Appendix across a network of interventions in a single study and make more
meaningful comparisons (White, 2015).
An appendix in a meta‐analysis typically provides supporting data and
analyses, such as studies included in the meta‐analysis and its coding
(Siddaway et al., 2019). The authors could include complete 5.3 | Machine learning in meta‐analysis
descriptions of all pairwise relationships in the appendix, along with
ancillary data analyses and results. For example, Gremler et al.'s Machine learning is a computer algorithm that learns from experience
(2020) appendix included a list of journals manually searched by to perform a specific task through the statistical modeling of data
researchers, studies included in the meta‐analysis, sample character- (Mitchell, 1997). One of the important implications of machine
istics, moderator and control variable coding, study characteristics, learning is handling large‐scale data and unstructured data. A sample
and results of all pairwise analyses and extra analyses. of data is entered into the algorithm as a training data set to teach the
algorithm; then, the task is completed based on this training. This
process can facilitate data analysis, especially when manual work
5 | M E TA‐ A N A LY S I S A D V A N C E M E N T S would be very time‐consuming. In a meta‐analysis, researchers follow
various strategies to include all related publications in their databased
In this section, we describe meta‐analytic advancements in recent development, and devote considerable time to extracting the
years, which have enhanced the accelerating of scientific knowledge required information from each individual study. Through training
development and its accuracy. and test data, the authors set out to develop an algorithm that could
help meta‐analysts complete these tasks and increase the quality of
their work (Marshall et al., 2018). Marshall and Wallace (2019)
5.1 | Bayesian analysis discuss the role of machine learning in facilitating their meta‐analysis,
and provide practical suggestions for using machine learning
Bayesian meta‐analysis is based on Bayes' theorem, which asserts algorithms to extract aspects of the meta‐analysis process, including
that the probability of an event is based on prior knowledge of that data collection, screening, and data extraction.
event (Joyce, 2003). The main advantage of the Bayesian approach is
incorporating prior knowledge or information about a phenomenon
into inferences, which improves analysis (Rossi et al., 2012). Bayesian 6 | CONCLUSION
analysis has recently been introduced to meta‐analysis (Higgins et al.,
2009). A distinctive feature of Bayesian meta‐analysis is in the effect Meta‐analysis is an effective way to advance current knowledge in
size combination step, when researchers take a different approach to business and management, and is more scientific than a pure
pool effect sizes from individual studies and create a combined effect bibliometric type of SLRs. Therefore, there is increasing interest
among researchers to publish meta‐analysis papers because of their Blume, B. D., Ford, J. K., Baldwin, T. T., & Huang, J. L. (2010). Transfer of
impact on knowledge development. However, the technical nature of training: A meta‐analytic review. Journal of Management, 36(4),
1065–1105.
meta‐analyses may prove daunting for academics and practitioners to
Blut, M., & Wang, C. (2020). Technology readiness: A meta‐analysis of
understand and conduct. Thus, in the current research, we demon- conceptualizations of the construct and its impact on technology
strate this study method to facilitate researchers' understanding of usage. Journal of the Academy of Marketing Science, 48(4), 649–669.
how to conduct meta‐analysis. Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2021).
Introduction to meta‐analysis. John Wiley & Sons.
A meta‐analysis begins with a fruitful and novel research
Borenstein, M., & Higgins, J. P. T. (2013). Meta‐analysis and subgroups.
question, such as reconciling the conflicting findings in a research Prevention Science, 14(2), 134–143.
domain. This question definition helps researchers develop a Canabal, A., & White III, G. O. (2008). Entry mode research: Past and
Meta‐analytic framework to guide the whole meta‐analysis future. International Business Review, 17(3), 267–284.
Card, N. A. (2015). Applied meta‐analysis for social science research.
process. The authors then engaged in data collection, employing
Guilford Publications.
different strategies to include different types of publication in the
Chang, W., & Taylor, S. A. (2016). The effectiveness of customer
process and applying logical inclusion/exclusion criteria to finalize participation in new product development: A meta‐analysis. Journal
the meta‐analytic database. Then, the meta‐analyst uses a coding of Marketing, 80(1), 47–64.
manual to code primary variables, moderators, and control Chen, Y., Mandler, T., & Meyer‐Waarden, L. (2021). Three decades of
research on loyalty programs: A literature review and future
variables in each individual study, extracting or calculating the
research agenda. Journal of Business Research, 124, 179–197.
effect sizes, assessing outliers and publication bias, and combining Cooper, H. (2015). Research synthesis and meta‐analysis: A step‐by‐step
effect sizes. Once the meta‐analyst has selected a fixed‐effects approach (Vol. 2). Sage Publications.
model or random‐effects model, heterogeneity is assessed and the Crook, T. R., Ketchen, D. J., Jr., Combs, J. G., & Todd, S. Y. (2008). Strategic
resources and performance: A meta‐analysis. Strategic Management
overall testing pairwise relationships in the framework are
Journal, 29(11), 1141–1154.
conducted. Moderators are then analyzed. Various software Crook, T. R., Todd, S. Y., Combs, J. G., Woehr, D. J., & Ketchen, D. J., Jr.
packages are available for conducting meta‐analyses including (2011). Does human capital matter? A meta‐analysis of the
commercial programs (e.g., CMA, Review Manager, and Stata) and relationship between human capital and firm performance. Journal
of Applied Psychology, 96(3), 443–456.
open‐source (R packages) software. Finally, the meta‐analyst
Dabić, M., Vlačić, B., Paul, J., Dana, L.‐P., Sahasranamam, S., & Glinka, B.
reports the result. This manuscript proposes an overarching (2020). Immigrant entrepreneurship: A review and research agenda.
structure to cover all important aspects of a meta‐analysis in Journal of Business Research, 113, 25–38.
business and management. Dahlke, J. A., & Wiernik, B. M. (2019). psychmeta: An R package for
psychometric meta‐analysis. Applied Psychological Measurement,
It is worth noting that meta‐analysis is an evolving method that
43(5), 415–416.
has seen several advancements in recent years that expand the Dulebohn, J. H., Bommer, W. H., Liden, R. C., Brouer, R. L., & Ferris, G. R.
effectiveness and accuracy of results. Meta‐analytic Bayesian (2012). A meta‐analysis of antecedents and consequences of leader‐
analysis and network analysis are examples of promising advance- member exchange: Integrating the past with an eye toward the
future. Journal of Management, 38(6), 1715–1759.
ments in meta‐analyses. Finally, employing machine learning in meta‐
Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel‐plot–based
analysis has been shown to facilitate the meta‐analytic process and method of testing and adjusting for publication bias in meta‐analysis.
increase its quality. This promising approach is in its initial stages and Biometrics, 56(2), 455–463.
needs more development before use in practice. Fischer, R., & Mansell, A. (2009). Commitment across cultures: A meta‐
analytical approach. Journal of International Business Studies, 40(8),
1339–1358.
ORCID Geyskens, I., Krishnan, R., Steenkamp, J.‐B. E., & Cunha, P. V. (2009). A
Justin Paul https://orcid.org/0000-0001-5062-8371 review and evaluation of meta‐analysis practices in management
Mojtaba Barari https://orcid.org/0000-0002-6905-4897 research. Journal of Management, 35(2), 393–419.
Goyal, K., & Kumar, S. (2021). Financial literacy: A systematic review and
bibliometric analysis. International Journal of Consumer Studies, 45(1),
REFERENCES 80–105.
Aguinis, H., Dalton, D. R., Bosco, F. A., Pierce, C. A., & Dalton, C. M. Gremler, D. D., Van Vaerenbergh, Y., Brüggen, E. C., & Gwinner, K. P.
(2011). Meta‐analytic choices and judgment calls: Implications for (2019). Understanding and managing customer relational benefits in
theory building and testing, obtained effect sizes, and scholarly services: A meta‐analysis. Journal of the Academy of Marketing
impact. Journal of Management, 37(1), 5–38. Science, 48(3), 565–583.
Aguinis, H., Pierce, C. A., Bosco, F. A., Dalton, D. R., & Dalton, C. M. Gremler, D. D., Van Vaerenbergh, Y., Brüggen, E. C., & Gwinner, K. P.
(2011). Debunking myths and urban legends about meta‐analysis. (2020). Understanding and managing customer relational benefits in
Organizational Research Methods, 14(2), 306–331. services: A meta‐analysis. Journal of the Academy of Marketing
Bahoo, S., Alon, I., & Paltrinieri, A. (2020). Corruption in international Science, 48(3), 565–583.
business: A review and research agenda. International Business Grewal, D., Puccinelli, N., & Monroe, K. B. (2018). Meta‐analysis:
Review, 29(4):101660. Integrating accumulated knowledge. Journal of the Academy of
Barari, M., Ross, M., Thaichon, S., & Surachartkumtonkun, J. (2021). A Marketing Science, 46(1), 9–30.
meta‐analysis of customer engagement behaviour. International Haidich, A.‐B. (2010). Meta‐analysis in medical research. Hippokratia,
Journal of Consumer Studies, 45, 457–477. 14(Suppl 1), 29–37.
Higgins, J. P., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M. J., & Motyka, S., Grewal, D., Puccinelli, N. M., Roggeveen, A. L., Avnet, T.,
Welch, V. A. (2019). Cochrane handbook for systematic reviews of Daryanto, A., de Ruyter, K., & Wetzels, M. (2014). Regulatory fit: A
interventions. John Wiley & Sons. meta‐analytic synthesis. Journal of Consumer Psychology, 24(3),
Higgins, J. P., Thompson, S. G., & Spiegelhalter, D. J. (2009). A re‐ 394–410.
evaluation of random‐effects meta‐analysis. Journal of the Royal Orlitzky, M., Schmidt, F. L., & Rynes, S. L. (2003). Corporate social and
Statistical Society: Series A (Statistics in Society), 172(1), 137–159. financial performance: A meta‐analysis. Organization Studies, 24(3),
Hofstede, G., Hofstede, G. J., & Minkov, M. (2005). Cultures and 403–441.
organizations: Software of the mind (Vol. 2). McGraw Hill. Palmatier, R. W., Dant, R. P., Grewal, D., & Evans, K. R. (2006). Factors
Huffcutt, A. I., & Arthur, W. (1995). Development of a new outlier statistic influencing the effectiveness of relationship marketing: A meta‐
for meta‐analytic data. Journal of Applied Psychology, 80(2), 327–334. analysis. Journal of Marketing, 70(4), 136–153.
Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta‐analysis: Correcting Palmatier, R. W., Houston, M. B., & Hulland, J. (2018). Review articles:
error and bias in research findings. Sage. Purpose, process, and structure. Springer.
Iyer, G. R., Blut, M., Xiao, S. H., & Grewal, D. (2020). Impulse buying: A Pansari, A., & Kumar, V. (2017). Customer engagement: The construct,
meta‐analytic review. Journal of the Academy of Marketing Science, antecedents, and consequences. Journal of the Academy of Marketing
48(3), 384–404. Science, 45(3), 294–311.
Jak, S. (2015). Meta‐analytic structural equation modelling. Springer. Pattnaik, D., Hassan, M. K., Kumar, S., & Paul, J. (2020). Trade credit
Jeyaraj, A., & Dwivedi, Y. K. (2020). Meta‐analysis in information systems research before and after the global financial crisis of 2008—A
research: Review and recommendations. International Journal of bibliometric overview. Research in International Business and Finance,
Information Management, 55, 102226. 54, 101287. https://doi.org/10.1016/j.ribaf.2020.101287
Jiang, K., Lepak, D. P., Hu, J., & Baer, J. C. (2012). How does human Paul, J. (2019). Marketing in emerging markets: A review, theoretical
resource management influence organizational outcomes? A meta‐ synthesis and extension. International Journal of Emerging Markets,
analytic investigation of mediating mechanisms. Academy of 15(3), 446–468.
Management Journal, 55(6), 1264–1294. Paul, J., & Benito, G. R. (2018). A review of research on outward foreign
Joyce, J. (2003). Bayes' theorem. direct investment from emerging countries, including China: What
Kahiya, E. T. (2018). Five decades of research on export barriers: Review do we know, how do we know and where should we be heading?
and future directions. International Business Review, 27(6), Asia Pacific Business Review, 24(1), 90–115.
1172–1188. Paul, J., & Criado, A. R. (2020). The art of writing literature review: What
Kepes, S., McDaniel, M. A., Brannick, M. T., & Banks, G. C. (2013). Meta‐ do we know and what do we need to know? International Business
analytic reviews in the organizational sciences: Two meta‐analytic Review, 29(4):101717.
schools on the way to MARS (the Meta‐Analytic Reporting Paul, J., Lim, W. M., O'Cass, A., Hao, A. W., & Bresciani, S. (2021).
Standards). Journal of Business and Psychology, 28(2), 123–143. Scientific procedures and rationales for systematic literature reviews
Keupp, M. M., & Gassmann, O. (2009). The past and the future of (SPAR‐4‐SLR). International Journal of Consumer Studies, 45(4),
international entrepreneurship: A review and suggestions for O1–O16.
developing the field. Journal of Management, 35(3), 600–633. Paul, J., Parthasarathy, S., & Gupta, P. (2017). Exporting challenges of
King, W. R., & He, J. (2006). A meta‐analysis of the technology acceptance SMEs: A review and future research agenda. Journal of World
model. Information & Management, 43(6), 740–755. Business, 52(3), 327–342.
Kirca, A. H., & Yaprak, A. (2010). The use of meta‐analysis in international Paul, J., & Rosado‐Serrano, A. (2019). Gradual internationalization vs born‐
business research: Its current status and suggestions for better global/international new venture models: A review and research
practice. International Business Review, 19(3), 306–314. agenda. International Marketing Review, 36(6), 830–858.
Knoll, J., & Matthes, J. (2017). The effectiveness of celebrity Pigott, T. D. (2012). Missing data in meta‐analysis: Strategies and
endorsements: A meta‐analysis. Journal of the Academy of approaches, Advances in meta‐analysis (pp. 79–107). Springer.
Marketing Science, 45(1), 55–75. Rana, J., & Paul, J. (2020). Health motive and the purchase of organic food:
Kumar, A., Paul, J., & Unnithan, A. B. (2020). ‘Masstige’ marketing: A A meta‐analytic review. International Journal of Consumer Studies,
review, synthesis and research agenda. Journal of Business Research, 44(2), 162–171.
113, 384–398. Randhawa, K., Wilden, R., & Hohberger, J. (2016). A bibliometric review of
Lim, W. M., Yap, S.‐F., & Makkar, M. (2021). Home sharing in marketing open innovation: Setting a research agenda. Journal of Product
and tourism at a tipping point: What do we know, how do we know, Innovation Management, 33(6), 750–772.
and where should we be heading? Journal of Business Research, 122, Rebouças, R., & Soares, A. M. (2021). Voluntary simplicity: A literature
534–566. review and research agenda. International Journal of Consumer
Lipsey, M. W., & Wilson, D. B. (2001). Practical meta‐analysis. SAGE Studies, 45(3), 303–319.
Publications, Inc. Rosado‐Serrano, A., Paul, J., & Dikova, D. (2018). International franchising:
Littell, J. H., Corcoran, J., & Pillai, V. (2008). Systematic reviews and meta‐ A literature review and research agenda. Journal of Business
analysis. Oxford University Press. Research, 85, 238–257.
Marshall, I. J., Noel‐Storr, A., Kuiper, J., Thomas, J., & Wallace, B. C. (2018). Roschk, H., & Hosseinpour, M. (2020). Pleasant ambient scents: A meta‐
Machine learning for identifying randomized controlled trials: An analysis of customer responses and situational contingencies. Journal
evaluation and practitioner's guide. Research Synthesis Methods, 9(4), of Marketing, 84(1), 125–145.
602–614. Rossi, P. E., Allenby, G. M., & McCulloch, R. (2012). Bayesian statistics and
Marshall, I. J., & Wallace, B. C. (2019). Toward systematic review marketing, John Wiley & Sons.
automation: A practical guide to using machine learning tools in Rowley, C., & Paul, J. (2021). Introduction: The role and relevance of
research synthesis. Systematic Reviews, 8(1), 1–10. literature reviews and research in the Asia Pacific. Asia Pacific
Mitchell, T. M. (1997). Does machine learning really work? AI Magazine, Business Review, 27, 1–5. https://doi.org/10.1080/13602381.2021.
18(3), 11. 1894839
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Group, P. (2009). Rubera, G., & Kirca, A. H. (2012). Firm innovativeness and its performance
Preferred reporting items for systematic reviews and meta‐analyses: outcomes: A meta‐analytic review and theoretical integration.
The PRISMA statement. PLoS Medicine, 6(7):e1000097. Journal of Marketing, 76(3), 130–147.
Samaha, S. A., Beck, J. T., & Palmatier, R. W. (2014). The role of culture in Viechtbauer, W. (2010). Conducting meta‐analyses in R with the metafor
international relationship marketing. Journal of Marketing, 78(5), package. Journal of Statistical Software, 36(3), 1–48.
78–98. Vishwanathan, P., van Oosterhout, H., Heugens, P. P., Duran, P., &
Siddaway, A. P., Wood, A. M., & Hedges, L. V. (2019). How to do a Van Essen, M. (2020). Strategic CSR: A concept building meta‐
systematic review: A best practice guide for conducting and analysis. Journal of Management Studies, 57(2), 314–350.
reporting narrative reviews, meta‐analyses, and meta‐syntheses. White, I. R. (2015). Network meta‐analysis. The Stata Journal, 15(4), 951–985.
Annual Review of Psychology, 70, 747–770. Xie, E., Reddy, K., & Liang, J. (2017). Country‐specific determinants of
Södergren, J. (2021). Brand authenticity: 25 Years of research. cross‐border mergers and acquisitions: A comprehensive review and
International Journal of Consumer Studies, 45, 645–663. https://doi. future research directions. Journal of World Business, 52(2), 127–183.
org/10.1111/ijcs.12651 Zhao, X., Wu, C., Chen, C. C., & Zhou, Z. (2022). The influence of
Steel, P. D., Beugelsdijk, S., & Aguinis, H. (2021). The anatomy of an corporate social responsibility on incumbent employees: A meta‐
award‐winning meta‐analysis: Recommendations for authors, analytic investigation of the mediating and moderating mechanisms.
reviewers, and readers of meta‐analytic reviews. Journal of Journal of Management, 48(1), 114–146. https://doi.org/10.1177/
International Business Studies, 52(1), 23–44. 0149206320946108
Steel, P. D., & Kammeyer‐Mueller, J. D. (2002). Comparing meta‐analytic
moderator estimation techniques under realistic conditions. Journal
of Applied Psychology, 87(1), 96–111.
Steel, P. D., Kammeyer‐Mueller, J. D., & Paterson, T. A. (2015). Improving How to cite this article: Paul, J., & Barari, M. (2022).
the meta‐analytic assessment of effect size variance with an Meta‐analysis and traditional systematic literature
informed Bayesian prior. Journal of Management, 41(2), 718–743.
reviews—What, why, when, where, and how? Psychology &
Verma, V., Sharma, D., & Sheth, J. (2016). Does relationship marketing
matter in online retailing? A meta‐analytic approach. Journal of the Marketing, 39, 1099–1115. https://doi.org/10.1002/mar.21657
Academy of Marketing Science, 44(2), 206–217.

SLR 2 JP

Uploaded by

Document Informationclick to expand document information

Copyright:

Available Formats

SLR 2 JP

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

SLR 2 JP

Uploaded by

Copyright:

Available Formats

15206793, 2022, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/mar.21657 by Indian Institute Of Management, Wiley Online Library on [24/10/2022].

Meta‐analysis and traditional systematic literature

Justin Paul1,2 | Mojtaba Barari3

Psychol Mark. 2022;39:1099–1115. wileyonlinelibrary.com/journal/mar | 1099

TABLE 1 Differences between traditional SLRs and meta‐analysis

Differences Traditional SLRs Meta‐analysis

Abbreviation: SLR, systematic literature review.

Meta‐analysis provides an overview of relationships in a research area,

TABLE 2 Meta‐analysis process

TABLE 3 Meta‐analysis process

Defining research question

Searching literature • Define keyword search in related or unrelated research field

Define inclusion/exclusion • Include quantitative research which tested desired relationships

Abstract • Overview of the main question in the meta‐analysis

Introduction • Start with a real example to highlight the topic of interest

Conclusion • Research theoretical and empirical contributions in a summary table

characteristics, work environment, and training interventions) on the Stata

Comprehensive Meta‐Analysis 4.5.1 | Title

You might also like