Narrative Review & Meta-Analysis

- A Step by Step Guidance -

Dr.med. Dr.Sc. dr. Yanto Sandy Tjang, SpBTKV(K),
MBA, MPH, MSc, PhD, FACS, FACC, FETCS, FICS
 What Is Literature

 Books  Audio-visual material

 Journals  CDs/DVDs
 Conference papers  Electronic databases
 Theses and  Government reports
dissertations  Magazines
 Bibliographies  Newspapers
 Maps  Grey literature
 Internet  Interviews and other
 Indexes/Abstracts unpublished research
Types of Literature
Research Loop
Types of Knowledge Synthesis
Level of Evidence
 Review of Literature
 A body of text that aims to review the critical points of current
knowledge on a particular topic
 A comprehensive survey of publications in a specific field of study
or related to a particular line of research
 A summary of existing published literature made by experts who
select and weigh findings available from the literature
 A summary and interpretation of research findings
reported in the literature
 A process and documentation of the current relevant research
literature regarding a particular topic or subject of interest
 Review of Literature - Purposes
 Define and limit problem
 Develop familiarity with topic
 Limit research to a subtopic within larger body of knowledge
 Place study in historical perspective
 Analysis of way in which study relates to existing knowledge
 Avoid unintentional and unnecessary replication
 Awareness of prior studies so as to avoid unneeded replication
 Replication is reasonable if it is needed to verify prior results, investigate results that failed
to be significant, or relate problem to a specific site
 Select promising methods and measures
 Knowledge of and insight into specific research designs for investigating a problem
 Awareness of specific instruments, sampling procedures, and data analyses
 Relate findings to previous knowledge and suggest future research needs
 Relating prior research to what is known places current studyin perspective
 This knowledge allows researcher to focus problem on what is not known
 Develop research hypotheses
 Suggestions for specific research hypotheses
 Review of Literature - Designs
Narrative Review  Selective review of the literature that
broadly covers a specific topic
 Does not follow strict systematic
methods to locate and synthesize
articles
Systematic Review  Utilizes exacting search strategies to
make certain that the maximum
extent of relevant research has been
considered
 Original articles are methodologically
appraised and synthesized
Meta-Analysis  Quantitatively combines the results of
studies that are the result of a
systematic literature review
 Capable of performing a statistical
analysis of the pooled results of
relevant studies
Review of Literature - Designs (2)
Narrative Review
 Steps in Narrative Review

 Identifying a topic
7. Write & present
 Searching and finding Literature review
1. Identify topic

literature
 Evaluating literature 6. Syntesize 2. Search & find
literature literature
 Reading literature critically
 Analyzing literature
5. Analyze
 Synthesizing literature literature 3. Evaluate
literature

 Writing and presenting 4. Read

Literature

literature review critically

 Searching and Finding Literature

 Bibliographic information, abstract

and full text
 Formal and informal sources of
literature
 Print and online literature
 Evaluating Literature

 Evaluation for relevance

 Index of a book, chapter or
section headings, abstract
of an article, introduction
and conclusion, references
or bibliography
 Evaluation for reliability
 Audience, authority, bias,
currency, scope
 Critical Reading

 Passive vs active reading

 Previewing
 Reading
 Taking notes
 Responding critically
 Analyzing Literature

 Varying definitions of key

terms
 Methodology used
 Enough evidence for
claims?
 Findings consistent with
those of similar studies?
 Synthesizing Literature

 How does each reading

relate to topic and purpose?
 Define argument/thesis
 Identify major trends or
patterns emerging from the
readings
 Reassemble notes based on the
results of readings, using
organizational aids such as
post-its, flags, etc.
 Create a detailed topic outline
 Synthesizing Literature (2)
 Note on topic outline relationships among studies: which
researchers, what page, etc. support each point?
 Note consistency of results from study to study.
 Note discrepancies among studies and provide possible
explanations such as dates of studies, different
methodologies.
 Note landmark studies and if replicated.
 Note how individual studies help illustrate or advance
theoretical beliefs.
 Note gaps or areas needing more research.
 Make sure detailed outline follows a logical sequence of
topics and subtopics. This will give the literature review
the coherence it needs.
 Writing and Presenting Literature Review

 A general organization looks like a funnel

 Broader topics
 Subtopics
 Studies like yours
 How to Organize Studies
 Chronological
 By publication date
 By trend
 Thematic
 A structure which considers
different themes
 Methodological
 Focuses on the methods of
the researcher, e.g.,
qualitative versus
quantitative approaches
 Making Links Between Studies
Agreements
 Similarly, author B points to…
 Likewise, author C makes the case
that…
 Author D also makes this point…
 Again, it is possible to see how author
E agrees with author D…
Disagreements
 However, author B points to…
 On the other hand, author C makes the
case that…
 Conversely, author D argues…
 Nevertheless, what author E
suggests…
 Summary Table
 It is useful to prepare.
 Such a table provides a quick
overview that allows the
reviewer to make sense of a
large mass of information.
 The tables could include
columns with headings such
as:
 Author
 Type of study
 Sample
 Design
 Data collection approach
 Key findings
 A Good Literature Review Is…

 Focused - The topic should be narrow. It should only present ideas and
only report on studies that are closely related to topic.
 Concise - Ideas should be presented economically. Don’t take any more space
than it needs to present the ideas.
 Logical - The flow within and among paragraphs should be a smooth,
logical progression from one idea to the next.
 Developed - Don’t leave the story half told.
 Integrative - The paper should stress how the ideas in the studies are
related. Focus on the big picture. What commonality do all the studies share?
How are some studies different than others? The paper should stress how all
the studies reviewed contribute to the topic.
 Current - The review should focus on work being done on the cutting
edge of the topic.
Meta-Analysis
 History

1990s:
Mid-1980s, explosion in
methods popularity,
1977: first develop. Eg. esp. in
modern Hedges, medical
meta- Olkin, research
1904: analysis Hunter &
quant. lit. published by Schmidt
review by Smith &
Pearson Glass (1977)
 The Popularity of Meta-Analysis
publications

3000

2500

2000
Number of Publications

1500

1000

500

0
93-94 94-95 95-96 96-97 97-98 98-99 99-00 2000-1 2001-2 2002-3 2003-4
Year of Publications

Number of Meta Analysis publications are steadily increasing since 1993. We graphed
the counts of journal articles included “meta analysis” as “publication type” from
Pubmed, from years 1993 through 2004
 Definition

• Quantitative approach for systematically combining

results of previous research to arrive at conclusions
about the body of research.
– Quantitative: numbers
– Systematically: methodological
– Combining: putting together
– Previous research: what's already done
– Conclusions: new knowledge
 Function of Meta-Analysis

 Identifies heterogeneity
 Increases statistical power and precision of the study
 Develop ,refine, and tests hypothesis
 Calculates sample size for future studies
 Identifies data gaps
 Reduces the subjectivity of study comparisons
 Advantages of Meta-Analysis
 Focuses attention on trials as an evaluation tool to increase the
impact of trials on clinical practice.
 Encourages designing of good trial and increases strength of
conclusions.
 Make the results fit for generalising to a larger population.
 Improves precision and accuracy of estimates through use of
more data sets.
 May increase the statistical power to detect an effect.
 Inconsistency of results across studies can be quantified,
analyzed and corrected.
 Hypothesis testing can be applied on summary estimates.
 Moderators can be included to explain variation between studies.
 The presence of publication bias can be investigated.
 Disadvantages of Meta-Analysis
 Meta-Analysis may discourage large definitive trials.
 Increases tendency to unwittingly mix different trials
and ignore differences.
 Potential for tension between meta-analyst and
conductors of original trials may introduce biasness.
 Meta-Analysis of several small studies may not predict
the results of a single large study.
 Sources of bias are not controlled by the method.
 A good meta-analysis of badly designed studies will still
result in bad statistics.
 How a Meta-Analysis Work
 Individual studies – collecting similarity studies from previous research
 Effect sizes – transform data (analysis results) into effect size reflect
the magnitude of treatment effect or the strength of a relationship
between two variables
 Precision – The effect size for each study is bounded by as confidence
interval, reflect the precision of effect size
 Study weight – ideal studies (sample size are larger) are assigned
relatively high weight
 P-Value – a p-value for a test of the null hypothesis, if p<0.05 reject
null hypothesis
 Summary effect – summary the effect size from all studies, including
mean effect size (fix effect), CI, weight, p-value, ES heterogeneity,
random effect, publication bias, etc
 When Can We Do Meta-Analysis?
• Meta-analysis is applicable to collections of research:
• Examine same constructs and relationships
• Have findings that can be configured in a comparable statistical form (eg.
effect sizes, correlation coefficients, odds-ratios, etc.)
• “Comparable” given the question at hand
o Objective of study (effect or variability)
o Population of study
o Type of study (RCT, Case-Control, or Case Report)
o Patient characteristics
• Research findings suitable to Meta-Analysis
• Central tendency research: prevalence rates
• Pre-post contrasts: growth rates
• Group contrasts:
o Experimentally created groups
o Naturally occurring groups
• Association between variables
Steps in Meta-analysis
Define the research question

Perform the literature search

Select the studies

Extract the data

Analyze the data

Examine heterogeneity

Assess publication bias

Report the results

 Define Research Question

• In patients with coronary artery disease, does

vitamin E supplementation decrease the risk of
death?
• Patients digest carotenoids will decrease the chance
of lung cancer occurrence
 Literature Searching

• Be methodological: plan first

• List of popular databases to search
– Pubmed/ Medline/ Embrase
– Cochrane Review/ Trials register
– List every possible database
• Other strategies
– Hand search (library)
– Personal references and email
– Web, eg. Google scholar
 How to Search for literature

• Formulate your question appropriately

• If you are searching Pubmed
– Use Medical Subject Headings (MeSH) [1]
– Lookup word in text word, abstract, title [2]
– Combine [1] with [2] using boolean logic
– Set up proper filters
• For Others, use text word, abstract
Boolean Logic

AND Zone,
covering common
area between
two ellipses

OR Zone, covering
the two ellipses
 Example: Research Question

• Let's say we want to know whether streptokinase is

protective for death from acute myocardial infarction.
How should we set up a search strategy?
• We will search Pubmed only
• The search:
– “streptokinase”[text word] OR “acute myocardial
infarction”[text word] produces ALL articles that contain
EITHER streptokinase OR acute myocardial infarction
anywhere in the text – inclusive, many
– streptokinase [text word] AND “acute myocardial infarction”
[text word] will capture ONLY those subsets that have BOTH
streptokinase AND acute myocardial infarction anywhere in
the text – restrictive, few
Pubmed Screen
 Select Studies
• Cannot include all studies
• Keep the ones with
– high levels of evidence
– good quality
– check with QUOROM guidelines
• Inclusion and exclusion criteria
• Usually, Meta-Analysis are done with
Randomized Clinical Trials
• Case series, and case reports
definitely out
• Selection problems are major
problems
• Use PRISMA flow diagram
 Extract Data
• Create a spreadsheet (Excel, or Open Office Calc)
• For each study, create the following columns:
– Name of the study
– Name of the author, year published
– Number of participants who received intervention
– Number of participants who were in control arm
– Number who developed outcomes in intervention
– Number who developed outcomes in control arm
 Analyze Data

• Combine data to arrive at a summary, 3 measures

– Effect size (Odds Ratio, RR)
– Variance with 95% Confidence Interval
– Test of heterogeneity
• Two graphs
– Forest Plot
– Funnel Plot
• Examine why the studies are heterogeneous, if they are
• Use Statistical Packages
– Free Software: EpiMeta, Revman, R Statistics Package
– Non-free: STATA, SAS
 Heterogeneity

• Statistical test for heterogeneity

• Visual inspection graphical approach – Forest Plot
• Meta-Regression
– Unit of regression study
– Dependent variable: study-specific effect estimate
– Independent variables: study-specific
characteristics (eg. Study design, geographic
location, length of follow-up)
 Publication Bias

• Study showing beneficial effects of new treatment

more likely to be published than one showing no
effect.
• Negative trials assumed to contribute less; never
show up in the literature base.
• Use several approaches to avoid this.
• Use Funnel Plots to examine the influence of
publication bias.
 Report Result
• Abstract
– Background (include statement of the main research question)
– Methodology/PrincipalFindings
– Conclusions/Significance
• Introduction
• Methods
– Searching - describe information sources, restrictions
– Selection - inclusion and exclusion criteria
– Data abstraction
– Validity Assessment
– Study characteristics e.g. type of study designs, participantsʼ characteristics
– Quantitative data synthesis - measures of effect, method of combining results, missing data;
how statistical heterogeneity was assessed
• Results
– Flow of included studies
– Studycharacteristics
– Quantitative data synthesis-agreement on the selection and validity assessment, simple
summary results, Funnel plots, Forest plots
• Discussion
• Limitations