REVIEW

Clinical Practice Guidelines: A Primer on

Development and Dissemination
M. Hassan Murad, MD, MPH

Abstract

Trustworthy clinical practice guidelines should be based on a systematic review of the literature, provide
ratings of the quality of evidence and the strength of recommendations, consider patient values, and be
developed by a multidisciplinary panel of experts. The quality of evidence reflects our certainty that the
evidence warrants a particular action. Transforming evidence into a decision requires consideration of the
quality of evidence, balance of benefits and harms, patients’ values, available resources, feasibility of the
intervention, acceptability by stakeholders, and effect on health equity. Empirical evidence shows that
adherence to guidelines improves patient outcomes; however, adherence to guidelines is variable.
Therefore, guidelines require active dissemination and innovative implementation strategies.
ª 2017 Mayo Foundation for Medical Education and Research n Mayo Clin Proc. 2017;92(3):423-433

C
linical practice guidelines are system- credentials granted by medical schools or state
atically developed statements that authorities; however, such credentialing
intend to assist clinicians and patients proved to be an insufficient guarantee of qual- For editorial
in making decisions about appropriate health ity.5 Further standardization and organization comment, see
care in specific circumstances.1 Guidelines of the medical profession necessitated the page 327
aim to improve the quality of patient care by development of guidelines. Guidelines in their
From the Division of Pre-
encouraging interventions of proven benefit current form started in the 1970s and were ventive Medicine, Mayo
and discouraging the use of ineffective or primarily based on the consensus of expert Clinic, Rochester, MN.
potentially harmful interventions; to reduce panels (eg, the National Institutes of Health
unnecessary variation in practice; to lessen dis- Consensus Development Program).6 Experts
parities; to empower patients; and to influence recommended management approaches they
public policy.2 have used in their practice and cited references
Production of guidelines has skyrocketed they recalled or were able to identify without
during the past 30 years. Currently, the library an explicit systematic search. With the emer-
of the Guidelines International Network has gence of evidence-based medicine as a princi-
6187 documents from 76 countries, and the ple for decision making in the 1980s and
National Guideline Clearinghouse in the United coining of the term in 1991,7 more rigorous
States has 2017 guideline summaries.3,4 Guide- approaches for guideline development have
lines are critical for developing disease perfor- emerged. The next generation of guidelines
mance measures and defining high-value care. would emphasize research evidence over
This primer includes a description of the opinion and base recommendations on the
modern approach to developing guidelines; design of studies contributing evidence
the criteria for trustworthy guidelines; advice on benefits and harms of interventions.
on how clinicians can appraise, interpret, For example, the American College of
and implement practice recommendations; Cardiology/American Heart Association used
challenges and limitations of guidelines; and ratings of A, B, and C that were exclusively
a future research agenda to address current dependent on study design (level A, multiple
knowledge gaps. randomized trials or meta-analyses; level B, a
single trial or nonrandomized studies; and
HISTORICAL PERSPECTIVE level C, consensus opinion of experts, case
Until the 1970s, medical actions were indi- studies, or standard of care).8 In 2000, many
rectly regulated through the training and different guideline systems existed, which

Mayo Clin Proc. n March 2017;92(3):423-433 n http://dx.doi.org/10.1016/j.mayocp.2017.01.001 423

www.mayoclinicproceedings.org n ª 2017 Mayo Foundation for Medical Education and Research
 MAYO CLINIC PROCEEDINGS

low.13 Thus, this newer approach uncovered

ARTICLE HIGHLIGHTS additional factors affecting the quality of evi-
dence that were otherwise implicit. Judgments
d Trustworthy clinical practice guidelines require a systematic
made using GRADE had good reproducibility
review to select the best available evidence and should explicitly and reliability compared with intuitive or
evaluate the quality of evidence. global judgments and were consistent among
d Factors that reduce the quality of evidence are risk of bias, raters (interrater reliability of 0.72)14 even
indirectness, inconsistency, imprecision, and likelihood of pub- when panel members had short training in
GRADE (two 1-hour didactic sessions).15
lication and reporting bias.
GRADE was adopted by more than 100 orga-
d Transforming evidence into a decision requires consideration of nizations and has become, to some extent, the
the quality of evidence, balance of benefits and harms, patients’ gold standard (when other systems are
values, resources, feasibility, acceptability, and equity. used,16,17 they usually depend on GRADE
d Empirical evidence shows that guidelines improve patient out- domains and components).
In 2011, the National Academy of Medi-
comes; however, guidelines require active dissemination and
cine (formerly the Institute of Medicine) pub-
innovative implementation strategies. lished criteria for trustworthy guidelines that
greatly overlapped with GRADE (in emphasis
on the systematic review process, rating
was confusing for stakeholders. The 6 most the quality of evidence and strength of recom-
prominent systems had low reproducibility mendations, and considering nonevidence
of judgments and did not fit the needs of all factors).18 These criteria were highly dissemi-
stakeholders.9 nated and cited, and they motivated guideline
Subsequently, it became apparent that study developers to improve the rigor of guidelines
design (a surrogate for the risk of bias) was insuf- (Table 1). Similar criteria were also produced
ficient.9 Studies with the same design (eg, by the Guidelines International Network,19
randomized trials) can have high risk of bias or the National Institute for Health and Clinical
low risk of bias. Factors other than risk of bias Excellence in the United Kingdom,20 and the
affect certainty in the evidence (eg, precision of World Health Organization.21
estimates of effect, consistency of effect across
studies). Moreover, factors other than evidence EVALUATING THE QUALITY OF EVIDENCE
(eg, patient values) affect decision making.10 A systematic review is a mechanism to reduce
Considering these challenges and to unite the risk of biased selection of evidence and
many of these frameworks, the GRADE should be conducted once the scope and
approach (Grading of Recommendations, preliminary questions of the guideline are
Assessment, Development, and Evaluation) determined. Meta-analysis may or may not
was developed in 2003. It advanced guideline be appropriate, but a systematic review is
methodology further by providing a frame- always needed. In the context of a guideline,
work for rating the quality of evidence based the quality of evidence (also called certainty
on 8 distinct domains (as opposed to study in the evidence, strength of the evidence,
design only). The construct of the quality of and confidence in the effect estimates) reflects
evidence was then defined to reflect the extent the extent of our confidence that the estimates
of our confidence that the estimate of an effect of an effect are adequate to support a partic-
is adequate to support a particular decision or ular decision or recommendation.11 A good
recommendation.11 An evidence to decision starting place to determine this confidence
framework based on 8 criteria (as opposed to depends on study design (higher confidence
intuitive or global judgment) was provided in randomized trials and lower confidence in
by GRADE to assist in developing actions observational studies). There are, however,
based on evidence.12 Empirical evaluation other factors that can moderate this initial
showed that recommendations with a rating rating of confidence. We are less confident of
of A (based on multiple randomized trials or the effect of an intervention on benefits
meta-analyses) would have been rated using and harms (1) if the evidence is derived
GRADE as high, moderate, low, and very from studies with methodological limitations
n n
424 Mayo Clin Proc. March 2017;92(3):423-433 http://dx.doi.org/10.1016/j.mayocp.2017.01.001
www.mayoclinicproceedings.org
CLINICAL PRACTICE GUIDELINES

(high risk of bias), (2) if the evidence is indi-

TABLE 1. Criteria for Trustworthy Guidelines
rect (eg, the studies addressed populations or
outcomes different from those targeted by d Be based on a systematic review of the literature
d Be developed by a knowledgeable, multidisciplinary panel of experts and
the guideline), (3) if the evidence is inconsis-
representatives from key affected groups
tent or heterogeneous (ie, the effect is different d Consider important patient subgroups and patient values and preferences
across studies, which can be evaluated qualita- d Be based on an explicit and transparent process that minimizes distortions, biases,
tively or statistically using the I2 statistic and and conflicts of interest
other tests of heterogeneity), (4) if the esti- d Provide a clear explanation of the logical relationships between alternative care

mates of effect were imprecise (ie, the CI in- options and health outcomes
cludes appreciable benefits and harms, d Provide ratings of the quality of evidence and the strength of recommendations

making the decision different across the d Be reconsidered and revised as appropriate when important new evidence warrants

boundaries of the CI), or (5) if there was an modifications of recommendations

indication of publication or reporting bias This Table was adapted from the National Academy of Medicine/formerly the Institute of
(ie, studies with positive results were more Medicine.18
likely to be published than were studies with
negative results or no effect, which may be
detected statistically using funnel plots and development have been proposed, although
other tests). little comparative evidence is available to sup-
Conversely, there are scenarios that port one preferred approach.34,35
increase confidence in evidence derived from Table 3 presents factors to be consid-
observational studies (eg, when a large effect ered when making a recommendation as
is noted). These factors are considered the described in the recently developed evidence
domains of the construct of quality of evidence to decision framework.12 It is important to
and are used to reach a final rating of the quality recognize that some of these factors may
of evidence (after starting with a rating based on not be critical for decision making in certain
study design). Table 2 includes examples to situations. For example, when the balance
illustrate how these domains are applied. of benefits and harms is clear, patient
values may be less relevant. Similarly, cost
FROM EVIDENCE TO RECOMMENDATION may be critical for some decisions but not
The beginnings of evidence-based medicine for others.
have demonstrated a great desire to base deci-
sion making on evidence (as opposed to the DETERMINING THE STRENGTH OF A
previous approach of depending on expert RECOMMENDATION
opinion). However; evidence-based practi- The National Academy of Medicine and other
tioners realized very quickly that evidence authorities18-21 have indicated that a key
alone is insufficient for decision making. criterion of a trustworthy guideline is that a
Hence, the second principle of evidence- recommendation given to a patient or a popula-
based medicine acknowledged that decisions tion should have a strength attached to it. For
should also consider several nonevidence example, in GRADE, recommendations are
factors.30 For example, chemotherapy can labeled as strong or weak (also called condi-
extend survival in lymphoma but causes tional or discretionary). This strength reflects
serious adverse effects. Variation in patient the extent to which we can be confident
values can lead to different recommendations that the desirable effects of an intervention
(eg, making a stronger recommendation for outweigh the undesirable effects.36 An example
chemotherapy for younger patients and mak- of a strong recommendation is recommending
ing a weaker recommendation for older antiviral therapy to patients with immune-
patients who may favor quality of life over active chronic hepatitis B infection (where the
extending survival).31 Guideline developers benefits clearly outweigh the harms); whereas
can obtain information on patient values only a weak recommendation was given for
from surveys, from qualitative studies, or by using antiviral agents in those with compen-
engaging patients in the process of guideline sated liver disease and low-level viremia.37
development.32,33 Frameworks for engaging Strong recommendations usually require
patients and other stakeholders in guideline high-quality evidence. Less commonly, a

Mayo Clin Proc. n March 2017;92(3):423-433 n http://dx.doi.org/10.1016/j.mayocp.2017.01.001 425

www.mayoclinicproceedings.org
 MAYO CLINIC PROCEEDINGS

TABLE 2. Examples of the Domains Used to Assess the Quality of Evidence (Also Called Certainty in Evidence)
Domain (explanation) Example
Quality of evidence is rated down
Methodological limitations (What is the extent A meta-analysis of 14 randomized trials showed that perioperative insulin
of bias in the available studies?) infusion reduced mortality in patients undergoing surgery (relative risk, 0.69;
95% CI, 0.51-0.94). However, less than half of the trials concealed allocation
sequence from the investigator enrolling patients; less than half of the trials
blinded outcome assessors, and most trials did not report on those lost to
follow-up.22 These limitations suggest increased risk of bias that leads to
rating down the quality of evidence.
Indirectness (Does the available evidence fit the Trials of angiotensin-converting enzyme inhibitors in patients with diabetes can
population and interventions of interest?) designate death, incidence of end-stage renal disease, or proteinuria as
outcomes. Trials using death and incidence of end-stage renal disease
provide the highest-quality evidence for decision making. Conversely, trials
that use proteinuria as a surrogate for clinical outcomes produce indirect
evidence that leads to rating down the quality of evidence.23
Inconsistency (Do the results substantially differ A meta-analysis of 22 studies showed that medical students who learned using
across published studies?) a self-directed approach had a moderate increase in knowledge compared
with those who learned using traditional didactic curricula. The increase in
knowledge was estimated to be 0.42 SD (95% CI, 0.14-0.70 SD). However,
the effect was very inconsistent across studies (I2¼94%) and suggested
important heterogeneity beyond what was expected by chance.24 This
heterogeneity leads to rating down the quality of evidence.
Imprecision (Would our decision differ across the At 30 d and compared with endarterectomy, carotid stenting was associated
boundaries of the CI?) with a nonsignificant reduction in the risk of death (relative risk, 0.61; 95%
CI, 0.27-1.37).25 If the lower boundary of CI was to represent the truth, this
would mean that stenting reduced death by 73%. If the upper boundary of
the CI was to represent the truth, this would mean that stenting increased
death by 37%. Because the CI included appreciable benefit and harm, this
evidence is considered imprecise, which leads to rating down the quality of
evidence.
Publication bias (Are there unpublished studies Data on 74% of patients enrolled in trials evaluating the antidepressant
that show less impressive results than reboxetine remained unpublished. Published data overestimated the benefit
published ones?) of reboxetine vs placebo by up to 115% and underestimated harm.26 This is
an example of publication bias in which sponsors of trials chose the data
that are most impressive to publish, which leads to rating down the quality
of evidence.
Quality of evidence is rated up
Large effect (Relative risk >2.0 or <0.5 can be Meta-analysis of observational studies showed that infants with a front
used to define a large effect.) sleeping position had increased risk of sudden infant death syndrome
compared with a back sleeping position (odds ratio, 4.1; 95% CI, 3.1-5.5).
The large effect (4 times increased likelihood) increases certainty that a
strong association exists and can lead to rating up the quality of evidence.27
Dose-response effect (The higher the dose, the An observational study of 2154 patients with septic shock showed a strong
larger the effect.) relationship between the delay in effective antimicrobial initiation and
in-hospital mortality (adjusted odds ratio, 1.12 for each additional hour
delay). This dose-response relationship increases certainty that a strong
association exists and can lead to rating up the quality of evidence.28
Plausible confounding strengthens the Observational studies have documented lower mortality rates in not-for-
association (Despite a likely confounder that profit hospitals compared with for-profit hospitals. This is despite
would weaken the association, the effect confounding by the fact that sicker patients usually are hospitalized in not-
remains significant.) for-profit hospitals (thus, not-for-profit hospitals would be expected to
have higher mortality rates). Despite this, it was observed that not-for-profit
hospitals had lower mortality rates. This residual confounding strengthens
the observed association and can lead to rating up the quality of evidence.29

n n
426 Mayo Clin Proc. March 2017;92(3):423-433 http://dx.doi.org/10.1016/j.mayocp.2017.01.001
www.mayoclinicproceedings.org
CLINICAL PRACTICE GUIDELINES

strong recommendation can be justified with

TABLE 3. Factors to Consider When Making a Recommendation
low-quality evidence in 5 paradigmatic situa-
tions (Table 4).38 1. Is the question that the guideline is addressing a priority?
2. Balance of benefits and harms
a. How substantial are the desirable anticipated effects?
WHERE TO FIND GUIDELINES? b. How substantial are the undesirable anticipated effects?
The National Guideline Clearinghouse is a c. Does the balance between desirable and undesirable effects favor the interven-
public resource that is supported by the tion or the alternative?
Agency for Healthcare Research and Quality. 3. What is the overall quality of the evidence?
Guidelines indexed in this register have to 4. Is there important uncertainty or variability in how much patients value the
meet certain criteria, particularly, being based outcomes?
on a systematic review of the evidence and 5. Resources
documenting an assessment of the benefits a. How large are the resource requirements (costs)?
b. What is the certainty in the evidence of resource requirements (costs)?
and harms of the recommended care and alter-
c. Is the intervention cost-effective?
native care options.4 Another way to find
6. What is the effect on health equity?
guidelines is to search websites of professional
7. Is the intervention acceptable to patients, caregivers, and health care professionals?
societies relevant to a particular topic. Guide- 8. Is implementing the intervention feasible for patients, caregivers, and health care
lines can also be found through bibliographic professionals?
database searches (eg, MEDLINE). The search
This Table was adapted from BMJ.12
can use a database filter (eg, in PubMed, there
is a publication type called guideline), a
controlled vocabulary (eg, Medical Subject
preferences, and resources considered? Was
Heading terms in PubMed), or text words
the strength of recommendations appropriate?
(eg, guideline, consensus, recommendation, or
Was the influence of conflicts of interest mini-
standards). Incorporation of guideline recom-
mized?). These questions address the major
mendations in order sets and decision support
criteria of a trustworthy guideline.
tools in electronic medical records reduces the
need for searching for guidelines but hides the
rationale and rigor of guideline development IMPLEMENTATION
from end users.
Implementation From the Perspective of a
HOW TO JUDGE THE QUALITY OF A Clinician-Patient Dyad
GUIDELINE? The key information needed to implement a
There are more than 20 tools available to recommendation includes a clear description
appraise a guideline.39 Some tools focus on of the population to whom a recommendation
implementation.40 The AGREE (Appraisal of is appropriate, the baseline risk of this popula-
Guidelines, Research, and Evaluation) Collab- tion, the quality of evidence, and the strength
oration developed the AGREE II, which is the of the recommendation. An example of a
most validated and extensively used tool. The recommendation that contains these 4 pieces
AGREE II is a generic instrument that aims to of information is presented in Table 5.
assess the process of guideline development When the recommendation is strong,
and reporting and has 23 items grouped into clinicians should offer the intervention to
6 domains (scope and purpose, stakeholder almost all their eligible patients and convey
involvement, rigor of development, clarity of high confidence that the benefits outweigh
presentation, applicability, and editorial inde- the harms. Extensive discussion or exploration
pendence).41 The use of this tool is helpful of values is not always needed.
for researchers, journal editors, and policy Conversely, if the recommendation is weak
makers but not for a busy clinician reading a or conditional, clinicians need to be more
recently published guideline. A simplified discretionary in applying the recommendation
approach is suggested for clinicians42 that and offer additional discussion to elicit patients’
focuses on only a few questions (Was the values, preferences, cost implications, comor-
recommendation based on the best available bidities, and clinical and social context.42
evidence and is that evidence reported and Weak recommendations imply that although
easily understood? Were patients’ values, the recommended action is appropriate for

Mayo Clin Proc. n March 2017;92(3):423-433 n http://dx.doi.org/10.1016/j.mayocp.2017.01.001 427

www.mayoclinicproceedings.org
 MAYO CLINIC PROCEEDINGS

the condition and the options, and enhance

TABLE 4. Making a Strong Recommendation When the Quality of Evidence Is
their awareness of their own risk of various
Low
outcomes. A meta-analysis of 115 studies
Paradigmatic situation Example showed that decision aids increased patients’
Low-quality evidence suggests benefit Recommendation to increase the knowledge, the proportion of patients with
in a life-threatening situation glucocorticoid dosage in congenital accurate risk perceptions, and the likelihood
adrenal hyperplasia during a febrile
of patients choosing an option congruent
illness
with their values; reduced decisional conflict;
Low-quality evidence suggests benefit Recommendation against screening
and high-quality evidence suggests with whole-body computed
and increased visit time by a median of
harm or a very high cost tomographic scan only 2.6 minutes (range, 8 minutes shorter
Low-quality evidence suggests Recommendation for laparoscopic to 28 minutes longer).45 Developing decision
equivalence, but high-quality over open adrenalectomy in aids requires patient engagement in the design
evidence suggests harm with one patients with unilateral primary and testing process. The most helpful decision
alternative aldosteronism aids are the ones that encourage conversation
High-quality evidence suggests Recommendation for methyldopa during the clinical encounter and activate
equivalence, but low-quality over angiotensin-converting enzyme patients as opposed to those that merely
evidence suggests harm with one inhibitors for hypertension in provide education.46
alternative pregnancy
Low-quality evidence suggests harm in Recommendation against testosterone
Implementation From the Perspective of
a critical outcome that is valued replacement in men with prostate
much more than any of the benefits cancer
Health Care Systems
The strength of a recommendation is an essen-
tial factor that guides implementation of a
most patients, for some of them, the alternative guideline from the perspective of policy
action may be more appropriate. For example, a makers. Strong recommendations imply high
strong recommendation for intranasal gluco- confidence that benefits outweigh harms and
corticosteroid use in allergic rhinitis means that the recommended act is appropriate for
that clinicians should prescribe it to most almost all patients, is feasible, is acceptable
patients and just perhaps provide educational to patients and providers, and does not
material to the patient about how to use it. On exacerbate inequity. Such recommendations
the other hand, implementing a weak recom- may be candidates for quality improvement
mendation for intransal antihistamines requires projects that ensure that most patients would
more discussion during which clinicians need receive the intervention. Process measures
to convey the lack of high-quality evidence in can be structured to monitor health care deliv-
this setting and explore patient preferences, ery. Barriers and facilitators should also be
costs, and other factors affecting the decision explored in the context of health care delivery.
to start treatment.44 Some of these recommendations may be
Shared decision-making tools are highly appropriate for pay for performance initiatives
desired in the setting of conditional recom- and quality metrics.42 Clearly, lower-grade
mendations and uncertain evidence. These (weak, discretionary, conditional) recommen-
tools can present all the plausible options to dations are not candidates for incorporation
patients, increase their knowledge about into quality improvement projects and quality
metrics (unless the metric was to measure that
most patients were engaged in relevant discus-
TABLE 5. An Example of a Recommendation That Contains the 4 Elements sion and were introduced to all the available
Required for Proper Implementation (Description of the Population, Baseline treatment options). Such recommendations
Risk, Quality of Evidence, and Strength of Recommendation) remain, however, the best course of action
Recommendation: “For acutely ill hospitalized medical patients at increased risk of and from a coverage standpoint are what
thrombosis who are bleeding or at high risk for major bleeding, we suggest the most patients should be able to receive.
optimal use of mechanical thromboprophylaxis with graduated compression From a health system perspective, a guide-
stockings.
line could be adopted or adapted. Adaptation
(Weak recommendation, low quality evidence)”43
is a process that may be needed to improve
The recommendation is followed by 2 tools that help the clinician estimate the risk of
implementation in a targeted practice or
thrombosis and bleeding in hospitalized medical patients.
geographic location. The process includes
n n
428 Mayo Clin Proc. March 2017;92(3):423-433 http://dx.doi.org/10.1016/j.mayocp.2017.01.001
www.mayoclinicproceedings.org
CLINICAL PRACTICE GUIDELINES

identification of a guideline, followed by evidence is unreliable.55 Four cluster trials

appraisal and then modification based on the showed that active interventions are required
feedback of local stakeholders, local disease to enhance guideline uptake (eg, educational
prevalence or baseline risk, effect modifiers workshops, paper-based educational mate-
(eg, antimicrobial drug resistance), availability rials, order forms, and reminders).56 Multifac-
of resources, and cultural variations.47 eted interventions targeting different barriers
In addition to integration of guidelines as were more likely to be effective than single
decision support tools in electronic health interventions.55 Qualitative research also
records and initiating quality improvement demonstrated the need for building a culture
projects, dissemination and implementation that enabled guideline implementation.57
can be enhanced using point-of-care technol- An example is the successful implementation
ogy. An example is a mobile device application of performance measures for community-
that provides a multilayered presentation for acquired pneumonia in several academic insti-
point-of-care use. Starting with the recom- tutions in New Jersey. Success was attributed
mendation (the top layer), clinicians can to the wide engagement of stakeholders;
on-demand access a rationale for and key forming multidisciplinary teams championed
information about the recommendation, the by a nurse practitioner; making changes based
quality of evidence, balance between desirable on local input; dissemination via workshops,
and undesirable consequences, values and grand rounds, and mailings; and collecting
preferences, and resource considerations.48 data on process measures.58
Based on clinicians’ time, interest, and exper-
tise, they can choose which layers to view. LIMITATIONS AND FUTURE NEEDS
User testing led to extensive revisions of this A large proportion of published guidelines
application, and most stakeholders expressed were reported to have limited rigor and did
overall satisfaction with the final format.48 not meet all the criteria for a trustworthy
guideline.59-61 Patients or methodologists
Do Guidelines Improve Patient Outcomes? were not included in the guideline develop-
Adherence to guidelines lowered the risk of ment process in most guidelines (71% and
hospitalization in patients with chronic heart 86%, respectively).62 Guidelines also tend to
failure across several European countries.49 address the common or average patient. For
A study of patients with a new diagnosis example, 1 study demonstrated the absence
of primary breast cancer showed that the of incorporating the impact of multiple
greater the number of violations in guideline chronic conditions, sociopersonal context,
adherence, the lower the survival.50 In patients and patient preferences in 29%, 39%, and
with hospital-acquired or ventilator-associated 57% of a sample of guidelines, respectively.62
pneumonia, guideline-adherent initial intrave- This, indeed, suggests that recommendations
nous antibiotic drug therapy was clini- fit the needs of the average patient. True
cally superior, saved more lives, and was practice of individualized medicine requires
less expensive than noneguideline-adherent more nuanced recommendations, which
therapy.51 Guideline-adherent antithrombotic necessitates advances in genomics and the
prophylaxis in patients discharged with atrial basic sciences to produce reliable individual-
fibrillation was also associated with lower ized risk prediction and response to therapy
all-cause and cardiovascular mortality.52 estimates (which is not the case at the present
Numerous other examples exist to document time). The current research evidence is based
better outcomes with guideline adherence. on assessing average treatment effects in a
group of patients that is not conducive to indi-
Improving Guideline Uptake vidualized practice. The current evidence,
Although adherence to guidelines improves however, can be leveraged by conducting
outcomes, nonadherence to guidelines was N-of-1 trials (crossover trials in single individ-
shown to be frequent53,54 and was more likely uals to evaluate treatment effectiveness in
to affect patients who were older or had cancer chronic conditions)63 and rigorous subgroup
or other comorbidites.52 Low adherence to analyses in trials and meta-analyses. Subgroup
guidelines reinforces that passive diffusion of analyses can produce effect estimates stratified

Mayo Clin Proc. n March 2017;92(3):423-433 n http://dx.doi.org/10.1016/j.mayocp.2017.01.001 429

www.mayoclinicproceedings.org
 MAYO CLINIC PROCEEDINGS

with a mechanism for real-time updating) has

TABLE 6. Future Needs in the Field of Practice Guidelines
been proposed68 but not implemented. The
1. Better methodology to engage patients and obtain their perspective. British Medical Journal has launched a rapid
Current methods of patient engagement often lead to tokenistic engagement.33
recommendation initiative to compensate for
Randomized trials to test engagement methods have been performed and seem to
outdated guidelines.69
be feasible.71
2. Reliable mechanism to update guidelines and incorporate new
Another challenge that undermines guide-
evidence. line integrity and uptake is inconsistency in
Although frameworks for updating guidelines have been suggested,72 only 53% of recommendations across guidelines evaluating
surveyed guideline developers had a formal procedure for deciding when a the same evidence on the same condition.70
guideline becomes out of date73 and less than a third of methodology handbooks Conflicting recommendations confuse guide-
included an approach for updating guidelines.68 line users and undermine evidence-based
3. Collaborative initiatives to avoid contradictory recommendations by practice. A summary of key challenges and
different professional societies. future needs is presented in Table 6.
In 2011, five independent guidelines reported recommendations for the management Theoretical harms of guidelines include
of carotid artery stenosis.70 Recommendations from these committees differed. The
leaving insufficient room for clinicians to tailor
ACC/AHA suggested that stenting and endarterectomy were equal options for
care, adversely affecting insurance coverage for
symptomatic patients, whereas the Society for Vascular Surgery and the Australasian
Guidelines recommended endarterectomy as a first choice and suggested stenting
certain interventions, increasing health care
as an alternative only in patients with high anatomical or perioperative risk.70 costs; deskilling clinicians, and subjecting
4. Data sharing to facilitate conducting individual patient meta-analyses. clinicians to litigation or unfavorable profes-
Study-level meta-analysis, which is very frequently performed, produces subgroup sional judgments by auditors and man-
estimates that are highly subject to ecological bias. In comparison, individual patient agers.2,74 Nevertheless, surveys of primary
meta-analysis can produce reliable estimates of effect in patient subgroups; which care physicians showed that they had overall
allows individualized recommendations. However, individual patient meta-analysis positive attitudes toward guidelines and
requires data sharing by trialists. their effectiveness; only a minority was con-
ACC/AHA ¼ American College of Cardiology/American Heart Association. cerned that guidelines may be used to set
performance-related pay, reduce clinical
freedom, or stifle innovation.75-77
by patient characteristics (effect modifiers) and
can help with individualizing care. However, CONFLICTS OF INTEREST
spurious findings are the norm in the current It is well-known that conflicts of interest affect
practice of subgroup analysis.64 Individual the opinions and recommendations of experts
patient meta-analysis can offer reliable individ- and can compromise guideline validity.78
ualized estimates but requires data sharing by Intellectual conflicts are common but remain
trialists and remains quite uncommon. implicit and are rarely addressed (compared
In terms of validity and reproducibility of with financial ones). Forming guideline
judgments made by guideline developers, the panels without conflicted experts (who are
only available literature is on the GRADE more likely to have the most expertise and
approach. Although quality of evidence ratings knowledge in a particular field) may lead to
had good reproducibility and reliability,14,15 it guidelines devoid of expertise and may impair
remains by necessity a matter of judgment.65 guideline credibility and uptake. Therefore, if
Aside from reliability, the benefit of using discrete excluding conflicted panelists was not
domains of the quality of evidence and specific possible in certain situations, reliable strategies
criteria for the evidence to decision framework for managing these conflicts would be critical
is the transparency of the approach. Transpar- to ensure guideline credibility. Recent strate-
ency allows guideline users to understand the gies that were shown to be effective were
rationale for the judgments made, even if they (1) tasking conflicted experts with interpreting
disagree with the perspective taken.65 and commenting on evidence but not with
Clinical practice guidelines become voting on recommendations; (2) appointing
quickly outdated, with 1 of 5 recommenda- a nonconflicted methodologist as the panel
tions being out of date after 3 years.66 Recom- chair or co-chair with equal authority as con-
mendations that depended on multiple tent experts; and (3) providing explicit criteria
randomized trials were less likely to change.67 for determining intellectual and financial
The concept of a living guideline (ie, a guideline conflicts.79
n n
430 Mayo Clin Proc. March 2017;92(3):423-433 http://dx.doi.org/10.1016/j.mayocp.2017.01.001
www.mayoclinicproceedings.org
CLINICAL PRACTICE GUIDELINES

CONCLUSION therapy clarifies evidence limitations. J Clin Epidemiol. 2014;

67(1):65-72.
Trustworthy clinical practice guidelines 14. Mustafa RA, Santesso N, Brozek J, et al. The GRADE approach
require a systematic review to select the is reproducible in assessing the quality of evidence of quantita-
best available evidence and should rigorously tive evidence syntheses. J Clin Epidemiol. 2013;66(7):736-742;
quiz 742e1-742e5.
evaluate the quality of evidence and incorpo- 15. Kumar A, Miladinovic B, Guyatt GH, Schunemann HJ,
rate nonevidence factors to transform evi- Djulbegovic B. GRADE guidelines system is reproducible
dence into a decision. Empirical evidence when instructions are clearly operationalized even among the
guidelines panel members with limited experience with
shows that guidelines improve patient GRADE. J Clin Epidemiol. 2016;75:115-118.
outcomes; however, adherence to guidelines 16. Faraday M, Hubbard H, Kosiak B, Dmochowski R. Staying at the
is variable. Therefore, guidelines require cutting edge: a review and analysis of evidence reporting and
grading; the recommendations of the American Urological As-
active dissemination and innovative imple- sociation. BJU Int. 2009;104(3):294-297.
mentation strategies. 17. Methods and processes US Preventive Services Taskforce web-
site. http://www.uspreventiveservicestaskforce.org/Page/Name/
methods-and-processes#recommendation-process. Accessed
Abbreviations and Acronyms: ACC/AHA = American August 22, 2016.
College of Cardiology/American Heart Association; AGREE = 18. Institute of Medicine. Clinical Practice Guidelines We Can Trust.
Appraisal of Guidelines, Research, and Evaluation; GRADE = Washington, DC: National Academies Press; 2011.
Grading of Recommendations, Assessment, Development, 19. Qaseem A, Forland F, Macbeth F, Ollenschlager G, Phillips S,
and Evaluation van der Wees P. Guidelines International Network: toward in-
ternational standards for clinical practice guidelines. Ann Intern
Correspondence: Address to M. Hassan Murad, MD, MPH, Med. 2012;156(7):525-531.
Division of Preventive Medicine, Mayo Clinic, 200 First St 20. Hill J, Bullock I, Alderson P. A summary of the methods that the
National Clinical Guideline Centre uses to produce clinical
SW, Rochester, MN 55905 (Murad.mohammad@mayo.edu).
guidelines for the National Institute for Health and Clinical
Excellence. Ann Intern Med. 2011;154(11):752-757.
21. World Health Organization. World Health Organization Hand-
REFERENCES book for Guideline Development. 2nd ed. Geneva, Switzerland:
1. Field M, Lohr K. Clinical Practice Guidelines: Directions for a New World Health Organization Press; 2014.
Program, Institute of Medicine. Washington, DC: National Acad- 22. Gandhi GY, Murad MH, Flynn DN, et al. Effect of perioperative
emy Press; 1990. insulin infusion on surgical morbidity and mortality: systematic
2. Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J. Clinical review and meta-analysis of randomized trials. Mayo Clin Proc.
guidelines: potential benefits, limitations, and harms of clinical 2008;83(4):418-430.
guidelines. BMJ. 1999;318(7182):527-530. 23. Jun M, Turin TC, Woodward M, et al. Assessing the validity of
3. International Guideline Library. http://www.g-i-n.net/library/ surrogate outcomes for ESRD: a meta-analysis. J Am Soc Neph-
international-guidelines-library. Accessed September 24, 2016. rol. 2015;26(9):2289-2302.
4. National Guideline Clearinghouse. https://www.guideline.gov. 24. Murad MH, Coto-Yglesias F, Varkey P, Prokop LJ, Murad AL.
Accessed September 21, 2016. The effectiveness of self-directed learning in health profes-
5. Weisz G, Cambrosio A, Keating P, Knaapen L, Schlich T, sions education: a systematic review. Med Educ. 2010;
Tournay VJ. The emergence of clinical practice guidelines. Mil- 44(11):1057-1068.
bank Q. 2007;85(4):691-727. 25. Murad MH, Flynn DN, Elamin MB, et al. Endarterectomy vs
6. Woolf SH. Practice guidelines, a new reality in medicine, II: stenting for carotid artery stenosis: a systematic review and
methods of developing guidelines. Arch Intern Med. 1992; meta-analysis. J Vasc Surg. 2008;48(2):487-493.
152(5):946-952. 26. Eyding D, Lelgemann M, Grouven U, et al. Reboxetine for
7. Guyatt G. Evidence-based medicine [editorial]. ACP J Club. acute treatment of major depression: systematic review and
1991;114:A16. meta-analysis of published and unpublished placebo and selective
8. Tricoci P, Allen JM, Kramer JM, Califf RM, Smith SC Jr. Scientific serotonin reuptake inhibitor controlled trials. BMJ. 2010;341:c4737.
evidence underlying the ACC/AHA clinical practice guidelines. 27. Gilbert R, Salanti G, Harden M, See S. Infant sleeping position
JAMA. 2009;301(8):831-841. and the sudden infant death syndrome: systematic review of
9. Atkins D, Eccles M, Flottorp S, et al; GRADE Working Group. observational studies and historical review of recommendations
Systems for grading the quality of evidence and the strength of from 1940 to 2002. Int J Epidemiol. 2005;34(4):874-887.
recommendations, I: critical appraisal of existing approaches. 28. Kumar A, Roberts D, Wood KE, et al. Duration of hypotension
BMC Health Serv Res. 2004;4(1):38. before initiation of effective antimicrobial therapy is the critical
10. Guyatt G, Jaeschke R, Wilson M, Montori V, Richardson W. determinant of survival in human septic shock. Crit Care Med.
What is evidence-based medicine? In: Guyatt G, Rennie D, 2006;34(6):1589-1596.
Meade M, Cook D, eds. Users’ Guide to the Medical Literature: 29. Devereaux PJ, Choi PT, Lacchetti C, et al. A systematic review
A Manual for Evidence-Based Clinical Practice. 3rd ed. New and meta-analysis of studies comparing mortality rates of pri-
York, NY: McGraw-Hill Education/Medical; 2014:7-14. vate for-profit and private not-for-profit hospitals. CMAJ.
11. Balshem H, Helfand M, Schunemann HJ, et al. GRADE guide- 2002;166(11):1399-1406.
lines, 3: rating the quality of evidence. J Clin Epidemiol. 2011; 30. Murad MH, Montori VM, Guyatt GH. Incorporating patient
64(4):401-406. preferences in evidence-based medicine. JAMA. 2008;300(21):
12. Alonso-Coello P, Schunemann HJ, Moberg J, et al. GRADE 2483; author reply 2483-2484.
Evidence to Decision (EtD) frameworks: a systematic and trans- 31. Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging
parent approach to making well informed healthcare choices, consensus on rating quality of evidence and strength of recom-
1: introduction. BMJ. 2016;353:i2016. mendations. BMJ. 2008;336(7650):924-926.
13. Murad MH, Altayar O, Bennett M, et al. Using GRADE for 32. MacLean S, Mulla S, Akl EA, et al. Patient values and
evaluating the quality of evidence in hyperbaric oxygen preferences in decision making for antithrombotic therapy: a

Mayo Clin Proc. n March 2017;92(3):423-433 n http://dx.doi.org/10.1016/j.mayocp.2017.01.001 431

www.mayoclinicproceedings.org
 MAYO CLINIC PROCEEDINGS

systematic review: Antithrombotic Therapy and Prevention of 52. Proietti M, Nobili A, Raparelli V, et al. Adherence to antithrom-
Thrombosis, 9th ed: American College of Chest Physicians botic therapy guidelines improves mortality among elderly pa-
Evidence-Based Clinical Practice Guidelines. Chest. 2012; tients with atrial fibrillation: insights from the REPOSI study.
141(2 suppl):e1S-e23S. Clin Res Cardiol. 2016;105(11):912-920.
33. Domecq JP, Prutsky G, Elraiyah T, et al. Patient engagement 53. Geahchan N, Basile M, Tohmeh M; DIONYS registry.
in research: a systematic review. BMC Health Serv Res. 2014; Venous thromboembolism prophylaxis in patients undergoing
14:89. abdominal and pelvic cancer surgery: adherence and compli-
34. Armstrong MJ, Rueda JD, Gronseth GS, Daniel Mullins C. Frame- ance to ACCP guidelines in DIONYS registry. Springerplus.
work for enhancing clinical practice guidelines through contin- 2016;5(1):1541.
uous patient engagement. Health Expect. 2017;20(1):3-10. 54. Tinkle MB, Tigges BB, Boursaw B, McFarlane DR. Adherence
35. Legare F, Boivin A, van der Weijden T, et al. Patient and public to the Women’s Preventive Services Guidelines in the
involvement in clinical practice guidelines: a knowledge synthesis Affordable Care Act. J Obstet Gynecol Neonatal Nurs. 2016;
of existing programs. Med Decis Making. 2011;31(6):E45-E74. 45(6):813-824.
36. Andrews J, Guyatt G, Oxman AD, et al. GRADE guidelines: 14. 55. Grimshaw JM, Shirran L, Thomas R, et al. Changing provider
Going from evidence to recommendations: the significance and behavior: an overview of systematic reviews of interventions.
presentation of recommendations. J Clin Epidemiol. 2013;66(7): Med Care. 2001;39(8 suppl 2):II2-II45.
719-725. 56. Flodgren G, Hall AM, Goulding L, et al. Tools developed and
37. Terrault NA, Bzowej NH, Chang KM, et al. AASLD guidelines disseminated by guideline producers to promote the uptake
for treatment of chronic hepatitis B. Hepatology. 2016;63(1): of their guidelines. Cochrane Database Syst Rev. 2016;8:
261-283. CD010669.
38. Brito JP, Domecq JP, Murad MH, Guyatt GH, Montori VM. The 57. Sinuff T, Cook D, Giacomini M, Heyland D, Dodek P. Facili-
Endocrine Society guidelines: when the confidence cart goes tating clinician adherence to guidelines in the intensive care
before the evidence horse. J Clin Endocrinol Metab. 2013; unit: a multicenter, qualitative study. Crit Care Med. 2007;
98(8):3246-3252. 35(9):2083-2089.
39. Vlayen J, Aertgeerts B, Hannes K, Sermeus W, Ramaekers D. 58. Gross PA, Aho L, Ashtyani H, et al. Extending the nurse prac-
A systematic review of appraisal tools for clinical practice guide- titioner concurrent intervention model to community-acquired
lines: multiple similarities and one common deficit. Int J Qual pneumonia and chronic obstructive pulmonary disease. Jt
Health Care. 2005;17(3):235-242. Comm J Qual Saf. 2004;30(7):377-386.
40. Shiffman RN, Dixon J, Brandt C, et al. The GuideLine Imple- 59. Alonso-Coello P, Irfan A, Sola I, et al. The quality of clinical
mentability Appraisal (GLIA): development of an instrument practice guidelines over the last two decades: a systematic re-
to identify obstacles to guideline implementation. BMC Med view of guideline appraisal studies. Qual Saf Health Care.
Inform Decis Mak. 2005;5:23. 2010;19(6):e58.
41. Brouwers MC, Kho ME, Browman GP, et al. AGREE II: 60. Bancos I, Cheng T, Prokop LJ, Montori VM, Murad MH. Endo-
advancing guideline development, reporting and evaluation in crine clinical practice guidelines in North America: a systematic
health care. CMAJ. 2010;182(18):E839-E842. assessment of quality. J Clin Epidemiol. 2012;65(5):520-525.
42. Neumann I, Santesso N, Akl EA, et al. A guide for health pro- 61. Gorman SK, Chung MH, Slavik RS, Zed PJ, Wilbur K,
fessionals to interpret and use recommendations in guidelines Dhingra VK. A critical appraisal of the quality of critical
developed with the GRADE approach. J Clin Epidemiol. 2016; care pharmacotherapy clinical practice guidelines and their
72:45-55. strength of recommendations. Intensive Care Med. 2010;
43. Kahn SR, Lim W, Dunn AS, et al. Prevention of VTE in nonsur- 36(10):1636-1643.
gical patients: Antithrombotic Therapy and Prevention of 62. Wyatt KD, Stuart LM, Brito JP, et al. Out of context: clin-
Thrombosis, 9th ed: American College of Chest Physicians ical practice guidelines and patients with multiple chronic
Evidence-Based Clinical Practice Guidelines. Chest. 2012;141(2 conditions: a systematic review. Med Care. 2014;52(suppl 3):
suppl):e195S-e226S. S92-S100.
44. Brozek JL, Bousquet J, Baena-Cagnani CE, et al. Allergic Rhinitis 63. Vohra S. N-of-1 trials to enhance patient outcomes: identifying
and its Impact on Asthma (ARIA) guidelines: 2010 revision. effective therapies and reducing harms, one patient at a time.
J Allergy Clin Immunol. 2010;126(3):466-476. J Clin Epidemiol. 2016;76:6-8.
45. Stacey D, Legare F, Col NF, et al. Decision aids for people fac- 64. Sun X, Briel M, Walter SD, Guyatt GH. Is a subgroup effect
ing health treatment or screening decisions. Cochrane Database believable? Updating criteria to evaluate the credibility of sub-
Syst Rev. 2014;1:CD001431. group analyses. BMJ. 2010;340:c117.
46. Hargraves I, LeBlanc A, Shah ND, Montori VM. Shared decision 65. Murad MH, Mustafa R, Morgan R, Sultan S, Falck-Ytter Y,
making: the need for patient-clinician conversation, not just in- Dahm P. Rating the quality of evidence is by necessity a matter
formation. Health Aff (Millwood). 2016;35(4):627-629. of judgment. J Clin Epidemiol. 2016;74:237-238.
47. Schunemann HJ, Fretheim A, Oxman AD. Improving the use of 66. Martinez Garcia L, Sanabria AJ, Garcia Alvarez E, et al. The val-
research evidence in guideline development: 13. Applicability, idity of recommendations from clinical guidelines: a survival
transferability and adaptation. Health Res Policy Syst. 2006;4:25. analysis. CMAJ. 2014;186(16):1211-1219.
48. Kristiansen A, Brandt L, Alonso-Coello P, et al. Development of 67. Neuman MD, Goldstein JN, Cirullo MA, Schwartz JS. Durability
a novel, multilayered presentation format for clinical practice of class I American College of Cardiology/American Heart
guidelines. Chest. 2015;147(3):754-763. Association clinical practice guideline recommendations.
49. Komajda M, Lapuerta P, Hermans N, et al. Adherence to guide- JAMA. 2014;311(20):2092-2100.
lines is a predictor of outcome in chronic heart failure: the 68. Vernooij RW, Sanabria AJ, Sola I, Alonso-Coello P, Martinez
MAHLER survey. Eur Heart J. 2005;26(16):1653-1659. Garcia L. Guidance for updating clinical practice guidelines: a
50. Wockel A, Kurzeder C, Geyer V, et al. Effects of guideline systematic review of methodological handbooks. Implement
adherence in primary breast cancerea 5-year multi-center Sci. 2014;9:3.
cohort study of 3976 patients. Breast. 2010;19(2):120-127. 69. Siemieniuk RA, Agoritsas T, Macdonald H, Guyatt GH,
51. Wilke M, Grube RF, Bodmann KF. Guideline-adherent initial Brandt L, Vandvik PO. Introduction to BMJ Rapid Recommen-
intravenous antibiotic therapy for hospital-acquired/ventilator- dations. BMJ. 2016;354:i5191.
associated pneumonia is clinically superior, saves lives and is 70. Paraskevas KI, Mikhailidis DP, Veith FJ. Comparison of the five
cheaper than non guideline adherent therapy. Eur J Med Res. 2011 guidelines for the treatment of carotid stenosis. J Vasc
2011;16(7):315-323. Surg. 2012;55(5):1504-1508.

n n
432 Mayo Clin Proc. March 2017;92(3):423-433 http://dx.doi.org/10.1016/j.mayocp.2017.01.001
www.mayoclinicproceedings.org
CLINICAL PRACTICE GUIDELINES

71. Nilsen ES, Myrhaug HT, Johansen M, Oliver S, Oxman AD. 76. Alexander PE, Li SA, Tonelli M, Guyatt G. Canadian primary
Methods of consumer involvement in developing healthcare care physicians’ attitudes toward understanding clinical practice
policy and research, clinical practice guidelines and patient infor- guidelines for diabetes screening. Can J Diabetes. 2016;40(6):
mation material. Cochrane Database Syst Rev. 2006;3: 580-585.
CD004563. 77. Lunyera J, Jonassaint C, Jonassaint J, Shah N. Attitudes of pri-
72. Shekelle P, Eccles MP, Grimshaw JM, Woolf SH. When should mary care physicians toward sickle cell disease care, guidelines,
clinical guidelines be updated? BMJ. 2001;323(7305):155-157. and comanaging hydroxyurea with a specialist. J Prim Care Com-
73. Alonso-Coello P, Martinez Garcia L, Carrasco JM, et al. The munity Health. 2017;8(1):37-40.
updating of clinical practice guidelines: insights from an interna- 78. Wang AT, McCoy CP, Murad MH, Montori VM. Association
tional survey. Implement Sci. 2011;6:107. between industry affiliation and position on cardiovascular
74. Timmermans S, Mauck A. The promises and pitfalls of evidence- risk with rosiglitazone: cross sectional systematic review. BMJ.
based medicine. Health Aff (Millwood). 2005;24(1):18-28. 2010;340:c1344.
75. Siriwardena AN. Clinical guidelines in primary care: a survey of 79. Guyatt G, Akl EA, Hirsh J, et al. The vexing problem of guide-
general practitioners’ attitudes and behaviour. Br J Gen Pract. lines and conflict of interest: a potential solution. Ann Intern
1995;45(401):643-647. Med. 2010;152(11):738-741.

Mayo Clin Proc. n March 2017;92(3):423-433 n http://dx.doi.org/10.1016/j.mayocp.2017.01.001 433

www.mayoclinicproceedings.org