Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                

Health Promotion Evaluation: Recommendations To Policy-Makers

Download as pdf or txt
Download as pdf or txt
You are on page 1of 12

HEALTH PROMOTION

EVALUATION:
RECOMMENDATIONS TO
POLICY-MAKERS
Report of the WHO European Working Group
on
Health Promotion Evaluation
REGIONAL OFFICE FOR EUROPE EUR/ICP/IVST 05 01 03
SCHERFIGSVEJ 8 ORIGINAL: ENGLISH
DK-2100 COPENHAGEN UNEDITED
DENMARK EUR/HFA target 15
TEL.: (45) 39 17 17 17 1998
TELEFAX: (45) 39 17 18 18 E60706
TELEX: 15348 and 12000
WEB SITE: http://www.who.dk
CENTERS FOR DISEASE CONTROL
AND PREVENTION
TARGET 15
HEALTH COMPETENCE
By the year 2000, accessible and effective education and training in health promotion should be available
in all Member States, in order to improve public and professional competence in promoting health and
increasing health awareness in other sectors.
ABSTRACT
In June 1995, the WHO Regional Office for Europe established a Working Group on
Health Promotion Evaluation in cooperation with three government agencies:
the Centers for Disease Control and Prevention, United States
Health Canada
Health Education Authority, United Kingdom.
The Working Group had three objectives:
to provide guidance to policy-makers and practitioners to foster the use of appro-
priate methods for health promotion evaluation;
to examine the current range of evaluation methods, both quantitative and qualita-
tive; and
to provide guidance to policy-makers and practitioners to increase the quality of
health promotion evaluations.
To achieve its objectives, the Working Group:
commissioned and reviewed more than 30 background papers dealing with key is-
sues related to the evaluation of health promotion initiatives;
produced resource documents to guide the planning and implementation of health
promotion evaluations by evaluators, policy-makers and practitioners; and
sought and incorporated the views of health promotion policy-makers, practitioners
and evaluators in developing its reports and publications.
The present report, which is a summary of the work carried out by the Working Group
from June 1995 to March 1998, provides guidance to policy-makers and other deci-
sion-makers who can influence resources for planning, implementing and evaluating
health promotion initiatives. The conclusions and recommendations are summarized
in Table 1.
Keywords
HEALTH PROMOTION
PROGRAM EVALUATION methods
POLICY PLANNING
World Health Organization
All rights in this document are reserved by the WHO Regional Office for Europe. The document may nevertheless be freely re-
viewed, abstracted, reproduced or translated into any other language (but not for sale or for use in conjunction with commercial
purposes) provided that full acknowledgement is given to the source. For the use of the WHO emblem, permission must be sought
from the WHO Regional Office. Any translation should include the words: The translator of this document is responsible for the ac-
curacy of the translation. The Regional Office would appreciate receiving three copies of any translation. Any views expressed by
named authors are solely the responsibility of those authors.
This document was text processed in Health Documentation Services
WHO Regional Office for Europe, Copenhagen
CONTENTS
Page
Foreword....................................................................................................................................................... 1
Introduction .................................................................................................................................................. 2
The meaning of health promotion and evaluation........................................................................................ 3
Principles for the evaluation of health promotion initiatives........................................................................ 3
Conclusions and recommendations for the evaluation of health promotion initiatives................................ 3
Annex 1 Members of the Working Group................................................................................................... 7
Annex 2 Principles of health promotion...................................................................................................... 8
EUR/ICP/IVST 05 01 03
page 1
Foreword
As we approach the twenty-first century, throughout the public sector, those who plan and
deliver services and policies face increasing pressure to demonstrate that what they are doing is
worthwhile, effective and efficient. The field of health promotion is also under this kind of
pressure.
Is health promotion a good investment? How can the short- and long-term returns of such an in-
vestment be assessed? To what degree can social and economic benefits stemming from health
promotion initiatives be measured alongside health gains? Answering such questions is not a
simple task. Health promotion policies and programmes, if properly planned and implemented,
involve complex and sophisticated activities. Very often, health promotion action requires multi-
ple approaches, relies on interdisciplinary inputs and operates at several levels over long periods
of time. Despite this complexity, health promotion programmes are often forced to be evaluated
with methods and approaches that, although quite acceptable within medical care and prevention,
are totally unsuitable for this field.
Fortunately, over the last two decades knowledge and understanding about how best to evaluate
complex programmes and policies have significantly increased. Decision-makers and practitio-
ners, however, are not fully aware of these developments or their implications for the evaluation
of health promotion interventions.
For these reasons, in 1995 the WHO Regional Office for Europe established a WHO European
Working Group on Health Promotion Evaluation. The Regional Office conducted this much
needed exercise in cooperation with Health Canada and the Centers for Disease Control and
Prevention in the United States. In Europe, the Health Education Authority (England, United
Kingdom) gave strong support, both financial and in kind. The chairperson of the Working
Group was Dr Irving Rootman, Director of the Centre for Health Promotion in Toronto, Ontario,
Canada (a WHO collaborating centre). Throughout its work, the Working Group was in all
senses a model of effective international collaboration, thanks to the goodwill of the participating
governments, agencies and, most of all, individuals. The main purpose of its work was to
increase the quality, appropriateness and use of evaluations of health promotion programmes,
policies and other organized activities in developed countries. Another key purpose was to
increase the policy-makers understanding of their role in evaluation of health promotion
initiatives. This document presents recommendations to policy-makers on how they might more
effectively fulfil their role in this important area.
The WHO Regional Office for Europe thanks both the Working Group and everyone who
contributed to finalizing its work; the Regional Office also expresses appreciation to the
agencies, both within and outside Europe, that provided resources for this important international
collaborative effort.
J.E. Asvall
Regional Director WHO Regional Office
EUR/ICP/IVST 05 01 03
page 2
Table 1. Conclusions and recommendations
Conclusions Recommendations to policy-makers
Those who have a direct interest in a health promo-
tion initiative should have the opportunity to partici-
pate in all stages of its planning and evaluation.
Encourage the adoption of participatory approaches to
evaluation that provide meaningful opportunities for involve-
ment by all of those with a direct interest in health promotion
initiatives
Adequate resources should be devoted to the
evaluation of health promotion initiatives.
Require that a minimum of 10% of the total financial resources
for a health promotion initiative be allocated to evaluation
Health promotion initiatives should be evaluated in
terms of their processes as well as their outcomes.
Ensure that a mixture of process and outcome information is
used to evaluate all health promotion initiatives
The use of randomized control trials to evaluate
health promotion initiatives is, in most cases, inap-
propriate, misleading and unnecessarily expensive.
Support the use of multiple methods to evaluate health pro-
motion initiatives
Support further research into the development of appropriate
approaches to evaluating health promotion initiatives
Expertise in the evaluation of health promotion initia-
tives needs to be developed and sustained.
Support the establishment of a training and education infra-
structure to develop expertise in the evaluation of health pro-
motion initiatives
Create and support opportunities for sharing information on
evaluation methods used in health promotion through confer-
ences, workshops, networks and other means
Introduction
Background
Recognizing the importance of evaluation in
health promotion and need for appropriate
methods, in June 1995, the WHO Regional Of-
fice for Europe established a Working Group on
Health Promotion Evaluation (Annex 1) in co-
operation with three government agencies:
the Centers for Disease Control and Preven-
tion, United States
Health Canada
Health Education Authority, United Kingdom.
The Working Group had three objectives:
to examine the current range of evaluation
methods, both quantitative and qualitative;
to provide guidance to policy-makers and
practitioners to foster the use of appropriate
methods for health promotion evaluation; and
to provide guidance to policy-makers and
practitioners to increase the quality of health
promotion evaluations.
To achieve its objectives, the Working
Group:
commissioned and reviewed more than 30
background papers dealing with key issues
related to the evaluation of health promotion
initiatives;
produced resource documents to guide the
planning and implementation of health pro-
motion evaluations by evaluators, policy-
makers and practitioners; and
sought and incorporated the views of health
promotion policy-makers, practitioners and
evaluators in developing its reports and pub-
lications.
The present report provides guidance to policy-
makers and other decision-makers who can in-
fluence resources for planning, implementing
and evaluating health promotion initiatives
(programmes, policies and other organized ac-
tivities).
EUR/ICP/IVST 05 01 03
page 3
The meaning of health promotion and evaluation
Health promotion, as defined by the Ottawa
Charter for Health Promotion (1986), refers to
the process of enabling people to increase con-
trol over, and to improve, their health. The im-
plementation of this definition requires that
health promotion initiatives should be empow-
ering, participatory, holistic, intersectoral, equi-
table, sustainable, and multi-strategy. Defini-
tions of these key principles for health promo-
tion are provided in Appendix 2.
Evaluation, as defined by the Working Group, is
the systematic examination and assessment of
the features of an initiative and its effects, in
order to produce information that can be used by
those who have an interest in its improvement or
effectiveness.
Principles for the evaluation of health promotion initiatives
Based on the principles of health promotion, the
Working Group concluded that the following are
the core features of approaches appropriate for
the evaluation of health promotion initiatives:
Participation
At each stage of evaluation, health promotion
initiatives should involve, in appropriate ways,
those who have a legitimate interest in the initia-
tive. Those with an interest can include: policy-
makers, community members and organizations,
health and other professionals, and local and na-
tional health agencies. It is especially important
that members of the community whose health is
being addressed be involved in evaluation.
Multiple methods
Evaluations of health promotion initiatives
should draw on a variety of disciplines, and
should consider employing a broad range of in-
formation gathering procedures.
Capacity building
Evaluations of health promotion initiatives
should enhance the capacity of individuals,
communities, organizations and governments to
address important health promotion concerns.
Appropriateness
Evaluations of health promotion initiatives
should be designed to accommodate the com-
plex nature of health promotion interventions
and their long-term impact.
The Working Group believes that evaluations
premised on these principles provide an appro-
priate means of assessing and understanding
health promotion initiatives. To support the
adoption of appropriate evaluation methods in
the field of health promotion, policy-makers
should consider the conclusions and recommen-
dations outlined in the next section of this re-
port.
Conclusions and recommendations for the evaluation of
health promotion initiatives
Conclusion 1. Those who have a direct
interest in a health promotion initiative
should have the opportunity to partici-
pate in all stages of its planning and
evaluation.
Participation and empowerment are key princi-
ples of health promotion (see the Ottawa Charter
for Health Promotion). Participation must there-
fore be an integral feature of any evaluation pro-
cess.
EUR/ICP/IVST 05 01 03
page 4
Participation by those with a direct interest in a
health promotion initiative is an important pre-
requisite for the evaluation of health promotion
programmes and policies. Such participation in-
creases the relevance and credibility of evaluation
results, as well as the likelihood that the results
will be used. In addition, participatory approaches
to evaluation help foster the process of
empowerment and build stake-holders capacity
to address health needs, thus giving them more
control over the factors affecting their health.
Ideally, participation should extend beyond
those who are the primary focus of health pro-
motion initiatives, to include others who have a
direct stake in the initiatives. These additional
stakeholders include: health promotion practi-
tioners, community representatives, policy-
makers, and evaluators.
Substantial evidence indicates that the results of
programme evaluations are more likely to be
implemented when key stakeholders have par-
ticipated in all stages of the evaluation process.
The participation of stakeholder groups makes
the values underlying the evaluation explicit,
and allows the issues of different groups to be
addressed. This, in turn, helps to increase the
credibility and subsequent use of evaluation re-
sults.
The evaluation of health promotion initiatives
benefits from pooling professional and lay re-
sources, including the unique knowledge pos-
sessed by non-professionals. In addition, a par-
ticipatory approach to the evaluation of health
promotion grounds evaluation indicators in
practical reality, and ensures that information
gained through evaluation benefits all partici-
pants.
Most importantly, participatory evaluation en-
courages collaboration between different sec-
tors, forcing conscious choices and a multisecto-
ral approach to selecting indicators in complex
health promotion projects. Finally, participatory
evaluation provides a practical way to cross
boundaries between theory and practice.
Recommendation 1. Policy-makers should encourage the adoption of participatory ap-
proaches to evaluation that provide meaningful opportunities for involvement by all of
those with a direct interest in health promotion initiatives.
Conclusion 2. Adequate resources
should be devoted to the evaluation of
health promotion initiatives.
To maximize the benefits of an evaluation,
budgets for health promotion initiatives must
include sufficient funding for a thorough ex-
amination of their main features. The impor-
tance of adequate funding has been recognized
by a number of jurisdictions, where minimum
standards for the allocation of financial re-
sources to the evaluation of health promotion
programmes have been established. These have
ranged from 8% to 15% of the total programme
budget.
Analysis of previous experience supports the
Working Groups view that the allocation of
10% of total programme resources is a reason-
able standard to ensure the development and
implementation of appropriate evaluations in
health promotion. This does not, however, pre-
clude the allocation of additional resources when
necessary.
Recommendation 2. Policy-makers should require that a minimum of ten percent of the
total financial resources for a health promotion initiative be allocated to evaluation.
Conclusion 3. Health promotion initia-
tives should be evaluated in terms of
their processes as well as their out-
comes.
The evaluation of health promotion initiatives
requires evaluation methods to assess both the
process and contextual aspects of the activities,
in addition to evaluation of outcomes.
EUR/ICP/IVST 05 01 03
page 5
Outcome-focused evaluations predominate in
the current public health system. In many in-
stances, outcome measures provide an important
contribution to understanding the impact of
health promotion initiatives. Outcome measures
are not, however, a sufficient means of under-
standing the ways in which a health promotion
programme or policy has brought about change.
Although outcome measures can reveal if a pro-
gramme works (or does not work), they are nei-
ther intended nor designed to reveal why or how
a programme works. Understanding why or how
a health promotion initiative fosters change is as
important as knowing whether a desired change
took place, particularly when broader imple-
mentation of an initiative is planned or the con-
text of an initiative changes significantly.
Process evaluation, combined with indicators of
short- and long-term outcomes, provide the range
of information needed fully to assess and under-
stand the impact of health promotion initiatives,
and make appropriate programme decisions.
Recommendation 3. Policy-makers should ensure that a mixture of process and outcome
information is used to evaluate all health promotion initiatives
.
Conclusion 4. The use of randomized
control trials to evaluate health promo-
tion initiatives is, in most cases, inap-
propriate, misleading and unnecessarily
expensive.
A multidimensional focus on the determinants of
health and the impossibility of imposing tight
environmental controls, or their unacceptability,
are inherent features of most health promotion
initiatives. The randomized controlled trial is
often an inappropriate and potentially mislead-
ing means of evaluating these efforts. For a bet-
ter understanding of the impact of health pro-
motion initiatives, evaluators need to use a wide
range of qualitative and quantitative methods
that extend beyond the narrow parameters of
randomized controlled trials.
Randomized controlled trials are most effective
when the intervention can be delivered and re-
ceived in a standard way: that is, when varia-
tions in delivery and acceptance are minimized.
Health promotion programmes can vary greatly
in both of these dimensions. Variability may
occur, first, in the delivery of an information
campaign, implementation of a school pro-
gramme or enforcement of a policy, and, second,
in audience attention to or acceptance of cam-
paign messages or participation in a programme.
In addition, because health promotion is often a
long-term process, frequently involving envi-
ronmental modifications, attempts to keep envi-
ronmental conditions constant can undermine
the processes that health promotion attempts to
influence.
Recommendation 4. Policy-makers should support the use of multiple methods to evalu-
ate health promotion initiatives.
Recommendation 5. Policy-makers should support further research into the development
of appropriate approaches to evaluating health promotion initiatives
Conclusion 5. Expertise in the evaluation
of health promotion initiatives needs to
be developed and sustained.
As with other scientific activities, the evaluation
of health promotion initiatives requires specific
skills and capacities. Given the diverse nature of
health promotion programmes and policies,
these skills extend beyond the domain of health
sciences to include social science methods, or-
ganizational change theory, participatory action
research and other approaches to knowledge de-
velopment that are compatible with core health
promotion principles and values.
EUR/ICP/IVST 05 01 03
page 6
At present, the field of health promotion lacks
both an adequate infrastructure to develop ex-
pertise in evaluation and evaluators with the
knowledge and skills to make appropriate as-
sessments. This deficit, in many instances, has
resulted in the adoption of inappropriate criteria
for evaluating health promotion initiatives.
To ensure that evaluators possess the requisite
skills for examining health promotion initiatives,
the capacity for monitoring and evaluation must
be supported and strengthened at every level of
policy-making. This requires the establishment
of a strategy for the development and mainte-
nance of an adequate infrastructure for the de-
velopment of skills in the evaluation of health
promotion initiatives, as well as the dissemina-
tion of information about appropriate evaluation
methods.
Recommendation 6. Policy-makers should support the establishment of a training and
education infrastructure to develop expertise in the evaluation of health promotion initia-
tives.
Recommendation 7. Policy-makers should create and support opportunities for sharing
information on evaluation methods used in health promotion through conferences,
workshops, networks and other means.
EUR/ICP/IVST 05 01 03
page 7
Annex 1
Members of the Working Group
Dr Laurie Anderson
Health Scientist, Epidemiological Program
Office, Urban Health Center, Centers for
Disease Control and Prevention, Atlanta,
USA
Dr Lynda Anderson
Health Scientist, Division of Adult and
Community Health, National Center for
Chronic Disease Prevention and Health
Promotion, Centers for Disease Control and
Prevention, Atlanta, USA
Mr Nick Doyle
Head of Policy, Health Education Authority,
London, United Kingdom
Dr Brian Flay
Director, Prevention Research Centre,
University of Illinois, Chicago, USA
Dr Michael Goodstadt
Deputy Director, Centre for Health Promo-
tion, University of Toronto, Canada
Dr Igor Glasunov
Executive Director, CINDI Russia, Institute
of Preventive Medicine, Moscow, Russian
Federation
Mr Brian Hyndman
Consultant, Centre for Health Promotion,
University of Toronto, Canada (Secretary)
Dr Glenn Irwin
Senior Policy Analyst, Health Promotion
and Programs Branch, Health Canada,
Ottawa, Canada
Mr Paul Lincoln
Director, Health Education Authority,
London, United Kingdom
Dr Peter Makara
Director, National Institute for Health Pro-
motion, Budapest, Hungary
Dr David McQueen
Assistant Director for Global Health Promo-
tion, National Center for Chronic Disease
Prevention and Health Promotion, Centers
for Disease Control and Prevention, Atlanta,
USA
Dr Tapani Piha
Principal Medical Officer, Ministry of So-
cial Affairs and Health, Helsinki, Finland
Dr Louise Potvin
Professor, Groupe de recherche interdisci-
plinaire en sant, Universit de Montral,
Montreal, Canada
Dr Irving Rootman
Director, Centre for Health Promotion,
University of Toronto, Canada (Chairperson)
Dr Tom Schmid
Evaluator, Division of Nutrition and Physi-
cal Activity, Centers for Disease Control
and Prevention, Atlanta, USA
Dr Sylvie Stachenko
Director, Department of Health Promotion
and Disease Prevention, WHO Regional
Office for Europe, Copenhagen, Denmark
Dr Jane Springett
Professor, Health Promotion and Public
Health, Institute for Health, John Moores
University, Liverpool, United Kingdom
Dr Erio Ziglio
Regional Adviser for Health Promotion and
Investment, WHO Regional Office for
Europe, Copenhagen, Denmark
EUR/ICP/IVST 05 01 03
page 8
Annex 2
Principles of health promotion
Health promotion initiatives are programmes,
policies and other organized activities planned
and implemented in accordance with the fol-
lowing principles.
Empowering
Health promotion initiatives should enable indi-
viduals and communities to assume more power
over the personal, socioeconomic and environ-
mental factors that affect their health.
Participatory
Health promotion initiatives should involve
those concerned in all stages of planning, im-
plementation and evaluation.
Holistic
Health promotion initiatives should foster physi-
cal, mental, social and spiritual health.
Intersectoral
Health promotion initiatives should involve the
collaboration of agencies from relevant sectors.
Equitable
Health promotion initiatives should be guided by
a concern for equity and social justice.
Sustainable
Health promotion initiatives should bring about
changes that individuals and communities can
maintain once initial funding has ended.
Multi-strategy
Health promotion initiatives should use a variety
of approaches, including policy development,
organizational change, community development,
legislation, advocacy, education and communi-
cation, in combination with one another.

You might also like