Introduction to Economic

Evaluation
Denny John
Evidence Synthesis Specialist, Campbell Collaboration

Acknowledgement: Dr. Luke Vale, Professor- Health Economics, Newcastle University, UK

Designing essential package of services-
Context of UHC
Example: World Development Report 1983
An example of this is the World Bank ‘essential service
package’ idea, contained in the 1993
World Development Report. The criteria which were used to
select services here included:
◆ High burden of disease caused by the condition in
developing countries
◆ Cost-effectiveness of intervention
◆ Positive externalities associated with the treatment or
prevention (see above: knock-on effects of reducing
transmission etc.)
What is health economics?
 Health economics
 A branch of economics that evaluates health care
 “..provides a logical and explicit framework that enables health
care workers, governments, decision makers and society to
make choices as to how best to use health care resources”
What is health economics?
 Health economics
 Evaluates costs and consequences of health care interventions,
such as drugs, devices, procedures, services and programs
 It will cost me £X to use intervention A in population P
 As a consequence of using A, I will gain/lose Y (clinical outcome)
 As a consequence of using A, I will save/lose £Z (economic outcome)
What is health economics?
 Health economics: marginal analysis
 Usually health economics evaluates costs and consequences of
a (small) change in health care interventions or the incremental
costs and benefits between different options
 Marginal cost: is the incremental cost of one extra unit of a
healthcare intervention
Why do we need health economics?
 Because of...
 a scarcity of resources be it money, nurses, equipment, hospital
beds etc...
 a need to make difficult choices
 Is intervention A ‘worth it’ compared to other things we could do
with the same resources
 a need for a logical (objective) and explicit framework for
making decisions and setting (justifying) health care budgets
But why do we need health economics?
 Criticisms of health economics
 Health economics overrides clinical freedom: clinicians prefer
to make decisions based on their own clinical experience
 It is difficult/impossible
 to make an objective evaluation of health interventions
 or fair comparisons of disparate health interventions
 Emotive subject area: patients (and industry?) find negative
recommendations hard to accept
 Seemly logical economic evaluations can lead to illogical clinical
decisions,
 e.g. ARMD it’s more cost-effective to wait until patient has gone blind
in one eye before starting treatment
But why do we need health economics?
 and some common misconceptions
 Health economics is about cost-cutting
 Health economics is about rationing
But why do we need health economics?
 For the pharmaceutical industry, health technology
appraisals (HTAs; a formal economic evaluation) are now
an unavoidable part of a new product launch
 In Scotland all new medicines must be evaluated by the
Scottish Medicines Consortium shortly after receiving a UK
license
 In England and Wales, the National Institute for Health and
Clinical Excellence is likely to request manufacturers to submit
an economic evaluation of their product
 Without a positive recommendation from SMC or NICE it is
difficult for manufacturers to get their product on a UK
formulary
But why do we need health economics?
 Canada, Australia, The Netherlands, and Sweden all have
some form of HTAs
 Many other countries are influenced by HTAs conducted
by for example, NICE
 Examples of HTA in LMICs: HITAP (Thailand), HTAIn
(India)
Economic Evaluation: What it is
 A way of thinking and formulating problems of choice
 Systematic framework for identifying and organising the
information required for decision-making
 Set of techniques for the analysis of alternatives
 Aid for decision-making
Uses of EE
 Guiding clinical practice
 Guiding management and purchasing decisions
 Developing clinical and public health guidelines
 Developing public policy
 Developing insurance benefit packages
 Selecting pharmaceuticals for subisidies
 Setting priorities
Limitations of EE
 Requires a single unit of outcome
 Requires data, expertise, resources, commitment
 Rarely incorporates distributional considerations
 Not as objective as it first appears
 Usually compares interventions within (not across) broad
service categories
EE is not useful when
 Options are significantly different
 There is evidence of effectiveness
 There are large resource implications
EE is least useful when
 Over-riding political/cultural/environmental values
 Evidence of effectiveness is weak
 Democratic processes undermined
 Costs of evaluating greater than the benefits
Analyses are comparative
EE requires
 Some evidence of effectiveness

 Choice between two or more alternatives

 Both costs and effects (benefits) are examined

Applications of EE
 Methods of treatment
 Prevention strategies
 Screening programmes
 Diagnostic techniques
 Methods of health service organisation
 Locations of care
 Frequency of intervention

 Examples of each?
What do we mean by ‘costs’?
 Cost of resources:
• Hospital treatments
• Plasters
• Taxi trips
• Time spent giving care
• Etc.
 Expenditure= Resource inputs x Prices
 Price
• Market price = Cost of item (including distribution) +
profits
What should be considered when setting
health care priorities
 Effectiveness
Clinical improvements such as extending life and/or improving aspects of
quality of life

 Efficiency
Maximising benefits in the face of scarce resources
Ensure that the benefits of those activities which are pursued are greater than
their opportunity costs (benefits foregone)

 Equity
Concerned with the fairness of how health care resources are distributed
 The equity-efficiency trade-off

Five treatments for 5 different diseases which can save lives, up to a total of 100
each. Total budget is INR3 million.

Treatment Cost per patient (INR)

A 10,000
B 20,000
C 30,000
D 40,000
E 50,000
 The equity-efficiency trade-off

Five treatments which can save lives, up to a total of 100 each. Total budget is
INR3 million.

Treatment Cost per patient (INR) Number of patients

A 10,000 100
B 20,000 100
C 30,000
D 40,000
E 50,000
The equity-efficiency trad off

Five treatments which can save lives, up to a total of 100 each. Total budget is
INR3 million.

Treatment Cost per patient (INR) Number of patients

A 10,000 20
B 20,000 20
C 30,000 20
D 40,000 20
E 50,000 20
Incorporating the personal equity weights

 Economic evaluation focuses on creating the maximum effect

from the resources available

 It tends not to consider the distribution of those effects

 Should it?
The role of economics in priority settings

 An assessment of efficiency is integral when allocating scarce health care

resources amongst competing alternatives

 Assessing efficiency is the domain of economic evaluation

 Defined as comparative analysis of alternative courses of action in

terms of both costs (use of health services) consequences (health effects)

 An economic evaluation can take many different forms, but the tasks
involved remain very similar: to identify, measure and value all of the
relevant costs and consequences of the programme or intervention being
analysed
Economic quadrants
Measures of ’benefit’ in EE
 Health outcomes
• Mortality
• Deaths avoided
• Life years gained
• Clinical measures
• Cases avoided
• Disease specific scales
• Time to full recovery
• Probability of recurrence
 Intermediate indicators (how predictive is the indicator?)
• Risk factors (serum cholesterol level, BMI, blood pressure, etc.)
• Number of cases detected
• Immunization rate
• Side effects
 Personal measures
• Satisfaction, comfort, etc.
What do we mean by ‘Benefits’?
 Cost benefit
• Outcomes measured in monetary terms
• Willingness to pay

 Cost utility
• Preference based health measures of ‘utility’
• EQ-5D, Health Utilities Index
• QALY= utility value x time
Methods of economic evaluation

Denny John
Evidence Synthesis Specialist, Campbell Collaboration
 Healthcare Evaluation
Are both costs and outcomes of alternatives
assessed?
No Yes
Is the comparison of two or

No Examines only Examines only

outcomes costs
more alternatives?

Partial evaluation Partial economic Partial economic

Outcome description evaluation evaluation
Cost of illness Cost outcome description
Yes Partial evaluation Partial economic Full economic evaluation
Efficacy (Outcomes) evaluation Cost-consequences
analysis Cost analysis analysis
Cost minimization analysis
Cost effectiveness analysis
Cost utility analysis
Cost benefit analysis

Adapted from Drummond et al. Methods of Economic Evalua2on in Healthcare (2006)

Cost-of-illness (COI)
 COI identifies the economic burden of a disease or
medical condition
 In general these studies evaluate the resources consumed
as a direct result of an illness or condition
 COIs always take a specific cost perspective, e.g. UK NHS
 Also called cost-of-disease or burden-of-illness studies
Types of COI studies
Incidence-based studies
 Incidence-based studies, which estimate lifetime costs, measure the costs
of an illness from onset to conclusion for cases beginning within the
period of the study, usually a year.
 Incidence costs include the discounted, lifetime medical, morbidity, and
mortality costs for the incident cohort. set.
Prevalence-based studies
 Prevalence-based studies, which estimate annual costs, measure the costs
of an illness in one period, usually a year, regardless of the date of onset.
 Includes all medical care costs and morbidity costs for a disease within
the study year. However, the mortality and permanent disability costs of
prevalence-based studies are calculated differently from the other costs.
Discounted mortality and permanent disability costs are calculated for all
patients who die or become permanently disabled in the study year for
that year and each year until the expected age of death.
Types of full economic evaluations
Method of analysis Cost Outcome
measurement measurement
Cost-consequence $ Multi-dimensional listing
analysis of outcomes
Cost-minimization $ Equivalence demonstrated
analysis or assumed in
comparative groups
Cost-effectiveness $ Single ‘natural’ unit
analysis outcome measure
Cost-utility analysis $ Multiple outcomes-life-
years adjusted for quality-
of-life
Cost-benefit analysis $ $
Cost-Consequences Analysis (CCA)
 Systematic description and measurement of a set of
intervention attributes that should be considered when
making a decision
 Need not describe a decision rule
 Like everyday attribute-specific decision making provides
information in a simple disaggregated format for decision
maker to make their own choice
 Weighting of different attributes left to individual decision
makers
CCA- Example
Cost-minimisation analysis (CMA)
 A cost comparison of two or more interventions with
comparable clinical and quality-of-life outcomes
 It is unlikely that the outcomes of two different
interventions are equal so CMAs are rarely performed
 Useful for evaluating generically equivalent drugs where
the outcomes have been demonstrated to be equivalent
 Useful for evaluating same intervention but given in
different settings
Cost-effectiveness analysis (CEA)
 This analysis compares the monetary cost of an
intervention with a measure of effectiveness (clinical or
quality of life outcome)
 The outcomes are measured in natural units and usually
expressed as the incremental gain/loss resulting from an
intervention:
 bed days avoided
 deaths avoided
 life years gained
Cost-utility analysis (CUA)
 An extension of cost-effectiveness analysis where
effectiveness is expressed in utilities, such as QALYs
 This allows comparisons across different indications
 CUA very common form of economic analysis, e.g.
Required for NICE/ SMC
 One criticism of CUA is how useful it is for acute (short-
term conditions) or conditions where utility/quality-of-life
difficult to measure
Cost-benefit analysis (CBA)
 Cost-benefit analysis (CBA) compares the net costs of an
intervention with the net benefits where both costs and
benefits are expressed in monetary units
 However, it is difficult to convert clinical and QOL
outcomes to a monetary value
 ‘Proper’ CBA are hardly used (in the UK)
Types of costs
Cost type Example of resources
Direct medical costs Hospitalization (LOS)
Outpatient visits
Procedures & tests (X-ray, Blood tests, surgery)
Devices (Wheelchairs, Pacemakers etc)
Services (Home-care, Nursing care) hours or days
Direct non-medical costs Transportation
Services (Home help etc
Devices & other investments
Informal care (care by relatives)
Indirect costs Sick leave (days or weeks)
Reduced productivity (percentage or hours)
Early retirement (years to normal retirement)
Premature death (years to normal retirement)
Costs included in EE studies
Steps in cost assessment
 Resource use: counts/number of units of each type of
medical resources needed
4 steps:
• Identify the relevant resources used
• Quantify these resources in physical units, such as hospital
days, admissions, surgical procedures, physician visits etc
• Value the different resources used in terms of their
opportunity costs
• Adjust valuations to account for the differential timing at
which resource use can occur (discounting)
Costing approaches
 Bottom up costing:
 Quantifying the resource use and multiply by unit cost of that
resource
 Drugs and disposables
 Inpatient stay, outpatient visits, A&E attendance
 Consultations: GP, specialist, surgeon, nurse
 Tests and procedures
 Top down costing
 Estimate unit cost (or cost per patient) by dividing total costs
by number of patients/units
 Use of statistical databases and registries to estimate costs at
regional or national level. E.g. NSSO data
Perspectives
 When conducting an economic evaluation need to state
the perspective of the evaluation
 Cost perspective – who pays?

Patient Provider Third-party Society

payer
Discounting
 Often health care involves investing resources now but
the consequences will not be apparent for many years
 Discounting is used to adjust future costs and benefits to
their ‘present value’
 Discounting is based on the assumption that
 we prefer to receive benefits now rather than in the future
 we prefer to pay later rather than pay now
 The strength of this ‘time preference’ is expressed by a
discount rate
Discounting
 In UK it is recommended that costs and health benefits
are both discounted at 3.5% based on the following
formulae:
 If year 0 is the present then the present value (at the middle of
year 0) of £1 (made at the middle of year n) is given by
Dn = 1/(1 + r)n
where r is the discount rate and Dn is the discount factor.

Source: UK Treasury (Green Book 2003)

ICERs
 As mentioned previously, we tend to compare different
treatments options
 The incremental cost-effectiveness ratio (ICER) is the
incremental cost divided by the incremental clinical
benefits of one intervention compared to another
 e.g. If clinical outcomes expressed in QALYs then

ICER = Cost of A – Cost of B

QALYs with A – QALYs with B
The cost-effectiveness plane
NW £10,000 NE

£8,000

Reject
£6,000

£4,000

£2,000
WTP ?
Incremental Cost

£-
-0.3 -0.2 -0.1 0 0.1 0.2 0.3
-£2,000

-£4,000
Accept
-£6,000

WTA ? -£8,000

SW -£10,000 SE
Incremental Effect

Threshold £20k Threshold £30k

ICER thresholds versus net benefit
 ICERs are usually compared with a threshold value, e.g. An
acceptable threshold of £20,000 or £30,000 per QALY are common
in the UK

 However, NW £10,000 NE

there is difficulty in interpreting

£8,000

negative ICERs £6,000

there is difficulty in interpreting

£4,000

ICERs that fall in the SW £2,000

Incremental Cost
quadrant -0.3 -0.2 -0.1
£-
0 0.1 0.2 0.3
-£2,000

-£4,000

-£6,000

-£8,000

SW -£10,000 SE
Incremental Effect

Threshold £20k Threshold £30k

 Introduction to modelling in
Economic Evaluation
Denny John
Evidence Synthesis Speciialist, Campbell Collaboration
Overview of types of economic models
 Most economic evaluations are based on economic
models
 Common types of models used in health economics are
 Decision analysis models
 Budget impact models
 Care maps/ care pathways
Inputs and outputs to a model
Resource Clinical Health
use Effects state
valuation

Unit
costs Epidemiological
data
MODEL

Estimate
of ICER
Decision analysis
 Decision analysis is quantitative method to aid decision-making
especially where you have less than perfect data
 A set of calculations laid out in a logical sequence
 Compares at least two alternative approaches; e.g. two
treatment strategies, two screening programmes
 Decision analysis provides the expected value; by weighting
events by the probability that they will occur (i.e. the weighted
average)
 Informs a decision process; not intended merely to arrive at
‘perfect’ scientific answers
 Should be used as a decision aid
 Goal should be to provide decision-makers with information
that can allow them to judge
When to Use Decision-Analytic Modeling?
 When important questions can’t be answered by direct
observation because of
–Comparators (may differ from clinical trials)
–Time periods (extrapolation beyond trial)
–Patient selection (narrower/broader populations than in trial)
–Scope of disease impact (wider impact on health and economic
endpoints)
–Endpoint relevance (impact of clinical endpoints on future
health decisions)
–Uncertain evidence base (impact of uncertainty in effect size)
–Scoping (data from a range of disparate sources in a single
transparent framework)
–Setting (alternative countries/health care settings)
Properties of a good decision analytical
model
A good decision analytic model for the economic evaluation of health
technologies is one that:
• is tailored to the purposes for which it is to be used
• is useful for informing the decisions at which it is
aimed
• is readily communicated.
Transparency Internal consistency
Reproducibility Interpretability
Exploration of uncertainty Statement of scope
External consistency Parsimony
Inferential soundness
Modeling options
Decision Tree Model
No
Side response
effects
Response
Drug A
No
No side response
effects
Decision:Which Response
treatment shall I Final
use? No Outcome
Side response s
effects
Response
Drug B
No
No side response
effects
Choices Response
(Comparators) Branching
point =
alternative
events
When Do We Use a Markov Model?
 For modeling chronic conditions for which there are clear
stages of progression and severity
 or acute diseases with a lot of circulation between states
 When long-term data are not available to tell the whole
story
 To simplify the presentation of a recursive tree structure
Markov Model Components and
Considerations
 Health states
 Cycle length and model time horizon
 Transition probabilities
 Matrix/vector multiplication
 Markov assumption: memoryless property
 Parameters associated with health states
 Discounting
 Half-cycle correction
 Comparing interventions
 Sensitivity analysis
Identifying Health States

 Health states should be clinically and/or economically

relevant
 The selected set of health states must be exhaustive and
mutually exclusive
–Exhaustive: at every point in time, a person must be in one
of the health states
–Mutually exclusive: at any point in time, a person can be in
only one health state
Types of Health States
 Transient state: probability of transitioning to at least one
other state is positive
 Absorbing state: probability of transitioning to any other
state is zero (e.g., death)
 Tunnel state: probability of remaining in the state in the
next cycle is zero
How Many Health States?
 Enough states to ensure that individuals in any state are
relatively homogenous
–Transition probabilities for individuals in the state are the
same
–Costs and quality of life for individuals in the state are the
same
 Few enough states to be able to obtain good data for
each state
Transition Probabilities
 A transition probability is the
probability of moving from one
health state to another during a
given cycle
–Example: probability an individual
is in remission next year given
he/she had an episode this year
(arrow A)
–Individuals can stay in their
current state with some
probability (arrow B)
 Some transitions may have a
probability of zero
–In this example, the probability of
death during remission is zero
Transition Probability Matrix
Discounting
 Discounting accounts for the time value of money
–A dollar today is worth more than a dollar next year
 Separate discount rates can be applied to costs and
health outcomes
 The discount factor applied in each year, where α is the
annual discount rate and n is the year, is:

 Example: discount factor for year 5 with α = 3%:

Sensitivity Analysis
 One-way sensitivity analysis
– Transition probabilities, costs, utilities, etc.
– Used to assess drivers of model results
 Two-way sensitivity analysis
– Combine parameters above
 Probabilistic sensitivity analysis
– Vary all parameters simultaneously using Monte Carlo
simulation
– Used to assess uncertainty in model results as a result
of joint parameter uncertainty
 Subgroup and scenario analyses
Software Used for EE Modeling
Advanced spreadsheet packages such as Microsoft Excel
 Strengths: Most individuals have the software, models can be set up to be user
friendly, calculations can be customized and programmed to be transparent
 Limitations: Markov calculations must be programmed from scratch, programmer
must be familiar with Excel equations (and ideally Visual Basic for Applications)
TreeAge Pro
 Strengths: Software presents model structure in a visual diagram, Markov calculations
are built-in and executed automatically
 Limitations: Programmers and users need to purchase software, final model is less
user friendly for basic users, calculations are less transparent for basic users
Matlab
 Strengths: Computational powerhouse
 Limitations: Programmers and users need to purchase software, programmer must be
familiar with Matlab programming, final model is less user friendly for basic users,
calculations are less transparent for basic users
Budget Impact Models
 Budget impact models look at the affordability of
implementing a new intervention or switching between
interventions
 An intervention may be very expensive but the indication may
be very rare, e.g. enzyme replacement therapy for Gaucher’s
disease
 An intervention may be cheap but the indication very common,
e.g. statins for secondary prevention of MI
Budget impact models
 Budget impact models usually incorporate
 the number of patients who would be eligible for the new
intervention
 the resources/costs per patient needed to implement the new
intervention
 the resources/costs per patient saved from displaced
interventions
 the timing the uptake of a new intervention
Budget Impact Analysis
 Analysis of provider’s expenditures for a program over a
short period (often 1-3 years), including the effect of any
offsetting savings.
– Evaluates a scenario rather than a single action
– Includes comparison to the status quo
– Includes sensitivity analysis

 BIA excludes patient- incurred costs, but * BIA should

reflect impacts on enrollment and retention that could result
from affecting patients.
 Does not measure utility
BIA
Care maps/ care pathways
 Care maps/ care pathways cover an indication as a whole
rather than individual treatment decisions
 They are a representation of the (current) patient
management and treatment patterns
 Pathway leading to diagnosis
 Pathway for initiating treatment and subsequent treatment
 Pathway for following-up/ managing patient
 Care maps can be overlaid with costs to evaluate cost of
managing a patient or to calculate total budget impact of
implementing a new treatment pathway
Care maps/ care pathways
Measuring and valuing health

Denny John
Evidence Synthesis Specialist. Campbell Collaboration
Quality-of-life
 Quality-of-life (QOL) is a measurement of how
health, and therefore an intervention, impacts on
an individual’s well-being

 Quality-of-life instrument measures the impact of

health on different domains:
 physical well-being
 mental well-being
 social well-being
Methods for valuing HRQoL weights
 First step: Defining health states of interest
 Next step: valuing these health states i.e. individuals assess
different health states and place a value on each of them

 Direct elicitation methods:Visual Analogue Scale, Time

trade-off, Standard gamble
 Generic preference-based measures
 Condition-specific measures
Visual Analogue Scale

 Subject asked to judge where

his/her current health would be
on a visual scale of 0 (dead) to
1 (perfect health)

E.g. EQ-VAS
Time trade-off

 Individuals decide how many years of remaining life

expectancy they would exchange in return for perfect
health
Standard Gamble
Individuals chose between
A) remaining in their current health state or
B) the gamble: an X% chance of perfect health but (1-X%) chance of
death
X is varied until individuals are indifferent between choices A and B
 Generic instruments
e.g. EQ-5D, SF-36
 Used to compare across indications
 Preference-based instruments:
 integrate all factors contributing to QOL and provide a single
measure of how patients value their current health state

Instrument Domains Levels of Potential Valuation Original

response Health method population are
state used based on
EQ-5D 5 3 245 TTO Random sample of
approximately 3000
adults in the UK
The EQ-5D questionnaire
Specific instruments
e.g. Hospital Anxiety and Depression Scale, EORTC QLQ-
C30
 Disease-specific or population specific
 Can be more sensitivity than generic instruments
Utility and QALYs
 A QALY is quality-adjusted life year
 It is used in economic evaluations as a global measure of
both the quantity and quality-of-life
 A QALY is a utility-based measure (preference based
measure)
Utility and QALYs
 A utility is an concept used in economics to measure the
desirability or usefulness of one thing over another
 In health economics it is used to indicate a preference for
one health state over another
 It provides a ranking between health states
 It provides the weighting between health states
 In general a utility is a value between 0 (dead) and 1
(perfect health) though can have negative utilities
Utility and QALYs
 A QALY is a measure of the quantity and quality-of-life
 For example,
 if a treatment extends life expectancy by three years
 but at only 60% of full health (i.e. a utility of 0.6)
 then the QALY gain associated with the treatment is 1.8
QALYs (3 years x 0.6).
 The advantage of using QALYs is that this measure allows
for comparisons across different indications as well as the
evaluation of different therapy options within a particular
disease area.
Example of QALY calculation
 Health State 1: Change in health-related QoL over time
0.9
 2 years @ 0.8 =
0.8
 1.6 QALYs
 Health State 2: 0.7

 3 years @ 0.7 = 0.6

 2.1 QALYs 0.5

Utility
Health
 Health State 3: 0.4
State 1
Health State 2

 2 years @ 0.5 = 0.3

Health
State 3
 1 QALY 0.2
Health State 4
 Health State 4: 0.1

 3 years @ 0.3 = 0
1 2 3 4 5 6 7 8 9 10
 0.9 QALYs Years

 Total 5.6 QALYs

Methods for valuing HRQoL weights
Criticisms to QALYs
 Utility theory
 Does not take into account all dimensions of health
benefits
 ’QALY is a QALY is a QALY’- an intervention that results
in a small loss of QALYs for some but a greater gain of
QALYs for others will result in net efficiency gains and
hence social improvement, irrespective of the resulting
distribution
 Issues with Equity (some methods of equity weights,
ECEA, DCEA) and efficiency
Alternatives to QALYS- DALY
Disability Adjusted Life Year (DALY)
 Summary measure of population health to estimate global
burden of illness.
 DALY is an indicator of the relative impact of illnesses
and injuries on losses of healthy life years.
 Disability weights are applied to time intervals with the
disease. Disability weights were obtained in a valuation
exercise with a group of healthcare professionals

While QALY weights reflect relative preferences of an

individual for health states, DALY weights reflect the degree
to which health is reduced by a disease condition.
DALY-Calculation
 DALY: YLD + YLL

 YLD = Number of cases x duration till remission or

death x disability weight

 YLL = Number of deaths x life expectancy at the age of

death

 Age weighting:Value of life depends on age. Higher weight

is given to the healthy life years lived in the (assumed)
socially more important life span between 9 and 56
(Murray, 1994)
Alternative to QALY- HYE
Healthy year equivalent

 QALY measure is not consistent with utility theory

 HYE is an alternative, and is measure of quality of the life
that is based on a two-stage procedure using an SG
question to elicit preferences
 Better approach in principle compared to QALYs
 Criticism: Difficult of implementation
Alternative to QALY- WTP
 Willingness-to-pay

 Used within CBA (instead of CEA/CUA)

 Obtain valuations of health benefits in monetary terms by
asking individuals how much they would be willing to pay
to obtain or avoid the health effects.

 Criticism: WTP is closely associated with ability to pay

Cost of illness

Case Study
Cost-of-illness (COI)
 COI identifies the economic burden of a disease or
medical condition
 In general these studies evaluate the resources consumed
as a direct result of an illness or condition
 COIs always take a specific cost perspective, e.g. UK NHS
 Also called cost-of-disease or burden-of-illness studies
Severity and cost of hospitalization for dengue in
public and private hospitals in Surat city, Gujarat,
India, 2017-18

Viral R Bajwala, Denny John, Daniel Rajasekar, Manoj V

Murhekar

Forthcoming publication
Background
 India accounts for approximately half of the 205 billion
people who are at risk of dengue fever
 High medical costs on dengue treatment in India
 All previous studies in India estimated cost of treatment
for dengue illness in private hospitals

Study Objective
Estimate economic cost of dengue hospitalization for the
year 2017-18 in Surat city
Methodology
 5 tertiary care hospitals (2 semi-government hospitals, 1
government hospital, 2 private hospitals) as study sites
 Review of medical records of patients hospitalized in any
of the selected hospitals with a clinical diagnosis of
dengue or laboratory confirmed dengue infection
 Use of pre-tested data extraction form to collect
information about socio-demographic, clinical and cost
details
 Cost of illness estimated using incidence-based approach
using societal perspective
Methodology
Direct medical costs ICU & Hospital Stay
Laboratory tests
Radiology investigations
Doctor visits
Intravenous blood transfusions
Procedures (fluid tapping, lumbar
puncture etc)
ICU management (Oxygen, infusion
pump, monitor, bipap ventilator)

Indirect medical cost Minimum wage as per wage floor index

reported by Ministry of Labour
Cost calculation
 Cost in private adult hospital and government hospital
was calculated as follows:
Cost of variable= Number of units X per unit cost of that
variable................(1) (per unit cost was obtained from same
hospital, pharmacy and lab). For other hospitals, the direct
medical costs were obtained directly from bills

 Wage loss=Duration of hospital stay X 160 (INR 160

is minimum wage per day)......(2)

 Per day cost of hospitalization per patient = (Mean

cost/Average length of stay).......... (3)
Description of cost of hospitalization in USD (2018 prices) for
dengue patients in Surat city, India, 2017-18
Type of cost Government Semi government Private hospitals Total
hospitals (n=257) hospitals (n=173) (n=732)
(n=302)

Direct Medical

Mean+SDa 1.0+1.4 17.4+22.7 311.3+221.6 81.1+168.0

Median(IQRb) 0.9 (0.6-1.0) 11.7 (8.8-16.0) 270.2 (193.5-368.2) 10.1 (1.0-62.2)

Wage loss

Mean+SD 11.9+4.2 13.0+7.0 12.1+4.5 12.3+5.5

Median(IQR) 8.2 (0.9-13.4) 17.3 (9.9-27.8) 274.9 (197.8-380.4) 12.5 (10.0-15.0)

Total cost

Mean+SD 7.5+6.8 23.6+26.5 320.5+225.1 88.2+170.3

Median(IQR) 8.2 (0.9-13.4) 17.3 (9.9-27.8) 274.9 (197.8-380.4) 15.8 (8.3-67.3)

Fee exempted

Mean+SD 0.8+0.2 20.2+24.5 0 15.3+22.8

Median(IQR) 0.9 (0.6-0.9) 12.3 (8.5-20.7) 0 9.0 (0.9-17.9)

Final cost after exemption

Mean+SD 7.5+6.8 20.2+19.4 320.5+225.1 86.9+170.7

Median(IQR) 8.2 (0.9-13.3) 15.3 (8.3-25.2) 274.9 (197.8-380.4) 14.8 (7.3-65.9)

Cost-effectiveness/Cost-utility
analysis
Case Study
Cost-effectiveness analysis (CEA)

 This analysis compares the monetary cost of an

intervention with a measure of effectiveness (clinical or
quality of life outcome)
 The outcomes are measured in natural units and usually
expressed as the incremental gain/loss resulting from an
intervention:
 bed days avoided
 deaths avoided
 life years gained
Cost-utility analysis (CUA)
 An extension of cost-effectiveness analysis where
effectiveness is expressed in utilities, such as QALYs
 This allows comparisons across different indications
 CUA very common form of economic analysis, e.g.
Required for NICE/ SMC
 One criticism of CUA is how useful it is for acute (short-
term conditions) or conditions where utility/quality-of-life
difficult to measure
Screening Protocol
Simulated screening
& treatment pathway
Screening strategies & Outcomes
 Option 1: Inviting population aged 40-69 years from low-
income urban areas.
 Option 2: Inviting population aged 40-69 years from rural
areas.
The main outcomes of the study were defined as
(a) Total net cost of each strategy (i.e. costs of screening
minus costs of case finding)
(b) Additional cases treated in the screening arm
(c) Cost per QALY gained by screened.
Methods
 Hypothetical population of 1 million aged 40-69 years in
both urban and rural parts of India
 Time-horizon: 10 years
 Discount rate: 3 %
 Perspective: Health system
Probabilities- With no screening

(1) Population seeking eye examination

(2) Glaucoma detection
(3) Proportion of POAG/PACG in eye clinics/hospitals
(4) Probability of POAG and angle closure disease (PAC +
PACG) cases requiring treatment
(5) Probability of POAG and angle closure disease (PAC +
PACG) requiring medical and surgical treatment
Probabilities- With screening
(1) Screening uptake
(2) Post test probabilities of screening tests
(3) Negative screening results (including unreliable test
results)
(4) Eye examination compliance post-screening
(5) Probability of cases diagnosed with glaucoma after
examination in the hospital
(6) Proportion of POAG and PAC+PACG cases
(7) Probability of POAG and angle closure disease (PAC +
PACG) cases requiring treatment
(8) Probability of POAG and angle closure disease (PAC
and PACG) requiring medical and surgical treatment
Utility values
Costs
(1) Screening costs- Screening invitation costs, Screening
clinic costs
(2) Examination costs
(3) Costs of treatment
Decision Tree
Tornado diagram
Cost-minimization analysis

Case Study
Cost-minimisation analysis (CMA)
• A cost comparison of two or more interventions with comparable clinical and
quality-of-life outcomes
• It is unlikely that the outcomes of two different interventions are equal so CMAs
are rarely performed
• Useful for evaluating generically equivalent drugs where the outcomes have been
demonstrated to be equivalent
• Useful for evaluating same intervention but given in different settings
Background
• WHO End TB Strategy 2035 targets: No TB-affected families facing
catastrophic costs due to TB, and removal of financial barriers to health-
care access is vital to achievement of universal health coverage and
prevention of catastrophic expenditures
• Recent meta-analysis has shown that ambulatory models of MDR-TB
treatment are equally effective and result in similar patient outcomes as
facility-based care.
• However, the World Health Organization (WHO) also recommends home
based treatment as a viable alternative for MDR-TB treatment

Study objective:
 Cost-minimisation analysis was conducted to assess the potential savings
associated with an ambulatory-based MDR TB model from the perspective
of the Nigerian national health system
Methods
 Model: Decision Tree
 Time-period: 2 years
 Intervention: Home-based treatment for MDR-TB
 Comparator: Facility-based treatment for MDR-TB
 Outcomes of interest: treatment success, treatment failure, treatment
default, and mortality obtained from a systematic review of
observational studies (Bassili et. al., 2013)
 Costs: Treatment costs included the cost of: drug therapy, hospital stay,
nurse care, physician care, nursing facility and transport
 Cost inputs: Cost of anti-TB medication from published International
Drug Price Indicator Guide, other costs based on internal analysis at
Aminu Kano Teaching Hospital, Nigeria
 Output: Potential cost savings associated with home based treatment
for all patients starting MDR TB treatment in Nigeria
Methods..
 Treatment efficacy was similar to both arms of the model
based on meta-analysis study
 Treatment probabilities

 One-way sensitivity analysis

 Software: MS-Excel
Decision Tree
RESULTS
• Average expected total treatment cost for a patient in Nigeria
treated for MDR TB was estimated at US$ 2095.82 for the facility
based model and $1535.06 based on the ambulatory care model, a
potential saving of 25%.

• One of the major drivers of this difference is the significantly more

intensive, and therefore costlier, nursing care in hospital.

• In the year 2013, an estimated 426 patients with MDR-TB were on

treatment, thus, the potential savings associated with
implementation of home based care is estimated at US$ 2.2 million
for the year 2013
STUDY IMPLICATIONS
 Our study provides evidence of cost savings for MDR-TB patients requiring
hospitalization for ambulatory care patients in comparison with facility-based
treatment with similar outcomes.

 These cost savings may improve equity, however covering of indirect costs such
as travel as part of the current government initiative for covering MDR-TB
costs under the state health insurance schemes could mitigate the costs impact
on low-income families as well.

 Conclusion: In Nigeria, treatment of MDR TB using home based care is

expected to result in similar patient outcomes at markedly reduced public
health costs compared with facility based care.
Cost benefit analysis (CBA)

Case study
Background
 Electronic Medical Record (EMR) systems have the
potential to provide substantial benefits to physicians,
clinic practices, and health care organizations
 For widespread adoption of EMR a business case using
cost-benefit analysis of implementing EMR was conducted
Methods
 Use by primary care physicians in ambulatory-care setting
 Data on costs and benefits from primary data of existing
EMR, published studies and expert opinion (using Delphi
technique)
 Comparator was traditional paper-based medical record
 Primary outcome measure was net financial costs or
benefits per provider during a 5-year period
 Model framed using a healthcare organization perspective
Results
 The estimated net benefit from using an electronic medical
record for a 5-year period was $86,400 per provider.
 Benefits accrue primarily from savings in drug expenditures,
improved utilization of radiology tests, better capture of
charges, and decreased billing errors.
 In one-way sensitivity analyses, the model was most sensitive
to the proportion of patients whose care was capitated
 Net benefit varied from a low of $8400 to a high of $140,100.
A five-way sensitivity analysis with the most pessimistic and
optimistic assumptions showed results ranging from $2300 net
cost to $330,900 net benefit
 Consolidated Health Economic
Evaluation Reporting Standards
(CHEERS)
Denny John, Evidence Synthesis Specialist
Campbell Collaboration
 Challenges with reporting of EE studies
 Has been called the “black box”[1]

 Require more space for resource use, valuation

procedures and (often) modeling

 Used for decision-making yet,

 No consensus format or checklist
 No registries or warehousing of information
 Evidence of wide variability in reporting

1. John-Baptiste AA, Bell C. A glimpse into the black box of cost-effectiveness

analyses. CMAJ. 2011 Apr 5;183(6):E307–308.
CHEERS-History
 Several existing guidelines that require
updating/consolidation (BMJ/Drummond, Annals/LDI,
Gold/CEA Task Force)
 BMJ considering updating their guidelines
 Task Force approved in November 2009
 Work began in 2010-change in scope/structure/leadership
in 2011
Purpose of CHEERS
 A paper that meets all the requirements in the checklist
will:
 Clearly state the study question and its importance to decision
makers
 Allow a reviewer and a reader to assess the appropriateness of
the methods, assumptions, and data used in the study
 Allow a reviewer and reader to assess the credibility of the
results and the sensitivity of the results to alternative data
choices
 Have conclusions that are supported by the study results
 Potentially allow a researcher to replicate the model
Recommendations
 The recommendations are subdivided into the five
sections generally found in a paper presenting an
economic evaluation
 Title and Abstract
 Introduction
 Methods
 Results
 Discussion
 CHEERS Checklist – Items to include when reporting
economic evaluations of health interventions (1)
Ite
Section/Item m Recommendation
No
Title and abstract
Identify the study as an economic evaluation, or use more specific terms
Title 1 such as ``cost-effectiveness analysis``, and describe the interventions
compared.
Provide a structured summary of objectives, perspective, setting, methods
Abstract 2 (including study design and inputs), results (including base case and
uncertainty analyses), and conclusions.
Introduction

3 Provide an explicit statement of the broader context for the study.

Background and
objectives Present the study question and its relevance for health policy or practice
decisions.
Methods
Target Population and Describe characteristics of the base case population and subgroups
4
Subgroups analyzed including why they were chosen.
State relevant aspects of the system(s) in which the decision(s)
Setting and Location 5
need(s) to be made.
Describe the perspective of the study and relate this to the costs being
Study Perspective 6
evaluated.
Describe the interventions or strategies being compared and state why
Comparators 7
they were chosen.
State the time horizon(s) over which costs and consequences are
Time Horizon 8
being evaluated and say why appropriate.
CHEERS Checklist – Items to include when reporting
economic evaluations of health interventions (2)
Item
Section/Item Recommendation
No
Report the choice of discount rate(s) used for costs and outcomes and
Discount Rate 9
say why appropriate.
Choice of Health Describe what outcomes were used as the measure(s) of benefit in the
10
Outcomes evaluation and their relevance for the type of analysis performed.
Single Study-Based Estimates: Describe fully the design features of the
11a single effectiveness study and why the single study was a sufficient
Measurement of source of clinical effectiveness data.
Effectiveness
Synthesis-based Estimates: Describe fully the methods used for identi-
11b fication of included studies and synthesis of clinical effectiveness data.
Measurement and
Valuation of If applicable, describe the population and methods used to elicit
12
Preference-Based preferences for outcomes.
Outcomes
Single Study-based Economic evaluation: Describe approaches used
to estimate resource use associated with the alternative interventions.
13a Describe primary or secondary research methods for valuing each
resource item in terms of its unit cost. Describe any adjustments made
Estimating
to approximate to opportunity costs.
Resources and
Costs Model-based Economic Evaluation: Describe approaches and data
sources used to estimate resource use associated with model health
13b states. Describe primary or secondary research methods for valuing
each resource item in terms of its unit cost. Describe any adjustments
made to approximate to opportunity costs.
CHEERS Checklist – Items to include when reporting
economic evaluations of health interventions (3)
Item
Section/Item Recommendation
No
Report the dates of the estimated resource quantities and unit costs.
Currency, Price
Describe methods for adjusting estimated unit costs to the year of
Date and 14
reported costs if necessary. Describe methods for converting costs into
Conversion
a common currency base and the exchange rate.
Describe and give reasons for the specific type of decision-analytic
Choice of model 15 model used. Providing a figure to show model structure is strongly
recommended.
Describe all structural or other assumptions underpinning the decision-
Assumptions 16
analytic model.
Describe all analytic methods supporting the evaluation. This could
include methods for dealing with skewed, missing or censored data,
Analytic Methods 17 extrapolation methods, methods for pooling data, approaches to
validate or make adjustments (e.g., half-cycle corrections) to a model,
and methods for handling population heterogeneity and uncertainty.
Results
Report the values, ranges, references and if used, probability
distributions for all parameters. Report reasons or sources for
Study parameters 18
distributions used to represent uncertainty where appropriate. Providing
a table to show the input values is strongly recommended.
For each intervention, report mean values for the main categories of
Incremental costs estimated costs and outcomes of interest, as well as mean differences
19
and outcomes between the comparator groups. If applicable, report incremental cost-
effectiveness ratios.
CHEERS Checklist – Items to include when reporting
economic evaluations of health interventions (4)
Item
Section/Item Recommendation
No
Single study-based economic evaluation: Describe the effects of
sampling uncertainty for estimated incremental cost, incremental
20a effectiveness and incremental cost-effectiveness, together with the
Characterizing impact of methodological assumptions (e.g. discount rate, study
Uncertainty perspective).
Model-based economic evaluation: Describe the effects on the results
20b of uncertainty for all input parameters, and uncertainty related to the
structure of the model and assumptions.
If applicable, report differences in costs, outcomes or cost-effectiveness
that can be explained by variations between subgroups of patients with
Characterizing 21 different baseline characteristics or other observed variability in effects
Heterogeneity that are not reducible by more information.
Discussion
Study Findings,
Summarize key study findings and describe how they support the
Limitations,
22 conclusions reached. Discuss limitations and the generalizability of the
Generalizability, and
Current Knowledge findings and how the findings fit with current knowledge.
Other
Describe how the study was funded and the role of the funder in the
Source of Funding 23 identification, design, conduct and reporting of the analysis. Describe
other non-monetary sources of support.
Describe any potential for conflict of interest among study contributors
in accordance with journal policy. In the absence of a journal policy, we
Conflicts of Interest 24
recommend authors comply with International Committee of Medical
Journal Editors’ recommendations
Campbell & Cochrane Economic Methods
Group (CCEMG)
Webpage
 https://methods.cochrane.org/economics/
 Further training materials

Cochrane Handbook

 Chapter 20, Economic Evidence, Cochrane Handbook for Systematic Reviews of

Interventions. London: Cochrane. Shemilt I, Aluko P, Graybill E, Craig D,
Henderson C, Drummond M, Wilson E, Wilson S, Vale L on behalf of the
Campbell and Cochrane Economics Methods Group. Chapter 20: Economics
evidence. Draft version (15 September 2018) in: Higgins JPT, Thomas J,
Chandler J, Cumpston MS, Li T, Page MJ, Welch V (editors). Cochrane
Handbook for Systematic Reviews of Interventions. London: Cochrane.
Correspondence

 Contact CCEMG janice.legge@newcastle.ac.uk