Acute care toolkit 11

Using data to improve care

January 2015
Working as a physician involves not only providing the best possible care for the
patients of today, but also working to improve care for the patients of tomorrow.
Collecting and understanding data is central to both of these aims, as without
information and data, a clinician has little to draw on to answer two fundamental
questions: How am I and my team doing? What can we do to improve?
The complexity of modern healthcare means that answering
these questions requires data from multiple sources. Data
from one source (such as patient experience data) may be
best understood in the context of other data (staffing ratios
or infection rates). For individual teams and clinicians, the
most powerful data for improvement is often not numerical
or quantitative, but rather qualitative information that
provides insight into the reasons for the care that patients
received and how it might be improved.
The aim of this toolkit is to help clinicians to draw together the
information that is necessary to understand the quality of care
that they and their team provide. This includes a description of
the main types of data, considerations of how it should (or should
not) be interpreted, and how data can be used to help improve
healthcare quality. It is not a comprehensive guide to clinical audit
or quality improvement, and sources of more detailed information
have been referenced.

Introduction
Most quantitative data about patient outcomes cannot
meaningfully be interpreted at the level of the individual clinician.
Small samples of data make comparisons difficult or impossible
the smaller the sample, the greater the chance that any difference
or change is due to random variation or other factors, such as
differences in patient characteristics, rather than being a true
indicator of performance.
For individual clinicians and teams, the most meaningful
information is qualitative (non-numerical) information about the
outcomes of care and quantitative information about the process
of care (see Table 1 on the fold-out). For example, while differences
in the rate of safety incidents between clinicians are likely to be
meaningful only under exceptional circumstances, all teams can
improve their care by understanding the nature of and reasons for
the safety incidents, and by measuring the care processes that are
necessary to ensure patient safety.

Small samples of data make comparisons difficult or impossible the

smaller the sample, the greater the chance that any difference or change
is due to random variation or other factors, such as differences in patient
characteristics, rather than being a true indicator of performance.

Acute care toolkit 11 January 2015

Box 1 Suggested approach to

mortality case-note review
1 Using a standard methodology (eg the UK modification
of the Institute for Healthcare Improvement (IHI) Global
Trigger Tool)8 will help to identify common themes.
2 If it is not practical to review the case notes of all patients
who have died, then consider:
> a two-stage process, with initial screening out of
straightforward cases
> a representative sample of case notes.
3 Use teams of reviewers, from different clinical
backgrounds and specialties, who have had training in the
methodology and understand the clinical issues involved
in a particular patients care.
4 Reviewers can review case notes alone, or in pairs so that
difficult cases can be discussed.
5 Have a policy for dealing with the occasional case where
the actions of an individual clinician could be an issue
(which usually involves escalation to the medical director).
6 Collate outputs into common themes, to support quality
improvement activity.

Box 2 Types of healthcare

quality data
Organisational measures. These describe how services are
structured and planned, such as the number of beds, the
availability of equipment and staffing numbers. They may
also be less tangible aspects of how care is planned for
example, the presence of care pathways and protocols.
Process measures. These describe the healthcare that is
provided: the assessments, investigations and treatments
that patients receive. For example, in a patient with acute
myocardial infarction, process measures might include doorto-needle time of primary angioplasty or the prescription of
antiplatelet therapy.
Outcome measures. These are direct measures of
the outcomes of care. For example, in a patient with
pneumonia, outcome measures might include mortality,
length of stay or the amount of time it took before they
could return to work. Some outcomes can be objectively
measured but some are subjective, for example quality
of life and pain after a hip replacement (an example of a
patient-reported outcome measure PROM), or a patients
own opinion about whether they were treated with care
and dignity (an example of a patient-reported experience
measure PREM).

There are many potential sources of data to help physicians to

measure, learn from and improve on what they do (Appendix
available online at www.rcplondon.ac.uk/resources/acute-caretoolkits). The types of data needed depend on the purpose for which
they are used and the level of measurement. All data sources have
strengths and weaknesses such as potential for error and bias,
effort of collection and ease of interpretation. Data collection can
be time consuming and labour intensive, so existing data sources
are usually preferable to additional data collection exercises.
In addition to collecting and using data for improving care quality,
data may also be collected for research (to test hypotheses about
treatments and processes) and for judgement (for accountability
to regulators, payers, policymakers and the public). Data should
be shared and used as fully as possible, while the ethical and
information governance issues that may be involved in sharing data
should be fully considered.

1 Learning from clinical incidents

and complaints
Complaints and clinical incident reports can be a rich source of
learning for clinicians about the quality and safety of their service.
Only a minority of clinical incidents are reported, so even where a
mandatory requirement to report exists, data about rates or numbers
of incidents (which many organisations and national bodies collect)
do not give meaningful information about safety. They may even
be misleading; high reporting rates can either arise because more
incidents occur or because there is a strong safety culture that
encourages reporting.
Clinical leaders and trusts should support a culture where reporting
and learning from incidents is encouraged. The main aim of incident
reporting should not be to identify individuals to blame, but to
understand the root causes of incidents as part of a process of
quality improvement. Consultants and trainees should routinely be
involved in reporting and reviewing clinical incidents. Trainees have
a unique perspective in understanding issues relating to day-today clinical care and should be supported in taking part in incident
reviews by their educational supervisors.
When reviewing incidents or complaints, clinical teams should
specifically look for:
> common themes where systems of care have broken down, eg
problems with handovers or systems for reviewing investigation
results
> warning signals of potential safety hazards, eg with faulty
equipment or devices
> high-risk situations, eg in specific clinical areas or in the treatment
of specific conditions such as sepsis or dehydration.
Most safety incidents occur as a result of a sequence of system errors
rather than individual action or failure, so a punitive, blame-seeking
approach is usually unhelpful and should be avoided. An understanding
of the conditions that can lead to errors and human factors science is
helpful in this.1 All humans make mistakes, and patient safety can be
improved by understanding the reasons why these occur, and designing
systems to reduce the risk of harm from error.
In managing complaints, while it is essential for organisations to
be responsive in communicating with complainants, opportunities
for learning can be missed if a systematic approach that looks for
common themes across complaints is not taken. Complainants often
highlight issues (eg a lack of recognition of, or response to, clinical
deterioration) that have not been apparent from other sources.
Learning from complaints is also central to improving the experience
of healthcare services by patients, families and carers.

Taking the opportunity to learn from and reflect on complaints is an

important part of continuing professional development. Complaints
often concern multiple aspects of care (eg nursing, therapy,
investigations, waiting times), and clinicians should contribute to and
support improvements in the overall care of patients, not only those
directly relating to the care provided by physicians.
Complaints should receive a prompt response. While some
complaints require written reports, there will be instances where an
early meeting to discuss a complaint may lead to a more satisfactory
resolution. Most complainants do not seek recompense, but they
instead seek reassurance of change and improvement. Proactive
clinical engagement helps to facilitate learning and change, as
well as improving patients and carers experience of care and their
perception that their complaint has been responded to appropriately.

2 Using feedback from staff and

colleagues
For individual clinicians, multi-source (or 360-degree) feedback
from colleagues is an important component of the appraisal and
revalidation process, and can help to highlight areas for personal
and professional development.2 Multi-source feedback needs to be
sought from a broad range of individuals and not just from those
who are anticipated to only provide favourable feedback. The list of
potential individuals from whom feedback is to be sought may need
to be approved beforehand by a supervisor or employer.
When providing feedback about colleagues, it is important that
clinicians give an honest and balanced account, providing positive
feedback about good practice but also constructive criticism
where issues need to be addressed. Written comments or face-toface feedback is usually more useful than rating scores in helping
colleagues to identify areas where practice can be improved.
Formal and informal feedback from staff can give useful insight into
issues at unit or hospital level, and there is evidence of an association
between staff satisfaction and patient experience.3 The NHS staff
survey is rarely accessed by clinicians, but it is freely available and has
been shown to have validity at specialty as well as hospital level.4
The General Medical Councils (GMCs) national training survey5
receives responses from almost all doctors in training, and results
are available to every trust. While the main focus is on the quality of
training, there are also important questions about patient safety and
trainees perceptions of being supported to raise concerns.
Trainees and other staff may not feel able to formally report
concerns about safety, but they may raise these informally with
consultants or educational supervisors. Such concerns should always
be taken seriously; the GMC has useful guidance for clinicians who
face such concerns.6

3 Understanding mortality
Around 5% of patients who are admitted to hospital will die. Most
deaths are in frail, elderly patients with multiple medical problems
who have been admitted as emergencies and in whom death is
unfortunately inevitable. While most deaths are inevitable, some are
not; studies suggest that around 5% of hospital deaths might have
been avoidable with higher-quality care.7 These estimates suggest
that approximately 1 in 400 patients admitted to hospital suffer an
avoidable death.

Clinicians should understand the factors that lead to hospital deaths.

They can do this by:
> case review of the notes of all, or a sample of, patients who have
died or have suffered a near miss such as a cardiac arrest, using
a standard methodology (see Box 1 left): this can provide
valuable information for clinicians and teams on opportunities for
improvement
> understanding data on mortality rates provided at hospital or
service level.
The primary purpose of reviewing the case notes of patients who
have died is to detect patterns where processes might have broken
down and to learn where care might be improved, even if the death
was unavoidable. This could be with a specific clinical issue (eg
recognition of, and response to, sepsis) or in a particular area of the
hospital (eg patients on general wards within a few days of transfer
from the acute medical unit AMU).
Common themes that are reported in mortality case-note review in
medical patients are:
> the recognition of, and response to, clinical deterioration
> fluid balance and acute kidney injury
> the recognition and management of sepsis
> medication safety issues
> frequency of senior clinical review
> inadequate end-of-life care planning.
Mortality rates are rarely meaningfully interpretable at the level of
a clinical team or individual physician. They may, however, provide
information on the safety and effectiveness at the hospital or service
level by acting as a smoke signal for potential problems in quality,
and to track progress over time.
Crude mortality rates are not a good measure of quality, as they
are affected by numerous factors, including patient characteristics
(hospitals that admit older or sicker patients have higher death rates)
and the provision of local services such as hospice and community
care. Standardised mortality measures were developed to correct for
these factors and are now in widespread use throughout most of the
NHS (see Table 2 above right). Clinicians need to understand the
strengths and weaknesses of standardised mortality measures, to
gain the most value from them.

4 Making measurement meaningful

Neither data collection without improvement activity nor quality
improvement without appropriate measurement are effective ways
of improving care.
Measurement can occur at different levels. Local audits can be
valuable for addressing specific local issues, although isolated
projects often lack follow through; the Royal College of Physicians
(RCPs) Learning to make a difference project13 coaches trainees in
quality improvement skills to address this gap. There are many other
resources available to help teams to apply quality improvement
methods and to learn more about using data for quality
improvement.14,15
National clinical audits and registries are the gold-standard
source of data for many conditions. They benefit from rigorous
methodological and statistical design, have high clinical credibility,
provide benchmarking against peers, and are supported by national
guidelines and recognised approaches to quality improvement.

Using data to improve care

Table 2 Standardised mortality measures

Standardised mortality measures, such as the hospital standardised mortality ratio (HSMR) or the summary hospital
mortality indicator (SHMI), are presented as a ratio of actual to expected mortality. An expected mortality rate is calculated
for each hospital using data derived from discharge coding: using a statistical model to forecast the number of deaths that a
hospital would be expected to have, based on the characteristics of the admitted patients. Because expected mortality rates
are based on discharge coding, it is important for clinicians to support accurate coding (for example, by avoiding the use of
symptom diagnoses such as chest pain). The SHMI and HSMR have a number of important differences:
SHMI

HSMR

Data source

Discharge codes

Discharge codes

Methodology

Statistical model to forecast expected

number of deaths

Statistical model to forecast expected

number of deaths

Mortality measure

Inpatient deaths or death within 30 days

of discharge

Inpatient deaths

Included patients

All deaths

Exclusions for certain diagnoses

These measures are controversial, especially when they are

used to compare hospitals.9,10,11,12 Much of the controversy
arises from the standardisation method, as it is based on
coding of variable quality. Hospitals with more detailed
coding tend to have higher expected mortality (because
there is more information on which to predict this) and
hence a lower standardised mortality.

Although HSMR and SHMI are often presented as a single

monthly or quarterly figure, this is unhelpful without
knowledge of the control limits and the pattern over time.
Observing changes in mortality data over time is generally
a much more useful guide to quality than a one-off
measurement.

Standardised measures are presented using statistical

process control (SPC) methodology with 100 as the reference
value and upper and lower control limits calculated
(analogous to confidence intervals). Control limits are usually
set so that the probability of a value lying outside them
by chance is less than 2 per 1,000. Rates are reported as
abnormal if they are outside the control limits.

> reliance on a single figure without reference to data

over time
> the use of mortality measures on their own as an indicator
of quality
> dismissing high readings as being due to coding defects
> taking reassurance from low readings without further
understanding the data
> use for ranking hospitals.

Clinical teams that contribute to the pathway of patients included in

national audit (eg stroke and myocardial infarction) should use these
data to plan and measure quality improvement activity.

Common pitfalls with standardised mortality measures:

The RCP therefore supports a multidisciplinary, clinically led,

integrated approach to presenting data through ward- or unit-level
look back and learn meetings, led by a senior clinician who can help
others to understand and learn from the data.

The types of data that can be measured fall into three categories
Meetings should be:
(see Box 2 left). Often, a good assessment of the quality of care
requires information about all three areas. For individual clinicians
> multidisciplinary, including nurses and other clinicians, managers
and teams, organisational and process data are the most meaningful
and administrative and support staff
and amenable to improvement.
> aimed at integrating data at ward level to understand the big
picture: operational data (eg bed occupancy and length of stay
5 Putting it all together: look back and
etc); quantitative data (eg clinical audits and safety measures); and
qualitative data (eg complaints and informal feedback from staff
learn meetings
and patients)
Most healthcare services have not developed a systematic approach
to bringing together information about the quality and performance
of care as a focus for quality improvement. For example, traditional
mortality and morbidity meetings are typically separate from
meetings that focus on audit results, quality improvement,
operational performance (eg waiting times, bed occupancy),
and education and training. Individual issues are therefore often
addressed in isolation, without insight into the system issues that
usually need to be addressed in order to improve care.

> structured and focused, following a common format, with the aim
of understanding common themes and developing approaches to
address these
> aimed at developing outputs that are used for quality improvement
activity
> documented, so that issues can be tracked (for example, in a risk
register) and improvement can be monitored
> a resource for trainees undertaking quality improvement work
> a source of learning for both individuals and organisations
> a source of data for clinicians to support individual appraisal and
revalidation. n

Using data to improve care

Table 1 Outcomes of care and quantitative information about the process of care
Team/clinician level

Service/hospital level

> Qualitative data on the outcomes of care is most useful.

>Q
 uantitative data on the outcomes of care is most useful.

> Quantitative data on the process of care is most useful.

> Quantitative data on the process of care is most useful.

Examples:

Examples:

> nature and root cause analysis of safety incidents

>m
 ortality rates

> complaints and compliments

>n
 ational clinical audit

> process audit, eg completion of national early

warning scores (NEWS)

>o
 utcomes audit, eg condition-specific clinical outcomes

> multi-source feedback.

> patient-reported outcomes.

Glossary of commonly used terminology in quality improvement

Term

Definition

Adverse event

Injury or harm caused by healthcare

Failure to rescue

Failure to respond appropriately to clinical deterioration or an urgent health problem

Global Trigger Tool

A standardised method for studying adverse events

Human factors

The study of human behaviour as it affects the design and implementation of equipment, environments and
ways of working

Lean

A type of quality improvement methodology that is focused on identifying wasteful activity and increasing
value

Never event

Serious, largely preventable patient safety incidents that should not occur if the available preventative
measures have been implemented

PDSA

Plan Do Study Act: the cycle of activities involved in quality improvement

Root cause analysis

A systematic method to identify the underlying causes of problems or errors

Safety culture

The commitment to safety within an organisation

SBAR

Situation Background Assessment Recommendation: a structured approach to communication between

members of a healthcare team

Six Sigma

A type of quality improvement methodology

Statistical process
control

A method of quality control that uses statistical methods to monitor performance

Table-top exercise

A simulated interactive exercise to test an organisations response to an emergency

Acute care toolkit 11 January 2015

References
1 Reason J. Human error. Cambridge: Cambridge University Press, 1990.
2 Royal College of Physicians. Appraisal and revalidation: guidance for
doctors preparing for relicensing and specialist recertification 2. Multisource feedback (360-degree assessment). London: RCP, 2007.
3 Maben J, Peccei R, Adams M et al. Exploring the relationship between
patients experiences of care and the influence of staff motivation, affect
and wellbeing. Final report. Southampton: National Institute for Health
Research Service Delivery and Organisation Programme, 2012.
4 Sullivan PJ, Harris ML, Doyle C, Bell D. Assessment of the validity of the
English National Health Service adult in-patient survey for use within
individual specialties. BMJ Qual Saf 2013;22:690696.
5 General Medical Council. National training survey 2013: key findings.
London: GMC, 2013. www.gmc-uk.org/National_training_survey_key_
findings_report_2013.pdf_52299037.pdf.
6 General Medical Council. Raising and acting on concerns about patient
safety. London: GMC, 2012. www.gmc-uk.org/guidance/ethical_guidance/
raising_concerns.asp.
7 Hogan H, Healey F, Neale G et al. Preventable deaths due to problems in
care in English acute hospitals: a retrospective case record review study.
BMJ Qual Saf 2012;21:737745.

8 IHI Global Trigger Tool for measuring adverse events. Institute for
Healthcare Improvement, 2013. www.ihi.org/knowledge/Pages/Tools/
IHIGlobalTriggerToolforMeasuringAEs.aspx.
UKmodification available at: www.institute.nhs.uk/safer_care/safer_care/
acute_adult_hospitals.html [Accessed on 18 November 2014].
9 Shojania, KG. Deaths due to medical error: jumbo jets or just small
propeller planes?. BMJ Qual Saf 2012; 21:709712.
10 Dr Foster Intelligence. Understanding HSMRs (version 9). London: Dr Foster
Intelligence, 2014. www.drfoster.com/wp-content/uploads/2014/09/
HSMR_Toolkit_Version_9_July_2014.pdf.
11 Lilford R, Pronovost P. Using hospital mortality rates to judge hospital
performance: a bad idea that just wont go away. BMJ 2010;340:c2016.
12 Flowers J, Abbas J, Ferguson B et al. Dying to know: how to interpret and
investigate hospital mortality measures. Association of Public Health
Observatories, 2010. www.apho.org.uk/resource/view.aspx?RID=95932.
13 Royal College of Physicians, 2013. Learning to make a difference. www.
rcplondon.ac.uk/projects/learning-make-difference-ltmd [Accessed on 18
November 2014].
14 
The Health Foundation. Quality improvement made simple. London:
The Health Foundation, 2013. www.health.org.uk/publications/qualityimprovement-made-simple.
15 Provost LP, Murray SK. The health care data guide: learning from data for
improvement. San Francisco: John Wiley & Sons, 2011.

Royal College of Physicians

11 St Andrews Place
Regents Park
London NW1 4LE
Tel: +44 (0)20 3075 1649
Fax: +44 (0)20 7487 5218
www.rcplondon.ac.uk

Royal College of Physicians 2015

You may copy or distribute this work, but you must give the author
credit, you may not use it for commercial purposes, and you may not
alter, transform or build upon this work.