International Journal for Quality in Health Care 2011; Volume 23, Number 3: pp. 258 268 10.

1093/intqhc/mzr019
Advance Access Publication: 30 April 2011

A brief questionnaire for assessing

patient healthcare experiences
in low-income settings
TASHONNA R. WEBSTER 1, JEANNIE MANTOPOULOS 1, ELIZABETH JACKSON 2,
HEATHER COLE-LEWIS 3, LILLIAN KIDANE 4, SOSENA KEBEDE 5, YIGEREMU ABEBE 4,
RUTH LAWSON 4 AND ELIZABETH H. BRADLEY 1
1
Section of Health Policy and Administration, Yale School of Public Health, New Haven, CT, USA, 2Yale School of Public Health,
60 College Street, PO Box 208034, New Haven, CT 06520-8034, USA, 3Section of Social and Behavioral Sciences, Yale School of Public
Health, New Haven, CT, USA, 4Clinton Health Access Initiative, Addis Ababa, Ethiopia, and 5Johns Hopkins University School of
Medicine, Department of Medicine; Department of International Health, Johns Hopkins School of Public Health, Baltimore, MD, USA
Address reprint requests to: Elizabeth H. Bradley, Yale University School of Public Health, 60 College Street, PO Box 208034, New Haven,
CT 06520-8034. Tel: 1-203-785-2937; Fax: 1-203-785-6287; E-mail: elizabeth.bradley@yale.edu

Abstract
Objective. The aim of this study was to develop and to assess the validity and reliability of two brief questionnaires for asses-
sing patient experiences with hospital and outpatient care in a low-income setting.
Design. Using literature review and data from focus groups (n 14), we developed questionnaires to assess patient experi-

Downloaded from by guest on November 14, 2015

ences with inpatient (I-PAHC) and with outpatient (O-PAHC) care in a low-income setting. Questionnaires were administered
in person by trained interviewers. Construct validity was assessed with factor analysis; convergent validity was assessed by
correlating summary scores for each scale with overall patient evaluations, and reliability was assessed with Cronbachs alpha
coefcients.
Setting. Eight health facilities in Ethiopia.
Participants. Patients .18 years old who had a hospital stay .1 day (n 230), and patients who received outpatient care
(n 486).
Main outcome measures. Patient evaluations of health care experiences.
Results. The factor analysis revealed 12 items that loaded on ve factors for the I-PAHC questionnaire. The O-PAHC
showed similar results with 13 items that loaded on four factors. Summary scores for nearly all factors were signicantly
associated (P-value , 0.05) with the patients overall evaluation score. The measure of reliability, Cronbachs alpha coefcients,
showed good to excellent internal consistency for all scales.
Conclusions. The I-PAHC on O-PAHC questionnaires can be useful in assessing patients evaluations of care delivery in
low-income settings. The questionnaires are brief and can be integrated into health systems strengthening efforts with the
support of leadership at the health facility and the country levels.
Keywords: health system strengthening, Ethiopia, quality improvement, validation, low-income country, primary care

Introduction health care delivery have been made and where to focus
future improvement efforts.
Health systems strengthening is an important international As part of the Ethiopian Hospital Management Initiative
priority for the World Health Organization (WHO) [1], the and broader health care reform efforts in Ethiopia, the
United States Agency for International Development Ethiopian Federal Ministry of Health sought to integrate
(USAID) [2] and numerous donor organizations [3]. One of ongoing measurement of patients experiences into its health
the WHOs six building blocks of health systems is the deliv- system strengthening efforts; nevertheless few studies have
ery of health services that are effective, safe and good quality been conducted to validate measures of patient experience in
for those who need them [1]. Strengthening health service low-income countries, and none exists within Ethiopia.
delivery requires special attention to the experiences of Although standardized patient surveys are widely used in
patients as it is a key indicator of whether improvements in countries such as the USA and UK, existing literature

International Journal for Quality in Health Care vol. 23 no. 3

# The Author 2011. Published by Oxford University Press in association with the International Society for Quality in Health Care;
all rights reserved 258
 Assessing patient experiences

measuring patient experience in low-income countries is translated to check the validity of the translation. The survey
limited. Many studies have used the SERVQUAL instrument was then pre-tested in one hospital in Addis Ababa with 50
[4 8], which was originally designed for the retail sector and patients purposefully sampled through a 2-week period to
has been shown to have limited convergent and construct reect different days of the week and times of day; with 10
validity [9]. Other survey instruments for assessing patient of these patients, we conducted cognitive interviews [24] to
experiences that have been validated in low-income settings identify questions that were unclear or confusing. Based on
have been designed for specic services, such as dental care these data, the survey items were modied, and nal ques-
[10], diabetes care [11], antiretroviral therapy services [12] or tionnaire were developed for elding. The nal I-PAHC and
primary care [13, 14]. We could nd no studies in low- O-PAHC questionnaires covered ve domains of care: nurse
income countries that utilized validated measures for hospital communication, doctor communication, physical environ-
care, and those that assessed patient experiences in primary ment, pain management and medication and symptom com-
care [13, 14] were developed and tested in West Africa; we munication. Items were scored using a 4-point Likert scale,
know of no published studies of an instrument that has been ranging from 1 (never) to 4 (always) in the I-PAHC survey
validated for use in Ethiopia or in East Africa. and 1(strongly disagree) to 4 (strongly agree) in the O-PAHC
Accordingly, we sought to develop and validate a brief survey. In both questionnaires, we added items asking
measurement tool for assessing patient experiences with hos- patients to provide an overall evaluation of care (scored 0
pital and outpatient care in Ethiopia. Using focus group data 10) and asking patients if they would recommend this facility
to identify potentially important concepts in patients evalu- to friends and family (on a 4-point scale from denitely no to
ation of health care and multiple revisions and stakeholder denitely yes). The nal questionnaires in their validated
pre-testing of survey items, we developed two surveys, which form are shown in Appendices 1 and 2.
were then validated using data from ve hospitals and three
health centers. Information from this study can be useful for
Sample and data collection
policy makers, clinicians and healthcare managers in low-
income settings seeking to promote patient-centered care. The validation study was conducted in ve hospitals and
three health centers, with the goal of recruiting 50 patients

Downloaded from by guest on November 14, 2015

per facility to participate. All eight health facilities were
located in urban or semi-urban areas in Addis Ababa or
Methods Amhara region of Ethiopia. We employed a non-random
quota sampling technique to recruit patients for participation
Questionnaire design
in the study to ensure a representative sample of demo-
We developed separate questionnaires for patient assessment graphic variables, such as age and sex. Data were collected
of healthcare for inpatient care (I-PAHC) and outpatient care using face-to-face interviews due to the prevalence of illiter-
(O-PAHC) in several steps. First, we conducted a thorough acy in Ethiopia. Prior to survey completion, the interviewers,
literature review to identify instruments that could be used to who were hired as data collectors and were not facility
assess patient healthcare experiences in low-income countries. employees, explained the study to potential participants and
Although we examined multiple instruments [15 19], we obtained their consent to participate. For both surveys, inter-
identied the Consumer Assessment of Healthcare Providers viewers conducted the surveys in-person on different week-
and Systems (CAHPS) questionnaires [20 23] as the most days and different times of day (i.e. morning, afternoons and
widely used and survey with the most empirical research sup- evenings) seeking to recruit 25 adult patients (cat 16 years
porting its reliability and validity. Nevertheless, given that it is old) per week with an effort to get a representative sample.
primarily used in the USA, we sought to evaluate the appro- To be eligible for the I-PAHC survey, patients had to have a
priateness of the CAHPS questions for use in Ethiopia and length of stay of at least one overnight stay. In order to be
make appropriate modications for use in Ethiopia. eligible for O-PAHC, the patient had to have received care at
Therefore, as a second step, we conducted 14 focus groups the health facility on the day the survey was conducted. The
in geographically diverse areas of Ethiopia to learn about survey was conducted at the time of discharge, after the
aspects of care that were most salient to individuals in this patient had been treated either as an inpatient or an outpati-
context. Focus groups included 8 10 people and were ent. A total of 50 patients were recruited from each facility
homogeneous in gender with seven male and seven female to ensure that there was sufcient power to detect a 1-point
groups. Based on focus group data, we identied additional change in the overall 10-point patient evaluation rating. The
potential domains that were viewed by individuals as impor- research procedures were approved by the institutional review
tant to their healthcare experience. Third, based on interviews board at the Yale University School of Medicine.
with stakeholders in the Ministry of Health, physicians and
hospital administrators in Ethiopia, we modied some items,
Data analysis
eliminated items that were not relevant (e.g. responsiveness of
nurse to call buttons, which are not used in Ethiopia) and We assessed the reliability as well as construct and convergent
added items that were expected to be important in the setting validity of the questionnaire. To evaluate reliability, we used a
of Ethiopia (e.g. ease of nding way around the facility). measure of internal consistency, Cronbachs alpha coefcient,
Fourth, the survey was translated into Amharic and back- with coefcients of 0.70 or higher interpreted as indicating

259
Webster et al.

good to excellent internal consistency, as recommended by Table 1 Demographic characteristics of patients

experts [25, 26]. Construct validity assesses the degree to
which items that are conceptually related (i.e. are meant to Characteristics I-PAHC O-PAHC
measure the same construct) are empirically associated as one (n 230) (n 486)
would expect if the measurement is valid. We assessed con- n (%) n (%)
struct validity of each questionnaire using factor analysis. ....................................................................................
Based on existing literature and focus groups, we anticipated Age
specic items would load on specic factors, anticipating a ,18 years 7 (5) 48 (15)
total of 5 factors in each questionnaire. A factor is the latent 18 34 years 9 (6) 11 (4)
concept that the questionnaire seeks to measure empirically. 35 64 years 110 (76) 219 (69)
In order to avoid correlations between factors and to deter- .64 years 19 (13) 9 (12)
mine distinct domains, an orthogonal rotation was used. Mean (SD) 36.80 (15.89) 38.16 (15.23)
Consistent with the current literature using factor analyses Gender
[27], factor loadings 0.40 indicated that the survey items Male 104 (48) 211 (49)
were correlated with the common factors that were deter- Female 113 (52) 223 (51)
mined a priori. Owing to structured missing data in the data Education
set (i.e. skip pattern questions), we ran two sets of factor ana- Illiterate 46 (21) 75 (16)
lyses for both PAHC questionnaires, one with and without Reading and writing ability 16 (7) 39 (8)
skip pattern questions. The statistical procedure used deletes 1 8th grade 60 (27) 110 (24)
the respondents with missing data on any included items, 9 12th grade 60 (27) 161 (34)
which results in the analyses being limited to only the respon- Diploma and above 33 (15) 75 (16)
dents who answered all of the questions, including those that Other 6 (3) 7 (2)
would have been appropriately skipped by respondents who, Self-reported overall health status
for instance, did not have pain or did not have medications. Poor 44 (20) 125 (28)
Therefore, we conducted the factor analysis in two ways: both Fair 81 (37) 215 (49)

Downloaded from by guest on November 14, 2015

with and without the items related to the skip patterns, in Good 63 (28) 73 (18)
order to ensure we used of all available data. We did not Excellent 34 (15) 22 (5)
impute missing data because the vast majority of missing data
were missing due to skip patterns. A total of n 146 of 230 Totals do not sum to sample size due to missing data for some
respondents for I-PAHC and n 347 of 486 respondents for demographic characteristics of participants, especially age, which
O-PAHC had missing data due to questionnaire skip patterns was frequently unknown.
(i.e. these respondent had not experienced pain or had not
been prescribed medication). Other missing data comprised
Table 2 Hospital characteristics
,10% of the samples and hence were not imputed [28]. We
created summary scores by summing item responses within
each construct, creating a scale. These summary scores were n (%)
....................................................................................
used to assess convergent validity by examining the statistical
Governance type
Pearson correlation of the summary scores with responses to
Ministry of Health 1 (13)
the overall patient evaluation item.
Ministry of Education 1 (13)
Addis Ababa City Health Bureau 6 (74)
Number of beds
Results ,100 beds 4 (50)
100 299 beds 3 (37)
Demographic characteristics of study sample .299 beds 1 (13)
Teaching status
A total of 230 of 242 patients approached participated in the
Non-teaching 4 (50)
inpatient survey (response rate of 95%) (See Appendix 1 for
Teaching 1 (13)
I-PAHC instrument); 486 of 512 patients approached partici-
Not applicable 3 (37)
pated in the outpatient survey (response rate of 95%) (See
Appendix 2 for O-PAHC instrument) (Table 1). In the
I-PAHC sample, the mean age of the participants was 37 percent of the outpatient sample was female. Sixteen percent
years, with a range of 16 to 88 years. About half (52%) of of the patients reported themselves as illiterate, and another
the participants were female. Twenty-one percent of patients sixteen percent reported having a diploma or more edu-
reported themselves as illiterate, and 18% reported that they cation. The majority of respondents (77%) reported their
had a diploma or more education. Many patients (57%) health as fair or poor.
reported their health as fair or poor. All health facilities recruited a minimum of 50 patients to
In the O-PAHC sample, the mean age of the participants participate in each survey, except for one hospital which only
was 38 years, with a range of 16 to 83 years. Fifty-one recruited only 27 patients for the I-PAHC survey due to

260
 Assessing patient experiences

Table 3 Factor analysis including skip pattern questions

Factors
..................................................................................................................
Communication Communication Physical Pain Medication
with nurses with doctors environment management communication
Factor 1 Factor 2 Factor 3 Factor 4 Factor 5
.............................................................................................................................................................................

I-PAHC items (n 79)

Q1 Nurses treat with courtesy and 0.75 0.12 0.02 0.48 0.06
respect
Q2 Nurses listen carefully 0.59 0.37 0.18 0.31 0.10
Q3 Nurses explain things in an 0.74 0.30 0.14 0.11 0.12
understandable way
Q4 Doctors treat with courtesy and 0.22 0.67 20.11 0.21 20.10
respect
Q5 Doctors listen carefully 0.09 0.86 0.06 0.11 0.00
Q6 Doctors explain things in an 0.25 0.66 0.17 0.04 0.07
understandable way
Q8 Hospital room was kept clean 0.05 0.13 0.58 0.18 0.00
Q9 Surrounding area was kept quiet 0.09 20.03 0.60 0.02 20.03
Q12 Pain was well controlled 0.25 0.11 0.18 0.86 20.01
Q13 Staff did everything they could to 0.22 0.20 0.10 0.80 0.01
help with pain
Q15 Staff explained what medication 0.08 0.05 20.26 0.20 0.82
was for

Downloaded from by guest on November 14, 2015

Q16 Staff explained possible medication 0.08 20.04 0.14 20.14 0.70
side effects
.............................................................................................................................................................................
Factors
..........................................................................................
Communication Communication Physical Medication
with nurses with doctors environment communication
Factor 1 Factor 2 Factor 3 Factor 4
.............................................................................................................................................................................
O-PAHC items (n 163)
Q1 Nurses treat with courtesy and 0.83 0.34 0.26 20.12
respect
Q2 Nurses listen carefully 0.90 0.30 0.21 20.13
Q3 Nurses explain things in an 0.75 0.32 0.11 20.19
understandable way
Q4 Doctors treat with courtesy and 0.41 0.77 0.28 20.18
respect
Q5 Doctors listen carefully 0.36 0.85 0.25 20.18
Q6 Doctors explain things in an 0.34 0.73 0.21 20.21
understandable way
Q8 Outpatient department was clean 0.16 0.22 0.82 0.07
Q9 Bathrooms/latrines were clean 0.13 0.12 0.43 20.11
Q10 Enough time to discuss medical 0.39 0.34 0.25 20.54
problem the doctor/health ofcer
or nurse
Q11 Given information in an 20.22 20.21 0.06 0.71
understandable way regarding symptoms
or health problems
Q13 Staff explained what medication 20.08 20.10 20.10 0.90
was for
Q14 Staff explained possible medication 20.03 20.05 20.04 0.70
side effects

Note: The table displays the factor loadings for an orthogonal rotation, i.e. varimax, using ve factors for I-PAHC and four factors for
O-PAHC.

261
Webster et al.

Table 4 Items factor analysis without skip pattern questions

Factors
....................................................................
Communication Communication Physical
with nurses with doctors environment
Factor 1 Factor 2 Factor 3
.............................................................................................................................................................................

I-PAHC items (n 199)

Q1 Nurses treat with courtesy and respect 0.71 0.19 0.29
Q2 Nurses listen carefully 0.80 0.31 0.15
Q3 Nurses explain things in an understandable way 0.65 0.40 0.15
Q4 Doctors treat with courtesy and respect 0.36 0.66 0.27
Q5 Doctors listen carefully 0.24 0.80 0.17
Q6 Doctors explain things in an understandable way 0.25 0.74 0.19
Q8 Hospital room was kept clean 0.10 0.17 0.73
Q9 Surrounding area was kept quiet 0.23 0.15 0.47
.............................................................................................................................................................................
Factors
.............................................................................................
Communication Communication Physical Health
with nurses with doctors environment communication
Factor 1 Factor 2 Factor 3 Factor 4
.............................................................................................................................................................................
O-PAHC items (n 264)
Q1 Nurses treat with courtesy and respect 0.81 0.30 0.26 20.17

Downloaded from by guest on November 14, 2015

Q2 Nurses listen carefully 0.81 0.39 0.21 20.17
Q3 Nurses explain things in an understandable way 0.72 0.31 0.18 20.34
Q4 Doctors treat with courtesy and respect 0.36 0.81 0.20 20.20
Q5 Doctors listen carefully 0.31 0.85 0.23 20.20
Q6 Doctors explain things in an understandable way 0.36 0.67 0.21 20.35
Q8 Outpatient department was clean 0.21 0.32 0.63 0.08
Q9 Bathrooms/latrines were clean 0.13 0.07 0.61 20.16
Q10 Enough time to discuss medical problem the 0.36 0.28 0.34 20.55
doctor/health ofcer or nurse
Q11 Given information in an understandable way 20.15 20.16 20.01 0.69
regarding symptoms or health problems

Note: The table displays the factor loadings for an orthogonal rotation, i.e. varimax, using three factors for I-PAHC and four factors for
O-PAHC.

poor response rate (Table 2). Half of the health facilities had that factor for all subsequent analyses. Three items regarding
fewer than 100 beds and were non-teaching. privacy (item 10), symptom recognition (item 17) and ease of
nding way around the hospital (item 18) did not load on
any of the ve factors, and, therefore, were excluded from
Factor analysis further analysis.
The factor analysis revealed 12 items in the I-PAHC that The factor analysis for the O-PAHC survey generated
loaded on 1 of 5 factors (Table 3). The ve-factor model similar results, with 13 items loading on four distinct factors
generally corresponded with the constructs that were antici- (Table 3). A ve factor model was determined a priori;
pated based on the literature and a priori hypotheses. Each of however, items 10 and 11 loaded on the same factor as items
the items had a loading of 0.40 or greater on only 1 of the 13 and 14. Items 10 and 11 were expected to measure health
factors, except for the item that measured nurses courtesy communication while items 13 and 14 were expected to
and respect, which loaded on both the communication with measure medication communication, suggesting much of the
nurses factor (factor 1, loading 0.75) and the pain man- communication inuencing patients experiences may be
agement factor (factor 4, loading 0.48). This is likely as about medications. Given that items 10 and 11 were thought
nurses are instrumental in pain management for patients. to be measuring a different domain conceptually, these two
Because the item had a higher loading on the communi- items were dropped from the factor summary scores. Three
cation with nurses factor, the item remained grouped under items regarding the ability to distinguish between doctors/

262
 Assessing patient experiences

Table 5 I-PAHC ad O-PAHC reliability estimates for multi-item scales and items

I-PAHC scale/item n Reliability (Cronbach a)

.............................................................................................................................................................................

Communication with nurses 225 0.85

Q1: Nurses treat with courtesy and respect
Q2: Nurses listen carefully
Q3: Nurses explain things in an understandable way
Communication with doctors 214 0.86
Q4: Doctors treat with courtesy and respect
Q5: Doctors listen carefully
Q6: Doctors explain things in an understandable way
Physical environment 213 0.54
Q8: Hospital room was kept clean
Q9: Surrounding area was kept quiet
Pain management 136 0.88
Q12: Pain was well controlled
Q13: Staff did everything they could to help with pain
Medication communication 116 0.70
Q15: Staff explained what medication was for
Q16: Staff explained possible medication side effects
O-PAHC scale/item
Communication with nurses 460 0.92
Q1: Nurses treat with courtesy and respect
Q2: Nurses listen carefully

Downloaded from by guest on November 14, 2015

Q3: Nurses explain things in an understandable way
communication with doctors 440 0.92
Q4: Doctors treat with courtesy and respect
Q5: Doctors listen carefully
Q6: Doctors explain things in an understandable way
Physical environment 300 0.59
Q8: Outpatient department was clean
Q9: Bathrooms/latrines were clean
Medication communication 283 0.77
Q13: Staff explained what medication was for
Q14: Staff explained possible medication side effects

health ofcers and nurses (item 7), availability of medication Reliability

in the dispensary (item 15) and ease of nding way around
Cronbachs alpha coefcients for all the scales for the
the health facility (item16) were excluded from further ana-
I-PAHC and O-PAHC surveys exceeded 0.70 (except for
lyses due to loadings ,0.40.
physical environment scale, which had a Cronbachs alpha
As an additional analysis, we re-ran the factor analyses
coefcient of 0.54 in I-PAHC and 0.59 in O-PAHC),
excluding the skip pattern questions (Table 4), which existed
suggesting good to excellent reliability for the scales pertain-
in both surveys. As expected, this resulted in fewer factors
ing to communication with nurses, communication with
(as skip questions pertained to medication use and pain man-
doctors, pain management and medication communication
agement and were eliminated from this additional analysis).
factors for both I-PAHC and O-PAHC surveys (Table 5).
The factor analysis for the I-PAHC survey revealed three
factors. All of the items loaded on the same factors as the
previous analysis, except for one item. The item nurses
explain things in an understandable way loaded on two Convergent validity
factors, communication with nurses (factor 1) and communi- Nearly all associations between each of the summary scores
cation with doctors (factor 2), reecting the correlation of and the item assessing patients overall evaluations of the
patient views concerning physicians and nurses interactions health care experience for both I-PAHC and O-PAHC scales
with patients. Excluding the skip pattern questions from the were statistically signicant (Table 6). For instance, strong
O-PAHC factor analysis did not substantially change the and statistically signicant associations were apparent with
factor loadings (Table 4). the communication with nurses scale for I-PAHC (rho

263
Webster et al.

Table 6 Correlations of scales and items with patients medication communication scale summary score for the
overall evaluation I-PAHC survey, however, did not correlate very highly
(rho 20.04, P-value 0.71) with the overall evaluation
I-PAHC scale/item (n 76) Overall rating. All of the correlations between the summary scores
evaluation for scales and patients overall evaluation for the O-PAHC
.................................................................................... survey were .0.40 (P-values , 0.05), except for the corre-
Communication with nurses 0.46* lation between the summary score for the physical environ-
Q1 Nurses treat with courtesy and respect 0.39* ment scale and patients overall evaluation (r 0.23, P ,
Q2 Nurses listen carefully 0.40* 0.05).
Q3 Nurses explain things in an 0.40*
understandable way
Communication with doctors 0.40*
Q4 Doctors treat with courtesy and respect 0.36* Discussion
Q5 Doctors listen carefully 0.37*
Q6 Doctors explain things in an 0.29* This study reports on the development and validation of
understandable way questionnaires to assess patients experience in health in a
Physical environment 0.32* low-income country, Ethiopia. The I-PAHC questionnaire
Q8 Hospital room was kept clean 0.35* includes 25 questions that comprise 5 constructs, and the
Q9 Surrounding area was kept quiet 0.17 O-PAHC questionnaire includes 23 questions that comprise
Pain management 0.36* 4 constructs. The results indicate that the scales in both
Q12 Pain was well controlled 0.34* questionnaires have good to excellent reliability, and both
Q13 Staff did everything they could to help 0.33* questionnaires have good construct validity. Together, these
with pain results indicate that the items that were conceptually related
Medication communication 20.04 were also highly correlated statistically, providing evidence
Q15 Staff explained what medication was 20.12 that they were measured by the instruments appropriately.

Downloaded from by guest on November 14, 2015

for Additionally, we found the survey adminstration to be feas-
Q16 Staff explained possible medication 0.06 ible and useful in detecting differences across hospitals.
side effects Patient experiences are frequently assessed as part of
O-PAHC scale/item (n 166) quality improvement efforts in high-income settings; however,
Communication with nurses 0.51* few validated questionnaires are available for use in low-
Q1 Nurses treat with courtesy and respect 0.47* income settings. The SERVQUAL, although used in other
Q2 Nurses listen carefully 0.48* studies, is not designed for health care facilities and therefore
Q3 Nurses explain things in an 0.47* may miss important contextual aspects of patient experiences,
understandable way such as experiences with physicians and nurses, particularly in
Communication with doctors 0.56* low-income settings. Additionally, the SERVQUAL assesses
Q4 Doctors treat with courtesy and respect 0.56* both individuals expectations and perceptions of actual per-
Q5 Doctors listen carefully 0.52* formance, which are measured at the same time and can intro-
Q6 Doctors explain things in an 0.53* duce bias, as well as result in a substantially longer survey
understandable way instrument and increased respondent burden. Some of the
Physical environment 0.23* other surveys that have been assessed focus on specic types
Q8 Outpatient department was clean 0.23* of care, such as diabetes [11] or HIV treatment [12] and there-
Q9 Bathrooms/latrines were clean 0.16* fore are less applicable to general hospital and outpatient care.
Medication communication 20.40* One additional survey, designed for use in primary outpatient
Q13 Staff explained what medication was 20.44* care rather than in hospitals, has been shown to be valid for
for populations in Upper Guinea and in Burkina Faso [13, 14],
Q14 Staff explained possible medication 20.30* but has not been validated in East Africa. Another survey was
side effects developed for assessing patient satisfaction with hospital care
in Bangladesh [15]; however, the survey was specic to the
*P-value , 0.05. Bangladesh context, again not necessarily reecting the health
care system issues in East Africa. The I-PAHC and O-PAHC,
which are brief, valid and reliable questionnaires, may be par-
0.46, P , 0.05) and with the communication with doctors ticularly helpful in evaluating the impact of health systems
scale for O-PAHC (rho 0.56, P , 0.05). Summary scores strengthening efforts on patients experiences with facility-
for the communication with doctors scale, physical environ- based care. These questionnaires will allow facilities to
ment scale and pain management scale showed moderate measure patient experiences at one point in time as well as
and statistically signicant associations with the overall evalu- monitor trends over time. Ongoing monitoring of such data
ation rating (rho coefcients 0.40, 0.32 and 0.36, respect- can be used at the national level to set benchmarks for hospi-
ively, P-values , 0.05) for the I-PAHC survey. The tals and health centers.

264
 Assessing patient experiences

Despite the benets of measuring patient experience, The international focus on strengthening health systems
implementation of the surveys for routine use requires includes expanded efforts in quality improvement of delivery
focused efforts and human resource capacity. First, inter- systems. Measuring impact of such efforts is challenging, par-
viewers must be identied, trained and supervised. Although ticularly in resource-limited settings where medical records
we anticipated some patients may complete the surveys are often limited and data capture systems can be unreliable.
themselves, we learned that due to illiteracy rates, physical In such contexts, the patients experience can be an impor-
disabilities and consistency of completion, in-person admin- tant indicator for comparing facilities and for evaluating
stration by a trained interviewer was most effective for gath- efforts to enhance patient-centered, higher quality care. The
ering high quality and consistently completed questionnaires. PAHC questionnaires are brief and can easily be adminis-
Nonetheless, training was important, particularly in the tered in health care settings with the support of leadership at
sampling and consent procedures. Training of interviewers the health facility level as well as the country level.
was accomplished in a one-day session. In addition, data Leadership support has been apparent in Ethiopia as the
were compiled on site and entered using a pre-formatted MS Federal Ministry of Health has endorsed the use of the
Access programme, which then exported data to Excel to PAHC questionnaires in its national reform guidelines, and
produce automatically summary tables and gures to track hospital managers are beginning to use the questionnaire reg-
hospital performance. Completion of the interview and its ularly. The presence of effective leadership at both ministry
data entry was estimated to require an average of 15 min per and facility levels is critical for facilitating the implementation
questionnaire for a total staff time of just more than 12 h for process as well as providing the resources needed for
50 questionnaires in the month of the survey. ongoing data collection, analysis and action to enhance the
The results of this study must be viewed in light of several quality of care and improve patients experiences.
limitations. First, the sample size is relatively small, although
we had adequate power to detect the expected statistical
associations. Second, although we tested internal consistency
reliability, we were unable to assess the questionnaires test
Acknowledgments
retest reliability due to the logistical challenges of having

Downloaded from by guest on November 14, 2015

We are grateful to Heather Sipsma, PhD, Jeph Herrin, PhD
patients complete the survey twice. Third, the physical
and Paul Cleary, PhD for providing statistical guidance and
environment construct had relatively lower internal consist-
reviews of earlier drafts.
ency reliability than the other scales in the instruments,
which would suggest that the two items do not fully capture
the physical environment construct. More items in this con-
struct may be needed to improve internal consistency Funding
reliability of the physical environment scale. Finally, we con-
ducted the study in one low-income country; results may This work was supported by a grant from the Clinton Health
differ in other low-income countries and evaluating question- Access Initiative and the Centers for Disease Control
naires in the context where they will be used is important. (200-2010-3g226).

265
Webster et al.

Appendix 1: In-Patient Assessment of Health Care (I-PAHC) Survey

Downloaded from by guest on November 14, 2015

266
 Assessing patient experiences

Appendix 2: Out-Patient Assessment of Health Care (O-PAHC) Survey

Downloaded from by guest on November 14, 2015

267
Webster et al.

References 14. Haddad S, Fournier P, Potvin L. Measuring lay peoples percep-

tions of the quality of primary health care services in develop-
1. WHO. Everybodys Business: Strengthening Health Systems to Improve ing countries. Validation of a 20-item scale. Int J Qual Health
Health Outcomes. Geneva, Switzerland: World Health Care 1998;10:93 104.
Organization, 2007. 15. Andaleeb SS. Service quality perceptions and patient satisfac-
2. USAID. Sustaining Health GainsBuilding Health Systems: Health tion: a study of hospitals in a developing country. Soc Sci Med
Systems Report to Congress. Washington, DC: United States 2001;52:1359 70.
Agency for International Development, 2009. 16. Bernhart MH, Wiadnyana IG, Wihardjo H et al. Patient satisfac-
3. World Bank. Healthy Development: The World Bank Strategy for tion in developing countries. Soc Sci Med 1999;48:989 96.
Health, Nutrition, and Population Results. Washington, DC: World 17. Carr-Hill RA. The measurement of patient satisfaction. J Public
Bank, 2007 Contract No. May 18. Health Med 1992;14:236 49.
4. Parasuraman A, Berry L, Zeithaml V. Renement and reassess- 18. Fitzpatrick R. Surveys of patients satisfaction: IImportant
ment of the SERVQUAL scale. J Retailing 1991;67:420 50. general considerations. BMJ 1991;302:887 9.
5. Parasuraman A, Zeithaml V, Berry L. SERVQUAL: A 19. Sitzia J, Wood N. Patient satisfaction: a review of issues and
multiple-item scale for measuring consumer perceptions of concepts. Soc Sci Med 1997;45:1829 43.
service quality. J Retailing 1988;64:5 6.
20. Goldstein E, Farquhar M, Crofton C et al. Measuring hospital
6. Anbori A, Ghani SN, Yadav H et al. Patient satisfaction and care from the patients perspective: an overview of the CAHPS
loyalty to the private hospitals in Sanaa, Yemen. Int J Qual Hospital Survey development process. Health Serv Res 2005;40(6
Health Care 2010;22:310 315. Pt 2):1977 95.
7. John J, Yatim FM, Mani SA. Measuring service quality of 21. Hargraves JL, Hays RD, Cleary PD. Psychometric properties of
public dental health care facilities in Kelantan, Malaysia. Asia the Consumer Assessment of Health Plans Study (CAHPS) 2.0
Pac J Public Health 2010, doi:10.1177/1010539509357341. adult core survey. Health Serv Res 2003;38(6 Pt 1):1509 27.
8. Kerssens JJ, Groenewegen PP, Sixma HJ et al. Comparison of 22. OMalley AJ, Zaslavsky AM, Hays RD et al. Exploratory factor

Downloaded from by guest on November 14, 2015

patient evaluations of health care quality in relation to WHO analyses of the CAHPS Hospital Pilot Survey responses across
measures of achievement in 12 European countries. Bull World and within medical, surgical, and obstetric services. Health Serv
Health Organ 2004;82:106 14. Res 2005;40(6 Pt 2):2078 95.
9. Babakus E, Boller G. An empirical assessment of the 23. Solomon LS, Hays RD, Zaslavsky AM et al. Psychometric prop-
SERVQUAL scale. J Bus Res 1992;24:25368. erties of a group-level Consumer Assessment of Health Plans
10. White JG, Slabber J, Schreuder A. Patient management: Study (CAHPS) instrument. Med Care 2005;43:5360.
measuring patients expectations and perceptions of service 24. Krause N. A comprehensive strategy for developing
quality in a dental training hospital. SADJ 2001;56:2038. closed-ended survey items for use in studies of older adults. J
11. Westaway MS, Rheeder P, Van Zyl DG et al. Interpersonal and Gerontol B Psychol Sci Soc Sci 2002;57:S263 74.
organizational dimensions of patient satisfaction: the moderating 25. Nunnally J. Psychometric Theory. New York: McGraw-Hill, 1978.
effects of health status. Int J Qual Health Care 2003;15:33744.
26. Landis JR, Koch GG. The measurement of observer agreement
12. Wouters E, Heunis C, van Rensburg D et al. Patient satisfaction for categorical data. Biometrics 1977;33:159 74.
with antiretroviral services at primary health-care facilities in the
Free State, South Africaa two-year study using four waves of 27. Floyd F, Widaman K. Factor analysis in the development and
cross-sectional data. BMC Health Serv Res 2008;8:210. renement of clinical assessment instruments. Psychol Assessment
1995;7:286 99.
13. Baltussen RM, Ye Y, Haddad S et al. Perceived quality of care
of primary health care services in Burkina Faso. Health Policy 28. Little R, Rubin D. Statistical Analysis with Missing Data.
Plan 2002;17:42 8. New York: John Wiley, 1987.

268