A

NATIONAL
RESOURCECENTER
GUIDE FOR
FOREARLY
HEARING
HEARING
ASSESSMENT
DETECTION& &MANAGEMENT
INTERVENTION

Chapter 37
Quality Assurance
& Improvement
Terry E. Foust, AuD

Many states, hospitals,

and programs are not
even aware of what
Q uality assurance refers to the
systematic monitoring and
evaluation of the various
aspects of a project, service, or facility
strategy, and the quality of healthcare. They
found that fewer than half of the boards
rated quality of care as one of their two top
priorities, and only a minority reported
to ensure that high standards of quality receiving specific training in quality. While
they do well and what are being met. A more direct and concise fewer than half did not name quality of care,
they do not . . . Program way of defining quality assurance and a majority of those responding said they
evaluation is critical to improvement is to simply say it is believed that the care at their hospital was
improving effectiveness, “tracking outcomes and adjusting care above average. Even among those hospitals
cost efficiency, and or practice.” It is essential that programs that Medicare data suggest are among the
overall sustainability of make rigorous efforts to analyze and worst in the country, 58% said they thought
improve practices for improved patient their hospital was above average, and not a
an EHDI program. care as well as for cost efficiency and single one rated their hospital below average.
sustainability. It is important to recognize The message from this survey is that many
that while most organizations are trying to of us are not aware of what we do well and
systematically improve value and quality, what we do not.
those that succeed have worked out the
operational system and culture to do it. Program evaluation is critical to improving
effectiveness, cost efficiency, and overall
Even though quality assurance and sustainability of an Early Hearing Detection
improvement concepts have been around and Intervention (EHDI) program. Outcomes
for some time, many states, hospitals, and are used to determine and improve clinical
programs are not fully aware of what they care, cost effectiveness, and sustainability.
do well and what they do not. For example, Rigorous program evaluation goals include:
a study published in the Journal of Health
Affairs surveyed the chairmen of more than • Improving how we meet the needs of
700 hospitals. The physicians who conducted the families we serve.
the survey (Jha & Epstein, 2009) asked the • Measurement and improvement of
chairmen to name their board’s two top performance.
priorities from a list of issues, including • Developing evidence-based approach/
financial performance, organizational strategies.

eBook Chapter 37 • Quality Assurance & Improvement • 37-1

 A RESOURCE GUIDE FOR EARLY HEARING DETECTION & INTERVENTION

• Understanding disparities.
• Allocation of resources.
State EHDI Program
• Motivating stakeholders. Evaluation
• Contributing to literature.
State program evaluation must focus on
All U.S. states and territories with each of the key elements of an effective
Maternal and Child Health Bureau EHDI program. There are seven major
(MCHB) grants or Centers for Disease key components and/or standards of a
Control and Prevention (CDC) cooperative successful EHDI program described by
agreements are required to do systematic MCHB, CDC, and the Joint Committee on
annual evaluations of their projects to Infant Hearing (JCIH). These components
implement, expand, and/or improve their provide the minimum standard elements
EHDI program. upon which evaluation of EHDI programs
should focus (see Table 1).
Quality improvement and evaluation of
Quality improvement EHDI programs should be applied at each As emphasized, the last key component
and evaluation of EHDI level of the program, including: specifically identifies evaluation for quality
programs should be assurance and improvement. All state
applied at each level of • The overall state EHDI program. programs receiving MCHB EHDI grants
• Individual hospital and birthing must have a plan for program evaluation
the program. center programs. and must complete a comprehensive
• Home births. evaluation of their entire EHDI program.
• Diagnostic and intervention programs
(pediatric audiologists, amplification, A common approach to conducting a
cochlear implants). statewide EHDI evaluation includes
• Tracking and surveillance programs. collecting data from each of the

Table 1
Key Components and/or Standards
of a Successful EHDI Program

1 All newborns will be screened for hearing loss before 1 month of age.

2 All infants referred from screening will have diagnostic evaluations before 3
months of age.

3 All infants identified with hearing loss will receive appropriate medical, audio-
logic, and educational intervention services before 6 months of age.

4 All infants with hearing loss will have a medical home.

5 Every state will have a complete EHDI Tracking and Surveillance System to
minimize loss to follow-up.

6 All families will receive culturally competent family support.

7 Every state will do regular systematic monitoring and evaluation to improve

the effectiveness of the EHDI program.

eBook Chapter 37 • Quality Assurance & Improvement • 37-2

 NATIONAL CENTER FOR HEARING ASSESSMENT & MANAGEMENT

program stakeholders, using data from advisory committee, and staff from
It is important to questionnaires, analysis of archival interested hospitals, etc. Typical reports
compare program information, and onsite evaluations include:
outcomes to other (see Tables 2 and 3). The overall goal is
to collect information about how well • Annual hospital report cards.
hospitals, programs, and hospitals and the state system are doing • Quarterly data reports to birth facilities
states to learn what is in achieving the seven objectives listed in and diagnostic centers with special
working well and what Table 1. emphasis on transferred infants.
can be improved. • Verification of screening results for
Data from all of the sources listed in infants listed as passed and later
Table 3 are summarized by the evaluation identified with hearing loss.
team, analyzed according to the six goals
previously described, and compiled It is important to compare program
into a report. The report can then be outcomes to other hospitals, programs,
discussed and used to identify program and states to learn what is working well
improvement activities at all levels, such as and what can be improved. It is essential
staff from the state department of health, that success documented by outcome data
the state’s newborn hearing screening be replicated (see Table 4).

Table 2
Examples of EHDI Data That Should Be
Evaluated at a Minimum
Screening Data

• Total number of live births.

• Number of babies screened.
• Missed screens, including homebirths and transferred infants.
• Number of babies not screened due to nonconsent.
• Number of babies passing screening or rescreening prior to discharge.
• Number of babies discharged not passing screening in one or both ears.
• Number of babies passing outpatient screening.
• Number of babies not passing outpatient screening in one or both ears.

Diagnostic Data

• Number of babies lost to follow-up (analyzed geographically).

• Number of infants diagnosed with hearing loss.
• Number of infants with confirmed hearing loss by type and degree.
• Number of infants diagnosed with late onset hearing loss through coordination with
early childhood screening programs, such as Early Head Start.
• Number of babies with risk indicators (number and percentage).
• Median age at diagnosis.

Early Intervention

• Number of infants enrolled in early intervention.

• Number of infants lost to follow-up.

eBook Chapter 37 • Quality Assurance & Improvement • 37-3

 A RESOURCE GUIDE FOR EARLY HEARING DETECTION & INTERVENTION

Table 3
EHDI Evaluation Common Elements
Hospital Analysis Questionnaires
Questionnaires of Tracking Data to Parents

Each hospital in the state is asked to Hospitals in most states use an A random sample of parents whose
complete a questionnaire about their information management system babies were screened during a defined
newborn hearing screening program. (IMS) to submit data to the state time period are sent a questionnaire
Questions are asked about: department of health regarding: regarding:
• The hospital’s screening protocol. • Number of babies born. • Their perceptions about the
• Equipment and practice. • Number of babies screened. screening program.
• Follow-up. • Results of screening. • What they saw as the strengths
• Referral for diagnosis. • Follow-up activities. and weaknesses.
• Satisfaction with the program. • Information about referral and • Suggestions for improving the
• Obstacles that have been diagnosis. program.
encountered.
• Areas in which assistance is An external evaluation team analyzes The sample of parents should include
needed. this data to identify strengths and those whose baby:
• Satisfaction with previous weaknesses of the program. The • Passed the inpatient screening,.
assistance. analysis of tracking data is useful in • Did not pass the inpatient
• Support provided by the state pinpointing areas where additional screening but passed an
department of health. technical assistance and training outpatient screening.
are needed. For example, when • Did not pass either the inpatient
Prior to distribution, drafts of the inpatient refer rates are too high, or outpatient screening.
questionnaire should be reviewed it is frequently because too many
by members of the state’s newborn people are screening babies, or one Respondents are assured of
hearing screening advisory committee or two screeners are having difficulty. anonymity, so they can be open about
for suggested revisions. Although this data is available to any concerns they may have, as well
hospitals, they often do not appreciate as identifying aspects of the program
its value for program improvement. that are working well. The results
Thus, it is helpful to have an external of these questionnaires are used
evaluation team review the state data to determine whether appropriate
to identify areas where assistance is information and support are being
needed. provided to parents.

Physician Onsite Visits

Questionnaires to Hospitals

A random sample of physicians listed A small sample of hospitals are • Observation of screening
in the IMS can also be surveyed. selected for onsite visits by a team of activities.
Physicians listed as the primary evaluators who are experienced with • Review of records and program
healthcare provider for babies who newborn hearing screening programs. policies.
are screened are sent a questionnaire The site visit should include: • Examination of materials for
assessing their attitude and knowledge • Interviews with program staff parents.
about the universal newborn hearing (e.g., screeners, screening
screening program. They are asked to program manager, nursery
identify strengths, weaknesses, and coordinator, director of women’s
suggestions for program improvement. services).

eBook Chapter 37 • Quality Assurance & Improvement • 37-4

 NATIONAL CENTER FOR HEARING ASSESSMENT & MANAGEMENT

Table 4
Program Evaluation Tools
National Center for Hearing Assessment and Management (NCHAM)

Examples of questionnaires, surveys, The specific URL for the NCHAM • Audiologist/diagnostic center
and checklists that have been used by Evaluation Tools is: http://www. surveys.
others can be found on the NCHAM infanthearing.org/programevaluation/ • Early intervention program
website under Evaluation in the evaluationtools.html activities surveys.
EHDI Components section. This • Parent and family reactions to
information is available to any EHDI Under this category, you will EHDI program activities surveys.
program at no charge. Programs find: • General EHDI surveys.
are free to use the materials, tools, • NCHAM statewide evaluation • Examples of evaluations already
and samples and to adapt them tools. conducted by states.
to individual program needs and • Hospital surveys and evaluation • Downloadable copies of
circumstances. tools. evaluation instruments.

Maternal and Child Health Bureau (MCHB) and Centers for Disease Control and Prevention (CDC)

States receiving MCHB funding for quality assurance evaluations. The audiologists, parents, the state
their EHDI programs must have a publication is available at www.cdc. department of health, and EHDI
formal plan for evaluation written gov/eval/framework.htm. Along with program officials.
into their grant. Funds can be used other valuable information, it gives
to pay an outside party to conduct an detailed suggestions about six iterative Step 2. Describe the program.
evaluation. steps and four standards that govern Provide a detailed, in-depth description
effective program evaluation, as of the program, including the need,
Federal grantees are also subject to depicted below. expected effects, activities, resources,
a formal review process through the stage, content, logic model, etc.
Office of Performance Review. This
formal review process occurs over Step 3. Focus the evaluation design.
several months via conference calls Address the purpose of the
with the grantee and key stakeholders. evaluation, users, uses, questions,
These calls focus on identification of methods, and agreements.
success and challenges pertaining to
the goals written into the federal grant Step 4. Gather credible evidence.
and culminate with a 2-day site visit Look at key indicators, sources,
with the grantee and key stakeholders. quality, quantity, and logistics.
A plan is formulated to address
challenges. If necessary, expert Step 5. Justify conclusions.
support and assistance can be funded Look at standards, analysis and
and provided to grantees. synthesis of data, interpretation,
Step 1. Engage program stakeholders. judgment, and recommendations.
The CDC has an excellent publication This includes all those involved,
entitled, “Framework for Program affected, and all primary users. For Step 6. Ensure use and share lesions
Evaluation in Public Health,” which example, in an EHDI program, this learned.
can be used to design, conduct, includes hospitals, pediatricians Focus on feedback, follow-up, and
and report results of program and primary care physicians, dissemination of information.

eBook Chapter 37 • Quality Assurance & Improvement • 37-5

 A RESOURCE GUIDE FOR EARLY HEARING DETECTION & INTERVENTION

Table 4
(continued)
MCHB and CDC (continued)

Four standards for effective evaluation are: An instructional video and workbook
to assist in implementing a successful
Standard 1. Utility. program evaluation is available at no cost
Will the evaluation serve the information from the CDC. The URL is http://www.
needs of the intended users? cdc.gov/eval/framework.htm

Standard 2. Feasibility.
Is the evaluation realistic, prudent,
diplomatic, and frugal?

Standard 3. Propriety.
Will the evaluators behave legally,
ethically, and with due regard for the
welfare of those involved and those
affected?

Standard 4. Accuracy.
Will the evaluation reveal and convey
technically accurate information?

Additional issues to consider regarding • What resources and Photo courtesy

of NCHAM
Documented outcomes quality assurance and improvement: expertise are available for
and improved practices external evaluation? How much
can prove a program’s • Should the EHDI program evaluation should a comprehensive evaluation
be contracted out to an independent of a state EHDI program cost?
value and result external evaluator, or should an internal Obviously, hiring an outside expert
in continued and evaluation be done by someone from to conduct an evaluation can be
committed funding. the department of health? Advantages of expensive. A legitimate question
external evaluation include removal of might be, “Doesn’t a comprehensive
internal biases and obtaining expertise evaluation involve money that
in evaluation. Some resources for could be better spent for serving
external evaluation may be found babies and families?” One response
in other state programs that have to this question is that an expert
developed expertise in EHDI program evaluation can enhance program
evaluation. Key personnel from those sustainability. Documented
programs may be hired to complete outcomes and improved practices
an external evaluation. NCHAM also can prove a program’s value and
provides qualified experts to conduct result in continued and committed
evaluations of comprehensive programs. funding.

eBook Chapter 37 • Quality Assurance & Improvement • 37-6

NATIONAL CENTER FOR HEARING ASSESSMENT & MANAGEMENT

Online Resources and Links

• Analytic Methods in Maternal and Child Health, University of Illinois School of
Public Health, http://www.uic.edu/sph/dataskills/publications/wrkbkpdfs/?ID=3938
• Centers for Disease Control and Prevention, Program Evaluation Tool Kit: A
Blueprint for Public Health Management, http://www.cdc.gov/eval/framework.htm
• National Center for Hearing Assessment and Management, www.infanthearing.org
• Program Evaluation Tool Kit: A Blueprint for Public Health Management, http://
www.ottawa.ca/residents/funding/toolkit/index_en.html
• User-Friendly Handbook for Mixed Method Evaluations: National Science
Foundation Directorate for Education and Human Resources, http://www.nsf.gov/
pubs/1997/nsf97153/start.htm

References
Jha, A. K., & Epstein, A. M. (2009, November 6). Hospital governance and the quality of
care. Millwood, WA: Health Affairs, doi: 10.1377/hlthaff.2009.0297
National Center for Hearing Assessment and Management. Program evaluation.
Utah State Department of Health. Utah’s Early Hearing Detection and Intervention program
evaluation model.

eBook Chapter 37 • Quality Assurance & Improvement • 37-7

A RESOURCE GUIDE FOR EARLY HEARING DETECTION & INTERVENTION

eBook Chapter 37 • Quality Assurance & Improvement • 37-8