Applying Human Factors and Usability Engineering To Optimize Medical Device Design

Contains Nonbinding Recommendations Draft - Not for Implementation
Draft Guidance for Industry and Food and Drug Administration Staff Applying Human Factors and Usability Engineering to Optimize Medical Device Design
DRAFT GUIDANCE
This guidance document is being distributed for comment purposes only. Document issued on: June 22, 2011
You should submit comments and suggestions regarding this draft document within 90 days of publication in the Federal Register of the notice announcing the availability of the draft guidance. Submit written comments to the Division of Dockets Management (HFA-305), Food and Drug Administration, 5630 Fishers Lane, rm. 1061, Rockville, MD 20852. Submit electronic comments to http://www.regulations.gov. Identify all comments with the docket number listed in the notice of availability that publishes in the Federal Register. For questions regarding this document, contact Ron Kaye at ron.kaye@fda.hhs.gov or (301) 796-6289, or Molly Story at molly.story@fda.hhs.gov or (301) 796-1456. When final, this document will supersede Medical Device Use-Safety: Incorporating Human Factors Engineering into Risk Management (Issued July 18, 2000). U.S. Department of Health and Human Services Food and Drug Administration Center for Devices and Radiological Health Office of Device Evaluation
Preface
Additional Copies
Additional copies are available from the Internet. You may also send an e-mail request to dsmica@fda.hhs.gov to receive an electronic copy of the guidance or send a fax request to 301827-8149 to receive a hard copy. Please use the document number 1757 to identify the guidance you are requesting.
Page 2 of 37
Table of Contents
1. 2.
3. 4. 5.
5.1 5.2 5.3
Introduction__________________________________________________________ 4 Scope _______________________________________________________________ 5 Overview ____________________________________________________________ 5 Regulations, Guidance Documents, and Standards for HFE/UE _______________ 9 Device Users, Use Environments and User Interfaces _______________________ 10
Device Users___________________________________________________________ 12 Device Use Environments ________________________________________________ 13 Device User Interfaces __________________________________________________ 14
6. Analytical Methods for Identifying, Evaluating and Understanding Use-Related Hazards_________________________________________________________________ 15

6.1 6.2
6.2.1 6.2.2 6.2.3 6.2.4 6.2.5
Identification of Known Problems_________________________________________ 15 Analytical Approaches to Hazard Identification and Task Prioritization_________ 16
Contextual Inquiry _________________________________________________________16 Interviews and Focus Groups _________________________________________________16 Function and Task Analysis __________________________________________________17 Heuristic Analysis__________________________________________________________18 Expert Review_____________________________________________________________19
7.
7.1 7.2
Formative Evaluations ________________________________________________ 19

Cognitive Walk-Through ________________________________________________ 20 Simulated Use Testing __________________________________________________ 20
8. 9. 10.
Mitigation and Control of Use-Related Hazards ____________________________ 21 Design Verification Testing ____________________________________________ 22 Human Factors Validation Testing ______________________________________ 23
Simulated Use Validation Testing _________________________________________ 23
Tasks and Use Scenarios_____________________________________________________24 Test Participants (Subjects)___________________________________________________24 Participant Training ________________________________________________________25 Data Collection ____________________________________________________________26 Interpretation of Validation Test Results and Addressing Problems ___________________27 10.1.1 10.1.2 10.1.3 10.1.4 10.1.5
10.1
10.2
Clinical Validation Testing_______________________________________________ 27
11. Documentation _______________________________________________________ 28 12. Conclusion__________________________________________________________ 28 Appendix A ______________________________________________________________ 30 Appendix B ______________________________________________________________ 33 Appendix C ______________________________________________________________ 35
Page 3 of 37
Draft Guidance for Industry and Food and Drug Administration Staff Applying Human Factors and Usability Engineering to Medical Devices to Optimize Safety and Effectiveness in Design
This draft guidance, when finalized, will represent the Food and Drug Administration's (FDA's) current thinking on this topic. It does not create or confer any rights for or on any person and does not operate to bind FDA or the public. You can use an alternative approach if the approach satisfies the requirements of the applicable statutes and regulations. If you want to discuss an alternative approach, contact the FDA staff responsible for implementing this guidance. If you cannot identify the appropriate FDA staff, call the appropriate number listed on the title page of this guidance.
1.
Introduction
FDA has developed this draft guidance document to assist industry in conducting appropriate human factors testing and identifying device features that manufacturers should optimize throughout the total product life cycle. The recommendations in this draft guidance document are intended to improve the usability of devices to reduce use error, injuries from medical devices, and product recalls. The FDA believes that these recommendations will help control current risks and reduce future risks associated with device use. FDA's guidance documents, including this one, do not establish legally enforceable responsibilities. Instead, guidance documents describe the Agency's current thinking on a topic and should be viewed only as recommendations, unless specific regulatory or statutory requirements are cited. The use of the word should in Agency guidance documents means that something is suggested or recommended, but not required.
Page 4 of 37
2.
Scope
This guidance provides recommendations for medical device design optimization through human factors analysis, testing and validation. The intent is to improve the quality of the device user interface such that errors that occur during use of the device are either eliminated or reduced. The recommendations in this document apply whenever a manufacturer performs human factors testing for a device. As part of their design controls 1 , manufacturers conduct a risk analysis that includes risks associated with device use. If the results of this analysis indicate that there is a moderate to high risk of use error, or if a manufacturer is modifying a marketed device due to problems associated with use, particularly as a corrective and preventive action (CAPA), then the manufacturer should perform appropriate human factors testing according to this guidance document. Additionally, FDA staff may request human factors testing if: (i) submission of human factors information is required (for example, as a special control); (ii) submission of human factors information is recommended in a specific guidance for a device type and the manufacturer cannot justify forgoing such testing; or (iii) on a forcause basis if it is the least burdensome method to address FDAs concerns regarding human factors issues. Under these circumstances, manufacturers should provide FDA with a report that summarizes the human factors processes, evaluations, and results of validation testing as part of their pre-market applications or submissions (see Appendix A).
3. Overview
To understand use-related hazards, it is necessary to have an accurate and complete understanding of how a device will be used. Understanding and optimizing how people interact with technology is the subject of human factors engineering (HFE) and usability engineering (UE). HFE/UE considerations that are important to the development of medical devices include three major components of the device-user system: (1) device users, (2) device use environments and (3) device user interfaces. This interaction and its possible results are depicted graphically in Figure 1.
21 CFR 820.30
Page 5 of 37
Figure 1. Interactions among HFE/UE considerations result in either safe and effective use or unsafe or ineffective use.
For safety-critical technologies such as medical devices, the process of eliminating or reducing design-related use problems that contribute to or cause unsafe or ineffective medical treatment is part of a process for controlling overall risk. For devices where harm could result from use errors, the dynamics of user interaction are safety-related and should be components of risk analysis and risk management. Ideally, medical device designers develop devices that are safe and reliable for their intended uses. To achieve this goal, they should consider the possibilities of hazards arising from use of and failures of the device and its components. Hazards traditionally considered in risk analysis include: Chemical hazards (e.g., toxic chemicals), Mechanical hazards (e.g., kinetic or potential energy from a moving object), Thermal hazards (e.g., high temperature components), Electrical hazards (e.g., electrical shock, electromagnetic interference (EMI)), Radiation hazards (e.g., ionizing and non-ionizing), and Biological hazards (e.g., allergic reactions, bio-incompatibility, and infection).
These hazards most often result from instances of device or component failure that are not dependent on how the user interacts with the device. In addition to the hazards mentioned above, hazards for medical devices that are associated with device use should also be considered. Hazards caused specifically by how a device is used are referred to in this document as use-related hazards (Figure 2). These include use errors involving failure to perceive, read, interpret, or recognize and
Page 6 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation act on information from monitoring or diagnostic testing devices, and improper treatment (e.g., ineffective or dangerous therapy) for devices that provide medical treatment.
Figure 2. Use-Related Hazards, Device Failure Hazards, and Their Intersection.
Use-related hazards occur for one or more of the following reasons: Device use requires physical, perceptual, or cognitive abilities that exceed the abilities of the user; The use environment affects operation of the device and this effect is not recognized or understood by the user; The particular use environment impairs the users physical, perceptual, or cognitive capabilities when using the device to an extent that negatively affects the users interactions with the device; Device use is inconsistent with users expectations or intuition about device operation; Devices are used in ways that were not anticipated; or Devices are used in ways that were anticipated but inappropriate and for which adequate controls were not applied.
HFE/UE considerations and approaches should be incorporated into device design, development and risk management processes. Three central steps, consistent with ISO 14971, Medical devices Application of risk management to medical devices, are essential for performing a successful HFE/UE analysis: Identify anticipated use-related hazards (derived analytically, see Section 6) and unanticipated use-related hazards (derived through formative evaluations, see Section 7), and determine how hazardous use situations occur; Develop and apply strategies to mitigate or control use-related hazards (see Section 8); and Demonstrate safe and effective device use through human factors validation testing (see Section 10).
Page 7 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation Figure 3 depicts the risk management process for addressing use-related hazards; HFE/UE approaches should be applied for this process to work effectively.
Figure 3: Addressing Use-Related Hazards in Risk Management.
Page 8 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation An outline and recommendations for the HFE/UE Report that device manufacturers should use to submit the results of their HFE/UE activities to FDA are included as Appendix A to this document.
4.
4.1.
Regulations, Guidance Documents, and Standards for HFE/UE

Regulatory Basis for HFE/UE Analysis and Testing
Human factors techniques play an important role in fulfilling the design control requirements of the Quality System regulation, 21 CFR Part 820. Specifically, human factors testing helps ensure proper design of the user interface. The risk analysis that fulfills quality system requirements should include use error. To establish the design input for the user interface and carry out design verification, human factors activities conducted throughout the development process can include task/function analyses, user studies, prototype tests and mock-up reviews (see Sections 6 and 7). Formative and validation testing fulfill the requirements to test the device under realistic conditions. Validation testing (see Section 10) should be used to demonstrate that the potential for use error has been minimized. The development process for a devices user interface should include review and incorporation of relevant standards and guidelines that are applicable to the design. FDA general and specific guidance documents, as well as consensus standards recognized by FDA, are listed on CDRHs home page, at www.fda.gov/cdrh.
4.2.
FDA Guidance Documents
To facilitate premarket review and assist manufacturers, FDA has published devicespecific and general guidance documents. As of this writing, guidance documents that contain recommendations that are particularly relevant for human factors are: Human Factors Implications of the New GMP Rule Overall Requirements of the New Quality System Regulation, Design Control Guidance for Medical Device Manufacturers, Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices, Guidance on Medical Device Patient Labeling, and Guidance for Industry and FDA Staff - Total Product Life Cycle: Infusion Pump Premarket Notification [510(k)] Submissions. These guidance documents also cite relevant standards, so they provide a good starting point for manufacturers seeking information on HFE/UE.
Page 9 of 37
4.3.
National and International Standards
FDA has officially recognized device-specific and general standards published by national and international standards bodies. Standards recognized by FDA as of this writing related to human factors and the application of HFE/UE to medical devices are listed in Table 1.
Table 1. National and international standards involving human factors and usability engineering.
Standard AAMI/ANSI HE75:2009 Title Human Factors Engineering Design of Medical Devices Main Purpose Comprehensive reference that includes general principles, usability testing, design elements, integrated solutions HFE/UE process applied to all medical devices, with emphasis on risk management Risk management process for medical devices HFE/UE process applied to alarm systems for medical electrical equipment and systems
ISO/IEC 62366:2007
ANSI/AAMI/ISO 14971:2007 IEC 60601-1-8:2006
Medical devices Application of usability engineering to medical devices Medical Devices Application of risk management to medical devices Medical electrical equipment Part 18: General requirements for basic safety and essential performance Collateral Standard: General requirements, tests and guidance for alarm systems in medical electrical equipment and medical electrical systems
This guidance document is generally consistent with the standards listed in Table 1. For specific issues that are not consistent with any given recognized standard, this document takes precedence. FDAs website presents a list of recognized standards, including those relevant to human factors. It is important to review the supplementary information sheets (SIS) for all recognized standards to understand the extent of Agency recognition of each standard.
5. Device Users, Use Environments and User Interfaces

Figure 4 presents a model of the interface between a human and a machine, the actions performed by each, and the interactions between them. (In the context of this document, the term machine means a medical device.) When users interact with a device, they perceive any information provided by the device, then interpret and process the information and make decisions. After that the user may interact with the device to change some aspect of it. The device then receives the user input, responds to the input, and provides feedback to the user. The user might then perceive the new information and might initiate another cycle of interaction.
Page 10 of 37
Figure 4: Device User Interface in Operational Context (adapted from Redmill and Rajan, 1997).
The user interface includes all components of a device with which the user interacts, such as controls and displays (i.e., those parts of the device that users see, touch, and hear). The user interface also includes the device labeling, which includes package labels, any instructions for use in user manuals, package inserts, instructions on the device itself, and any accompanying informational materials. To gain an understanding of the potential HFE/UE analyses that should be conducted for a particular device, you should consider: Device users: o Identification of the end-users of the device (e.g., patient, family member, physician, nurse, professional caregiver) o The level of training users will have and/or receive o User characteristics (e.g., functional capabilities, attitudes and behaviors) that could impact the safe and effective use of the device o Ways in which users might use the device that could cause harm Device use environment: o Hospital, surgical suite, home, emergency use, public use, etc. o Special environments (e.g., emergency transport, mass casualty event, sterile isolation, hospital intensive care unit) o Interoperability with other devices
Page 11 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation Device user interface: o E.g., functions, capabilities, features, maintenance requirements o Indicated uses
These considerations, discussed in the following sections, will help you identify specific aspects of device use that are associated with potential use-related hazards that should be investigated through HFE/UE analysis and testing.
5.1
Device Users
Individuals in the intended user populations should be able to use medical devices safely and effectively and without unintentionally making errors that could compromise positive outcomes. With proper application of HFE/UE, the design of a device can be modified to be either less dependent on the abilities of the user or more accommodating of disabilities. For example, people with diabetes often have some degree of retinopathy (a degenerative disease of the retina), which causes impaired eyesight. These users have difficulty reading displays, such as on blood glucose testing meters, especially when the text is small or the visual contrast is low. Depending on the specific device and its application, device users may be limited to professional caregivers, such as physicians, nurses, nurse practitioners, physical and occupational therapists, social workers, and home care aides. Other user populations may be non-professionals, including patients who operate devices on themselves to provide self-care and family members or friends who serve as lay caregivers to people receiving care in the home, including parents who use or supervise the use of devices for their children. Device user populations may also include the professionals who install and set up the devices and those who maintain, repair, clean and reprocess them. The ability of a user to operate a medical device depends on his or her personal characteristics, including: Physical size, strength, and stamina, Physical dexterity, flexibility, and coordination, Sensory abilities (i.e., vision, hearing, tactile sensitivity), Cognitive abilities, including memory, Medical condition for which the device is being used, Comorbidities (i.e., multiple conditions or diseases), Literacy and language skills, General health status, Mental and emotional state, Level of education and health literacy relative to the medical condition involved, General knowledge of similar types of devices, Knowledge of and experience with the particular device, Ability to learn and adapt to a new device, and Willingness and motivation to use a new device.
Page 12 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation You should evaluate and understand essential characteristics of all intended user groups and describe them for the purpose of HFE/UE evaluation and design activities.
5.2
Device Use Environments
The environments in which medical devices are used may present a range of complexities. Medical devices may be used under variable conditions involving such environmental attributes as space, lighting, noise levels, and activity. Examples of environmental hazards in the clinical setting include the following: Rooms can be physically crowded or cluttered, making it difficult for people to maneuver in the space. The lighting level can be low, making it hard to see device displays or controls. The noise level can be high, making it hard to hear device operation feedback or audible alerts and alarms. The room can be busy with other people and activities, providing distractions that can confuse the device operator.
Non-clinical environments can present additional challenges. For example: Carpeting or stairs might make it hard to move medical devices around the space. The environment might not be clean. The utility service might not be reliable. In addition, the electrical outlets might not be grounded, and the water might not be clean. The temperature might be very high, which could cause devices to overheat (and make users hands sweaty), or the temperature might be very low, which could make devices inoperable (and make users fingers stiff and decrease sensitivity). The humidity might be very high, which could cause condensation to form, or very low, which could produce static electricity. Other individuals and activities in the vicinity may cause distractions. Unauthorized users, such as children, might be present and could hurt themselves (e.g., playing with a syringe), damage the device (e.g., chewing on or misconnecting the tubing), or change device settings (which might not be noticed by the operator before using the device the next time). Pets or vermin could contaminate or damage devices in the home. Electromagnetic interference from other equipment (e.g., cell phones and computer accessories) could affect medical device performance.
Use environments can also limit the effectiveness of visual and auditory displays (lighted indicators, auditory alarms and other signals) if they are not designed appropriately. For example, in noisy environments, the user might not be able to notice a devices alarms if they are not sufficiently loud or distinctive. When multiple alarms occur for different devices or on the same device, or if alarms sound too often, i.e., nuisance alarms, the user could fail to notice them or be unable to make important distinctions among them.
Page 13 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation Similarly, motion and vibration can affect the degree to which people are able to perform fine physical manipulations such as typing on a keyboard or reading displayed information.
5.3
Device User Interfaces
The user interface (see Figure 4) includes all components of a device with which users interact while using the device, preparing it for use (e.g., unpacking, set up, calibration), or performing maintenance (e.g., cleaning, replacing a battery, repairing). It includes: The hardware components that control device operation such as switches, buttons, and knobs, Device elements that provide information to the user such as indicator lights, displays, auditory and visual alarms, The design of menu-driven interface systems, The logic that directs how the system responds to user actions including how, when, and in what form information (feedback) is provided to the user, The size and configuration of the device (particularly for hand-held devices), and Device labeling, packaging, training materials, operating instructions, and other reference materials. The most effective strategies to address use-related hazards in the premarket setting focus on improvements to the design of the device user interface. To the greatest extent possible, the user interface should convey the concept for correct operation through its appearance and operation (look and feel) so that safe and effective use is intuitive. A well-designed user interface will facilitate correct actions and will prevent or discourage actions that could result in hazards. Addressing use-related hazards by modifying the device design is generally more effective than revising the labeling or training. Labeling might not be accessible when needed, and training depends on memory, which might not be complete or accurate. An important aspect of the user interface is the extent to which the logic of information display and control actions is consistent with users abilities, expectations, and likely behaviors. Users will expect devices and device components to operate in ways that are consistent with their experience with other similar devices or user interface elements. For example, users may expect the flow rate of a liquid or gaseous substance to increase or to decrease by turning a control knob in a specific direction based on their previous experience with other similar devices. Hazards result when, for example, an electronically-driven device control is designed to operate in the opposite direction of controls that were previously mechanical. Increasingly, user interfaces for new medical devices are computer-based. In these cases, the interface controls may include: keyboards, mouses, styluses, and touchscreens. Other essential features of the user interface include the manner in which data is organized and presented, control and monitoring screens, screen components, prompts, navigation logic, alerting mechanisms, data entry requirements, and help functions. The design of these elements should take HFE/UE considerations into account. Page 14 of 37
6.
Analytical Methods for Identifying, Evaluating and Understanding Use-Related Hazards
This document describes two broad classifications for identifying, evaluating and understanding use-related hazards early in the design process: analytical approaches and formative evaluations. These techniques are discussed separately; however, they are interdependent and should be employed in complementary ways. The analytical approach is discussed here in Section 6, and formative evaluations are discussed in Section 7. Analytical approaches involve description and systematic decomposition and analysis of device use. These approaches can be used early in device development processes to identify specific tasks or scenarios, including specific user-device interactions, relevant for assessing inadvertent use errors that could possibly cause harm. The application of analytical approaches is particularly useful for identifying and resolving use-related hazards that occur infrequently or are too dangerous to study in an evaluation involving simulated uses. Analytical approaches include: (1) analysis of the expected use of new devices and of available information about the use of similar devices (Section 6.1); and (2) employment of methods that can include contextual inquiry, interview techniques, function and task analysis, and heuristic and expert analyses (Section 6.2). Hazards identified through analytical processes represent an initial set of assumptions regarding the risk profiles for various types of user interactions with the device and should be updated as necessary based on findings derived from formative evaluations (Section 7). You should determine which of the analytical approaches in Section 6.2 would be appropriate and informative for your design processes, particularly for assessing userelated hazards and risks, and for conducting subsequent formative evaluations and validation testing. When analytical approaches and formative evaluations identify unacceptable use-related risks, they should be followed by risk mitigation and device modification (or if necessary, revision of labeling or training; see Section 8). In addition, the results of analytical approaches and formative evaluations should be used to inform the development of validation testing protocols (see Section 10).
6.1
Identification of Known Problems
The first step in the analytical approach is to identify use errors and other problems that have occurred in the past with devices that are similar to the one under development so they may be addressed in the design of the new device. These devices might have been made by the same manufacturer or might include similar devices made by other manufacturers. Good sources of this type of information are customer complaint files and the knowledge of training and sales staff that are often familiar with common difficulties Page 15 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation and misunderstandings encountered by users. Other sources of information on known use-related hazards are current device users, journal articles, proceedings of professional meetings, newsletters, and relevant internet sites. Some of the most important web sites are: FDAs Manufacturer and User Facility Device Experience (MAUDE) database; FDAs Medical Device Reporting (MDR) Program Search; FDAs Adverse Event Reporting Data Files; FDAs MedSun: Medical Product Safety Network; CDRH Medical Device Recalls; CDRH Alerts and Notices (Medical Devices); CDRH Public Health Notifications; ECRIs Medical Device Safety Reports; The Institute of Safe Medical Practices (ISMP's) Medication Safety Alert Newsletters; and The Joint Commissions Sentinel Events.
You should incorporate all known use errors and problems into the risk analysis for a new device and take them into account when selecting the critical tasks to be evaluated as part of your human factors analyses. All known use errors and problems should also be incorporated into the risk analysis if you are making changes to a marketed device due to problems associated with use, particularly as a corrective and preventive action (CAPA), and are doing a human factors analysis, including validation testing, to assure those changes have properly mitigated risks associated with use error.
6.2
Analytical Approaches to Hazard Identification and Task Prioritization
Analytical approaches to identify use-related hazards and prioritize critical tasks associated with device use include contextual inquiry, interview techniques, function and task analysis, and heuristic and expert analyses. 6.2.1 Contextual Inquiry Contextual inquiry is a method of assessing user-device interactions in their naturally occurring contexts. The method generally involves a researcher observing and interviewing users while they use a device as they normally would. The users demonstrate how they use the device and the researcher asks questions to ensure that he or she understands what the users are doing and why they are doing it that way. The method is valuable for understanding user and patient needs, capturing current userdevice interactions, and identifying design input requirements for new devices. 6.2.2 Interviews and Focus Groups One-on-one interviews provide qualitative information regarding the perceptions, opinions, beliefs and attitudes of individual device users and patients. In an interview, the researcher can focus on topics of particular interest and explore specific issues in depth.
Page 16 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation Interviews should be structured to cover all relevant topics but allow for unscripted discussion as the interviewees responses require clarification or raise new questions. Interviews allow the researcher to understand the perspectives of individuals who, for example, might represent specific categories of users, aspects of device use, or applications of a device. Focus groups are similar to interviews but are conducted with groups of people who have the opportunity to interact with one another and discuss topics. The groups may be homogeneous participants have a common characteristic that affects device use, such as job title, experience level, or type of medical condition; or the groups may be heterogeneous participants have diverse characteristics, which may stimulate different types of discussions. 6.2.3 Function and Task Analysis Function or task analysis techniques systematically break down the device use process into discrete steps or sequences for the purpose of description and further analysis of potential use error. With respect to safety, function and task analyses can aid the device development process by: Identifying critical aspects of device use potentially resulting in hazards to users and patients; Providing a basis for the analysis of use-related hazards; and Evaluating known incidents or accidents to understand what led to the problem.
Analyzing functions and tasks in this way will allow you to identify possible hazards associated with device use. And, function and task analyses can provide a foundation for your subsequent HFE/UE efforts. For instance, test scenarios (see Section 10.1.1) should be developed to address use scenarios that involve tasks identified as critical or errorprone. A simplistic example of a task analysis component for a hand-held blood glucose meter includes the tasks listed in Table 2.
Table 2. A simple task analysis showing tasks that are performed by the user, the device, or a combination of the user and the device.
Task Patients finger is lanced with automatic lancing device Blood sample is placed on test strip Test strip is placed in device The sample is allowed to react with reagents in the test strip for a specific time 5 Blood glucose level in the sample is measured 6 The resulting value is displayed 7 The displayed value is read, interpreted, and acted upon 1 2 3 4
Device X
User X X X
X X X X
Page 17 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation After functions and tasks have been identified, the user tasks are analyzed to determine if and how HFE/UE considerations apply. For instance, in Task 2 in Table 2, the user places a sample of blood on a test strip. The following fundamental questions should be investigated: Are any use-related hazardous scenarios possible? How might they occur? How likely are they? What are the possible consequences of each? How might they be prevented?
To begin to address these, the analyst should pose more specific questions, such as: How difficult is it for users to use the device components and accessories to perform this task correctly? How much effort is required by the user to apply a sample correctly? What characteristics of the user population might cause some users to have difficulty with this task? Where will the testing be done, and could ambient conditions affect the test results or the users ability to perform the task? Is the proper use of test strips evident to the user? Will certain user interactions with the device degrade the accuracy, safety and effectiveness of the devices subsequent operations (and if so, what are these interactions and how are device operations affected)?
In early glucose monitors, the user had to perform Task 4 manually (the sample is allowed to react with reagents in the test strip for a specific time). Users had difficulty doing this task well, and the accuracy of the results too often suffered from the users failure to time the process accurately. In subsequent models, technology was developed to enable the device to perform this task automatically. Modification in device design and operation removed that use scenario and the associated hazard. 6.2.4 Heuristic Analysis Heuristic analysis is an analytical process in which analysts formally evaluate a devices user interface against well-established user interface design rules or heuristic guidelines. The object is to identify possible use-related hazards with a focus on the interaction of the user with the user interface and operating logic of the device. Heuristic analyses include careful consideration of accepted concepts for design and operation of the user interface, sometimes known as de-facto standards or population stereotypes which are essentially social and cultural norms and constraints for the use of device components. A simple example is a light switch oriented in a vertical direction being on when it is in the up position and off when in the down position. For medical devices, general de-facto standards are applicable at times, while others are unique for certain kinds or types of medical devices.
Page 18 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation 6.2.5 Expert Review Expert reviews rely on clinical and human factors experts to analyze device use, identify problems, and make recommendations for addressing them. The difference between expert review and heuristic analysis is that expert review relies more heavily on personal assessment done by individuals with expertise in a specific area. The success of the expert review depends on the experts knowledge of the device technology, its use, clinical perspectives, characteristics of the intended users, as well as the experts ability to predict actual device use.
7.
Formative Evaluations
Formative evaluations are conducted to inform product development in progress. These evaluations derive information from user interaction with devices under conditions of varying degrees of formality and may include various simulated-use testing approaches. Formative studies that involve use of the device by representative end users are useful for identifying problems that were not identified or sufficiently understood using analytical methods, early in the design process when they can be addressed more easily and less expensively. Formative studies can be conducted informally, with simple mock-up devices or preliminary prototypes and labeling (including the draft instructions for use), and with small numbers of test participants. Modifications should be made and then evaluated for adequacy during this phase of device development and can be performed in an iterative fashion until the device is considered to be optimized to a level at which validation testing is appropriate. Formative evaluations support decision making on design tradeoff analyses, user training requirements and the design of the instructions for use. Formative evaluation should focus initially on the major issues that preliminary evaluations indicate are most likely to have an impact on use safety and effectiveness and those areas where design options for the user interface are not final (i.e., aspects of use and design that are complex and need to be explored). Depending on the results of preliminary evaluation, certain aspects of the use environment or specific sub-groups of users can be included in these evaluations. The evaluation methods used should be chosen based on the need for clarification prior to developing final design specifications and based on resources available. Determining users needs for training and designing the content and format of the training are challenging, especially for complex medical devices. Some devices have lengthy and detailed instructions for use, which can be hundreds of pages long. It may be difficult to assess the effectiveness of training packages; however training requirements and training packages should be finalized prior to clinical use of the device, whether that use occurs within an IDE submission or following FDA clearance. The effectiveness of training might require simulated use, which might be conducted in the home for home users over a period of time during which they can acquaint themselves with device operation and the training materials. After simulated use of the device and the training
Page 19 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation materials, the test participants can be interviewed with regard to the adequacy of the training provided. Formative human factors assessments serve the following HFE/UE goals: Identify and prioritize tasks according to relative risk to the user beyond estimates derived from analytical techniques; Guide development of use scenarios to be employed during subsequent design validation testing; Identify use-related hazardous situations leading to the development of risk mitigation strategies; Evaluate trade-off considerations and effectiveness of design enhancements, training and instructions for use; Guide modification of the device design to optimize the user interface with respect to device safety and effectiveness; and Clarify the dynamics of device-interaction associated with known or suspected use error scenarios.
If formative evaluation has been carried out successfully, the subsequent validation testing should result in good performance and few or no user difficulties or other concerns.
7.1
Cognitive Walk-Through
A simple kind of formative study involving users is the cognitive walk-through. It is less time-consuming and less formal than simulated use testing (see Section 7.2). The cognitive walk-through technique is most useful early in the development process, and for developing and evaluating use scenarios to be explored in subsequent studies. In a cognitive walk-through, a user or small group of users are guided through a structured process of using a device, which may be represented as a simple mock-up or early-stage prototype. During the walk-through, participants are questioned and encouraged to provide feedback on difficulties they notice while using the device. Evaluators can also collect subjective information from participants about their thought processes, mental models, and perceived workload when using the device.
7.2
Simulated Use Testing
Simulated use testing, (also called usability testing and, occasionally, user testing) involves systematic collection of data from users (participants) using a device (or device component or system) in realistic situations. Data are obtained in a variety of ways, including subjective user feedback, manual and automated measures of user performance, and direct observation. Studies of device use under simulated conditions can be used early in the design process to clarify suspected or known problems with device use, demonstrate that use-related hazards have been addressed, evaluate candidate design alternatives, and validate safe and effective use by intended users. Beyond application to the safety and effectiveness of Page 20 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation device use, formative use studies provide a powerful means for creating effective labeling (including instructions for use) and user-friendly device designs. Simulated use studies can identify problems that were noticed by test participants but did not manifest themselves as errors during use. It is often most efficient to focus the evaluation and preliminary conclusions on user tasks that are associated with specific interactions with the user interface. However, while some tasks are directly observable and user performance can be assessed through observation, other tasks must be tested as knowledge tasks. For these tasks, the users knowledge regarding what should or should not be done with the device or when certain actions are necessary is best assessed through systematic questioning of the users. For instance, home users might need to understand critical device limitations, vulnerabilities to specific conditions of use, problems with taking shortcuts or reusing disposable components, or the need to maintain the device or its accessories. They might also need to understand when to replace a drug vial in an auto injector or when to calibrate a glucose monitor. For these devices and users it is essential to assess user performance but it is also important to determine whether users can acquire the necessary knowledge from the labeling materials. If the intended users have specific limitations in their abilities, one focus of simulated-use testing should be to establish whether these limitations affect device use. If so, you should conduct further testing to determine whether potential use problems associated with user limitations can be mitigated by modifying the design of the device interface or the operation of the device. Although simulated use studies are effective in identifying and understanding device use, you should take care not to underestimate the frequency of problems based on the experiences of test participants. Participants could be (despite test coordinators best efforts) unrealistically well trained, capable, or careful. Also, when people are observed they often try to do their best, tend to follow instructions more carefully than when they operate the device independently and often do not use the device long enough to experience problems that arise more infrequently. In addition, devices used in simulation testing are generally new and in good operating condition and it may be difficult to simulate the affects of long term use. Members of your team who are developing the device should not participate as users since their knowledge of how the device operates (or should operate) will influence how they use it. For the consideration of device use-related risks to be complete, empirical methodologies should include efforts that focus on identification and analysis of unanticipated userelated hazards and the incorporation of the results into the overall risk management process.
8.
Mitigation and Control of Use-Related Hazards
Use-related hazards that are identified through analytical approaches or formative evaluations should be designed out or controlled prior to submitting the device for
Page 21 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation HFE/UE validation testing. Use-related hazards often require a combination of mitigation and control strategies. The following list presents the order of overall priority for applying strategies to control or mitigate risks of use-related hazards: 1. Modify the device design to remove a hazard or reduce its consequences: For example, making the user interface intuitive and ensuring that critical information is effectively communicated to the user can reduce the likelihood of or eliminate certain use-related hazards. If hazards cannot be eliminated, the design should, to the extent possible, reduce their likelihood and the severity of any consequences. 2. Make the user interface, including its operating logic, error tolerant: When errors occur during device use, such as users pressing an adjacent key on a keypad, the device should act to preclude a hazardous outcome. Safety mechanisms such as physical safety guards, shielded controls, or software or hardware interlocks will make the design more tolerant of errors that users might make. 3. Alert users to the hazard: When neither design nor safety features will eliminate a use-related hazard or adequately mitigate the consequences, the device should detect the condition and provide an adequate warning signal to users. 4. Develop written procedures and training for safe operation: If it is impossible to eliminate hazards through any of the previous strategies, or to enhance other control or mitigation strategies, then written procedures, labeling enhancements, and training for safe operation are the remaining options. Instructions, labeling, and training can influence users to use devices safely and effectively and are critical HFE/UE considerations for safe device use. However, because they rely on the user to remember or refer back to the information, these approaches are less effective than modifications to the design of the user interface. In addition, training may be inconsistent or unavailable. Therefore, mitigation of userelated hazards should not focus on instruction, labeling, or training fixes in isolation, since these patches might not be adequate. A combination of these strategies might be your best approach. Regardless of the strategy used, subsequent testing should be done to ensure that you have successfully controlled the use-related hazards and that your risk mitigation efforts have not introduced new risks.
9.
Design Verification Testing
Verification confirms that the specific functional and operational requirements for the design of a device user interface have been met. The process for verifying individual user interface requirements will likely require focused effort for both functional and operational requirements. For instance, if a device will be used by elderly users with hearing abilities ranging from normal to moderate impairment, the design specification should assure that the devices alarm volume can be adjustable to a sufficient level to accommodate these users. The verification process would involve testing the device alarm to assure that the volume adjustment capability (and any other specifications developed to assist users) has been implemented successfully.
Page 22 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation Verification can be done in an iterative fashion as the need for modifications of the user interface are identified and implemented. It is essential that verification is done prior to validation testing so that the device user interface that will undergo validation testing represents the finished product.
10. Human Factors Validation Testing

The human factors validation test demonstrates that the intended users of a medical device can safely and effectively perform critical tasks for the intended uses in the expected use environments. It is particularly important during validation testing to use a production version of the device, representative device users, actual use or simulated use in an environment of appropriate realism, and to address all aspects of intended use. Validation is often carried out under conditions of simulated use, but, if necessary, further evaluation can be undertaken under conditions of actual use in a clinical study (see Section 10.2). For the device to be considered to be optimized with respect to safety and effectiveness of use, validation testing should be designed such that it is sufficiently sensitive to capture use-related problems that exist whether the users are aware of use errors or not. Further, the test results should show no patterns of use failure or difficulties that could be eliminated or reduced through further modification of the design of the user interface. We recommend that the validation process and the results obtained be presented in the form of a HFE/UE report, the elements of which are described in Appendix A to this document. The realism and completeness of validation testing should support generalization regarding safe and effective use by the ultimate population of users and actual types and conditions of use. The testing protocol should reflect a focus on the highest-priority tasks or use scenarios and describe methods to collect sufficient and appropriate data to demonstrate that all critical aspects of use can be performed well and that users do not report patterns of difficulty arising from features of the user interface or elements that are not provided but necessary. The results of the testing should facilitate identification and understanding of the root causes of use failures or problems that do occur. You should consider performing validation testing under conditions of actual clinical use when simulated use validation methods appear to be inadequate. Your determination of whether or not a clinical evaluation is necessary should be based on analyses (Section 6) of device use-related risks, intended uses, users and use environments and perhaps supplemented as necessary with additional information from formative evaluations (Section 7).
10.1 Simulated Use Validation Testing

The conditions under which simulated use testing is conducted should be sufficiently realistic to enable the results of the testing to be generalized to anticipate actual use. The need for realism is therefore driven by the analysis of users, use environments, the device user interface and intended uses. To the extent that environmental factors are found to Page 23 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation affect user performance, they should be incorporated into the simulated use environment (e.g., dim lighting, multiple alarm conditions, distractions, multi-tasking and workload). See Section 5.2 for more information about device use environments. Test participants should be given an opportunity to use the device independently and in as natural a manner as possible, without guidance, coaching, praise or critique from the test facilitator or moderator. Users should not be allowed a second chance to perform a task correctly after a failure unless this practice is consistent with actual conditions of use and is documented in the labeling that will accompany the device.
10.1.1 Tasks and Use Scenarios

You should include in the testing all tasks that are critical for users to perform to assure safe and effective outcome from use. Task or use scenario priorities should be defined in terms of the potential clinical impact of task failures or sub-optimal user performance on the device user or the patient. The test protocol should describe the user tasks and/or use scenarios containing tasks to be included in the test, information regarding task criticality or relative priority, and the process by which task inclusion and priority were determined. The test protocol should also provide a rationale for the extent of device use and the number of times that participants will attempt to use the device. For example, for devices like over-the-counter (OTC) automatic external defibrillators (AEDs), only one use event should be conducted since additional attempts would be irrelevant in an actual use setting. For devices that are used frequently and have a learning curve that requires repeated use to establish reasonable proficiency, multiple uses may be appropriate. All subjective and performance data should be retained, however, and all close calls and performance failures should be investigated and explained. Tasks or use scenarios with a low frequency of occurrence that are associated with userelated hazards and risk of harm require careful consideration. Rare or unusual use scenarios with serious consequences of inadvertent errors often prove to be the greatest threat to safe and effective medical device use after a device becomes available for general use. Users are often not prepared for infrequent or unexpected use scenarios because they are often not addressed adequately in device design, training, or operating instructions. Infrequent but dangerous use scenarios are often difficult to identify, which underscores the necessity for careful application of the analytical and formative evaluation approaches (Sections 6 and 7) early on, and throughout the design process. 10.1.2 Test Participants (Subjects) The most important consideration for test participants in validation testing whether in simulated or clinical setting is that they represent the population of intended users. (See Section 5.1 for additional information on device user populations.) If the device has more than one population of users, then the validation testing should be designed to evaluate each distinct user population. The FDA views populations as distinct when their abilities or the nature of their device interactions are expected to be different. For example, some devices will have users in different age categories (pediatric, adolescent, adult, or geriatric); others will have users in different professional categories (e.g.,
Page 24 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation biomedical engineer, nurse, non-professional family-member caregiver). The number of test participants involved in design validation depends on the purpose of the validation. For human factors validation, sample size is best determined by the results of preliminary analyses of risk and formative evaluations (See Appendix B for a discussion of sample size considerations.). For example, in draft guidance, 2 the FDA currently recommends that infusion pumps (e.g., IV pumps) should use a minimum of 25 test participants for simulated use validation testing. For other devices, manufacturers must make their own determinations of the necessary number of test participants based on valid statistical rationales. 3 For devices intended to be used by more than one group of users that have distinct abilities or use roles, at least 15 participants from each group should participate in validation testing. If some users will be pediatric, the testing should include a group of representative pediatric users. The most important aspect of sampling may be the extent to which the test participants correspond to the actual end users of the device, which requires that you accurately identify and describe your user populations. For devices with multiple user populations that have different personal characteristics, it may be advisable to test the maximum number of participants that your budgets and schedules allow. Your employees should not be test participants except in rare cases when all users necessarily are employees of the manufacturer. To adequately represent users in the United States population, the participants in the validation test should reside in the United States. English fluency or first-language abilities should only be used as a test participant inclusion criterion if this requirement is also stated in the device labeling. 10.1.3 Participant Training The training provided to test participants should approximate the training that actual users will receive. If it is anticipated that some users will receive no training, then the test participants in the validation testing should include a corresponding subset of untrained users. A validation test that is conducted immediately after training or given after participants experience an unrealistically high level of training is not considered to be valid. Training should represent the actual user training experience taking into account the environment in which training occurs and the fact that retention of training decays over time. For this reason, prior to testing, a period of time should elapse following training to provide an opportunity for training decay to occur.
See Draft Guidance for Industry and FDA Staff - Total Product Life Cycle: Infusion Pump - Premarket Notification [510(k)] Submissions, released on April 23, 2010. When final, this will represent FDAs guidance on this topic.
3
21 CFR 820.250
Page 25 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation 10.1.4 Data Collection During the process of developing the validation test protocol, it is important to determine the relevance of each type of data that will be collected in the test. Some data is best collected as performance data, e.g., the number of attempts required before success is achieved should be measured directly rather than by soliciting participant opinions. On the other hand, measuring the time it takes participants to conduct a specific task might be helpful for comparing the ease of use of different device models or other purposes, but timing is only considered to be a critical task if the nature of the device requires rapid interactions with the user. Data collected during validation testing should capture both subjective and performance responses. Data should also capture close calls, as explained below. Subjective Data The validation test should include a subjective assessment of study participant feedback on use difficulties experienced during the test (e.g., confusion, awkward manual manipulations, display legibility, alarm audibility). The most effective method for capturing user experiences during validation testing is to do a post-test interview comprised of open-ended questions. The questions should first address the device overall and should then address each critical task or use scenario. For example: Did you have any difficulty using this device? Was anything confusing? What might make the device (or instructions) better? Please tell me about this [error or problem observed].
The questions should address each critical aspect of use. The validation test should include essential subjective assessments by participants for all critical tasks. Simulated-use testing should include evaluation of the adequacy of the instructions for use for the device. You may conduct this evaluation either as part of the simulated-use testing or in a separate study in which representative users review the instructions and actually use the device or verbally describe what they would do based on what they read. The goal is to determine whether users can understand and follow the instructions and the extent to which the instructions for use supports the users safe and effective use of the device. If the device labeling is inadequate, it will be evidenced by subjective user feedback and maybe also by performance failures. Performance Data Validation testing should include objective (performance-based) evaluations of task success. The data should focus on high-priority tasks and on use errors that could result in harm to a patient or a user. The test plan should describe how test participant use failures were defined, identified, recorded and reported. You should investigate all performance failures by following up with the participant to determine how and why they believe the failure occurred.
Page 26 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation Observational data are also useful in assessing test participants adherence to protocol and proper technique, and especially in identifying and understanding the nature of problems experienced during device use. Close calls are instances in which a user experiences confusion, misinterpretation, difficulty, or error that would result in mistreatment or harm, but the user recovers and no actual performance failure occurs. Because close calls can be indicative of problems with the design of the user interface, they should be recorded if they are observable and discussed in the subjective data collection; this discussion might also reveal close calls that were not observable. 10.1.5 Interpretation of Validation Test Results and Addressing Problems Problems with the design of the device, labeling, or training requirements should have been identified and addressed prior to validation testing. When use problems do occur during validation testing, this usually indicates that the previous HFE/UE steps were not performed adequately. The root causes of problems identified during validation testing should be evaluated from the perspective of the test participants involved and direct performance data will support this determination. Data analysis should include subjective feedback regarding critical task experience, difficulties, close calls, and any task failures by test participants. Depending on the extent of the risk mitigation strategies required, revalidation may be necessary. You should address failures and difficulties associated with greater than minimal risk and attributable to the user interface by designing and implementing risk mitigation strategies and re-testing those elements to confirm their success at reducing risks to acceptable levels without introducing any new risks. Residual Risk It is impossible to make any device error-proof or risk-free. For this reason, some amount of residual risk will always remain. The fact that risk was identified from the results of validation testing does not necessarily mean that it is residual. True residual risk must be resistant to elimination or mitigation through any potential modifications to the user interface, accessories, labeling, or training. Failures or difficulties with use that have been determined to represent residual risk should be described, as well as whether or not failures that occurred were associated with the design of the device, its labeling, or the content or proximity of training, and the extent of the association. The analysis of residual risk should determine if design modifications are indicated or if not, the analysis should demonstrate the impossibility or impracticality of reducing these risks further and that the residual risk is outweighed by the advantages offered by the device. If design flaws that could have negative clinical impact on patients are identified, planning to address them in subsequent versions of the device is not acceptable.
10.2 Clinical Validation Testing

Due to the nature of some types of device use or use environments that may be
Page 27 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation particularly challenging or poorly understood, it might be necessary to validate a device under conditions of actual use in a clinical study. 4 These studies should follow the same general guidelines as simulated validation testing, described in Section 10.1. Validation under clinical conditions (clinical evaluation) will involve actual use conditions and should include representative users. The clinical environments that will be used in the evaluation should be representative of the actual use environment and the validation testing process should affect the clinical environment and use conditions as little as possible. Validation performed under clinical conditions should be preceded by appropriate simulated-use testing to ensure that the device is sufficiently well designed to be safe in actual use (to the degree afforded by simulated-use testing).
11. Documentation
Documenting your HFE/UE testing, risk management and design optimization processes provides evidence that you have adequately addressed the needs of the intended users and optimized the design of your device and therefore demonstrated that a new device is safe and effective for users. Submitting this information as part of premarket approval application (PMA) for a new device will facilitate the premarket review process, reduce the need for requests for additional information and directly support review of all HFE/UE relevant information contained in your submission. In addition, FDA staff may request human factors documentation with other submission types if: (i) submission of human factors information is required (for example, as a special control): (ii) submission of human factors information is recommended in a specific guidance for a device type and you cannot justify forgoing such testing; or (iii) on a for-cause basis if it is the least burdensome method to address FDAs concerns regarding human factors issues. If you have chosen to perform human factors testing for a non-PMA device, it need not be submitted to FDA unless it is specifically requested under one of the foregoing circumstances. All documentation related to such testing that is not required to be submitted to FDA should be kept in manufacturers files. An outline of the HFE/UE report to be submitted to FDA is shown in Appendix A.
12. Conclusion
The advantages of optimizing device design through application of HFE/UE extend beyond improved safety. Many device manufacturers have found that the application of HFE/UE in the design of their products reduces the need for modifications and costly updates after market introduction and offers competitive advantages. With increased safety, the likelihood of your incurring expenses associated with product recalls or liability is reduced; when HFE/UE approaches are used in the design of devices,
4
Clinical studies must comply with the Investigational Device Exemption requirements set out in 21 CFR 812.
Page 28 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation particularly if the perspective of users is taken into account, the overall ease of use and appeal of a device can simultaneously be enhanced.
Page 29 of 37
Appendix A
HFE/UE Report
A HFE/UE report included in premarket approval application (PMA) or as requested by FDA under certain circumstances (see Section 11, Documentation, above) should provide information pertaining to device use safety in summary form. The level of detail of documentation submitted should be consistent with the nature of the use-related hazards for the device. The report should highlight the major human factors considerations, issues, resolutions, and conclusions. When key portions of this information are contained in various parts of a submission, a comprehensive cross-reference should be provided to the specific and separate components of a HFE/UE evaluation. The information that should be included in the report is listed in Table A-1 and described in the text that follows.
Table A-1. Outline of HFE/UE Report
Sec. 1
Contents Intended device users, uses, use environments, and training

Intended user population(s) and critical differences in capabilities between multiple user populations Intended uses and operational contexts of use Use environments and key considerations Training intended for users and provided to test participants Graphical depiction (drawing or photograph) of device user interface Verbal description of device user interface Known problems with previous models Known problems with similar devices Design modifications implemented in response to user difficulties Risk analysis methods Use-related hazardous situation and risk summary Critical tasks identified and included in HFE/UE validation tests Evaluation methods Key results and design modifications implemented Key findings that informed the HFE/UE validation testing protocol Rationale for test type selected (i.e., simulated use or clinical evaluation) Number and type of test participants and rationale for how they represent the intended user populations Test goals, critical tasks and use scenarios studied Technique for capturing unanticipated use errors Definition of performance failures
2 3
Device user interface

Summary of known use problems
User task selection, characterization and prioritization
Summary of formative evaluations
Validation testing
Page 30 of 37

Test results: Number of device uses, success and failure occurrences Subjective assessment by test participants of any critical task failures and difficulties Description and analysis of all task failures, implications for additional risk mitigation
Conclusion
The <Name Model> has been found to be reasonably safe and effective for the intended users, uses and use environments. The methods and results described in the preceding sections support this conclusion. Any residual risk that remains after the validation testing would not be further reduced by modifications of design of the user interface (including any accessories and the IFU), is not needed, and is outweighed by the benefits that may be derived from the devices use.
Section 1: Intended Device Users, Uses, Use Environments, and Training

This section should include: A description of the intended user population or, if there is more than one distinct user population, each population; critical differences in capabilities between user populations (such as home vs. professional users who might use the same device to perform different tasks) should also be included; A summary of the operational context of device use (such as the user having been first trained on its use by a nurse, it is used in an operating room, or it is used differently for different applications) and its critical components such as set-up, maintenance, cleaning, reprocessing; A description of other conditions of use that might affect device use safety or effectiveness; A summary of the intended use environments (e.g., home, hospital, medevac vehicles) and the characteristics of those environments that could impact use (e.g., glare, vibration, ambient noise, etc.); any environments for which the device is unsuited should be stated; and A description of any training that users received during validation testing and how it corresponded to realistic training levels; a summary of the training provided (such as on DVD or other materials) may be appended to the report.
Section 2: Device User Interface

This section should include: A graphical depiction or depictions (e.g., a photo or line drawing) of the device user interface, including a depiction of the overall device, and all components of the user interface with which the user will interact (e.g., display and function screens, alarm speakers, controls, keypads, dedicated buttons or other interface features); A written description of the device user interface; A copy of the labeling materials that will be provided to the user with the device (e.g., instructions for use, user manual, etc.); and
Page 31 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation A summary of the operational sequence of the user interface consisting of user actions performed and resulting device responses, user alerting mechanisms, etc.
Section 3: Summary of Known Use Problems

This section should include the known use problems with previous models of the same device (as applicable) or problems with similar types of medical devices. The section should highlight any design modifications of the current device that were specifically developed in response to use problems in the field.
Section 4: User Task Selection, Characterization and Prioritization

This section should provide a summary of the risk analysis methods applied to the components of user interaction with the device, and the user tasks or use scenarios included in the validation test. It should include the method of identifying tasks or use scenarios that are essential for proper operation of the device and those that are most likely to be associated with use error that could cause clinical harm to the patient or the user. Critical tasks that are associated with hazardous use situations should be highlighted.
Section 5: Summary of Formative Evaluations

This section should include the formative evaluation methods used, key results of those evaluations and any modifications that were implemented to the user interface design in response to the results of the formative evaluations.
Section 6: Validation Testing

This section should include a synopsis of all activities conducted and the methods used in the validation testing and how the testing was designed to evaluate the HFE/UE considerations. A summary of how the testing was conducted, the test results, and a discussion of all performance failures and critical assessments by test participants should be included.
Section 7: Conclusion
The conclusion of the report should begin with a statement that the new medical device has been found to be adequately safe and effective for the intended users, its intended uses, and use environments. The conclusion should be supported by a summary of the methods and findings contained in the previous sections of the report with emphasis on the results of the validation testing. This section should discuss any residual risk that remained after analysis of validation test findings. If applicable, this section should provide a sound rationale that modifications to the user interface (including accessories, training, and labeling) would not further reduce risk, are not possible or not practical, and are outweighed by the benefits that may be derived from use of the device, as designed.
Page 32 of 37
Appendix B
Considerations for Determining Sample Sizes for Human Factors Validation Testing
Estimates of the number of HF/usability testing participants required to identify all problems that exist in a user interface 5 are based on a set of assumptions regarding: a fixed (and known) probability of encountering a problem, a uniform likelihood for each participant to encounter each problem, and the independence of the problems (that is, encountering one problem will not increase or decrease the likelihood of finding other problems). However, none of these assumptions reflects the real world. Most importantly, individual likelihoods of encountering a problem with a user interface vary considerably, depending on the users personal capabilities, knowledge and experience levels, nature of interaction with the device, frequency of task performance, attributes of the use environment and use conditions. The lower the chances of finding a problem (e.g., if the problem is subtle or the users are highly skilled), the more people you need to test to provide reasonable assurance that the problem will be identified. Faulkner (2003) conducted a study that collected empirical data from a sample of 60 individuals with varying levels of experience with computers in general and with the software used in the test specifically. The results suggested that a sample of 15 people was sufficient to find a minimum of 90% and an average of 97% of all problems with that software (see Table B-1). However, user interface problems may be more difficult to detect in medical devices than in software products, and given the variability of peoples chances of encountering problems and the differing types of device interactions for different user groups, it would be appropriate to test a cohort from all major user groups. If users with distinctly different characteristics (e.g., use responsibilities, age ranges, skill sets, or experience levels) will use the device, validation testing activities should include 15 from each major user group.
Table B-1. Percentage of Total Known Usability Problems Found in 100 Analysis Samples (Faulkner, 2003).
No. users 5 10 15 20 30 Min. % Found 55 82 90 95 97 Mean % Found 85.55 94.69 97.05 98.4 99.0 SD 9.2957 3.2187 2.1207 1.6080 1.1343 SE .9295 .3218 .2121 .1608 .1051
The most important aspect of sampling may be the extent to which the test participants correspond to the actual end users of the device, which requires that the manufacturer accurately identify and describe its user populations. For devices with multiple user
e.g., Virzi, 1992; Nielsen, 1993
Page 33 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation populations that have different personal characteristics, it may be advisable to test the maximum number of participants that budgets and schedules allow.
Page 34 of 37
Appendix C
References for Further Reading

AAMI/ANSI/ISO 14971:2007, Medical Devices Application of risk management to medical devices. International Organization for Standardization. Geneva, Switzerland. AAMI/ANSI HE75:2009, Human Factors Engineering Design of Medical Devices. Bahr, N., System safety engineering and risk assessment: A practical approach. Washington DC: Taylor and Francis; 1997 Chapanis, A., The business case for human factors in informatics. In: Shackel and Richardson (eds.). Human Factors for Informatics Usability. New York: John Wiley & Sons; 1994. Chapanis, A., Human factors in systems engineering. New York: John Wiley & Sons; 1996. Cooper, J., Preventable anesthesia mishaps: A study of human factors. Anesthesiology 1978, 49:399-406. Cooper, J., An analysis of major errors and equipment failures in anesthesia management: considerations for prevention and detection. Anesthesiology 1984, 60:3442. Dekker, S. (2011). Patient Safety: A Human Factors Approach. CRC Press. Dhillon, B. S. (2003). Human Reliability and Error in Medical System. River Edge: World Scientific Publishing Co. Pte. Ltd. Dumas, J., and Redish, J., A Practical Guide to Usability Testing. Ablex, New Jersey, 1993. Faulkner, L. (2003). Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. Behavior Research Methods, Instruments, and Computers, 35(3), 379-383. FDA, Design Control Guidance for Medical Device Manufacturers. March 11, 1997. FDA, Guidance for the Content of Premarket Submissions for Software Contained in Medical Devices. May 29, 1998. FDA, Quality system regulation (current good manufacturing practice (CGMP) final rule). October 7, 1996.
Page 35 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation Goodstein, L., Anderson, H. and Olson, S., Tasks, errors, and mental models. New York: Taylor and Francis; 1988. ISO/IEC 60601-1-8:2006, Medical electrical equipment Part 1-8: General requirements for basic safety and essential performance Collateral Standard: General requirements, tests and guidance for alarm systems in medical electrical equipment and medical electrical systems. Geneva, International Electrotechnical Commission. ISO/IEC 62366:2007, Medical devices Application of usability engineering to medical devices. Geneva, International Electrotechnical Commission. Kirwan, B., and Ainsworth, L.K., A guide to task analysis. London: Taylor & Francis Ltd; 1992. Kohn, L., and Corrigan, J., (eds.), Building a Safer Health System. Institute of Medicine (IOM), Committee on Quality of Health Care in America. Washington, D.C.: National Academy Press; 1999. Landauer, T. K. (1999). The Trouble with Computers: Usefulness, Usability, and Productivity. Cambridge: The MIT Press. Leape, L., Error in Medicine. Journal of American Medical Association, 1994 Dec; 21(3) 272. Meister, D., Human factors testing and evaluation. Amsterdam: Elsevier; 1986. MIL-STD-1472F, Change Notice 1, Human engineering. Washington, D.C.: Department of Defense, 12-05-2003. Nielsen, J. (1993). Usability Engineering. Boston: AP Professional. Norman, D., The Design of Everyday Things. New York: Doubleday; 1988. OBrien, T., and Charlton, S., Handbook of Human Factors Testing and Evaluation, New Jersey: Lawrence Erlbaum Assoc; 1996. Reason, J., Human Error. Cambridge, Mass: Cambridge University Press; 1992. Redmill, F. and Rajan, J. (1997). Human Factors in Safety-Critical Systems. Oxford: Butterworth-Heinemann. Rubin, J., Handbook of Usability Testing. New York: John Wiley and Sons, Inc; 1994. Salvendy, G. (ed.), Handbook of Human Factors and Ergonomics. New York: John Wiley and Sons, Inc; 1997.
Page 36 of 37
Contains Nonbinding Recommendations Draft - Not for Implementation Sanders, M., and McCormick E., Human Factors in Engineering and Design. New York: McGraw Hill; 1993. Shneiderman, B., Plaisant, C., Cohen, M. and Jacobs, S. (2010). Designing the User Interface: Strategies for Effective Human-Computer Interaction. (5th ed.). Menlo Park, CA: Addison Wesley. Trautman, K., The FDA and Worldwide Quality Systems Requirements Guidebook for Medical Devices. ASQC Press; 1997. Virzi, R.A. (1992). Refining the rest phase of usability evaluation: How many subjects is enough? Human Factors, 34, 457-468. Weinger, M. B., Wiklund, M. E. and Gardner-Bonneau, D. J. (2011). Handbook of Human Factors in Medical Device Design. CRC Press. Weinstock, C. B. and Goodenough, J. B. (2009). Towards an Assurance Case Practice for Medical Devices. Pittsburgh: Carnegie Mellon. Wickens, C. D., Gordon, S. E., and Liu, Y., An Introduction to Human Factors Engineering. Addison-Wesley (Longman Imprint): New York, NY; 1998. Wickens, C. D. and Hollands, J. G. (2000). Engineering Psychology and Human Performance (3rd Edition). Upper Saddle River: Prentice Hall.
Page 37 of 37

Applying Human Factors and Usability Engineering To Optimize Medical Device Design

Uploaded by

Copyright:

Available Formats

Applying Human Factors and Usability Engineering To Optimize Medical Device Design

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Applying Human Factors and Usability Engineering To Optimize Medical Device Design

Uploaded by

Copyright:

Available Formats

Contains Nonbinding Recommendations Draft - Not for Implementation

Contains Nonbinding Recommendations Draft - Not for Implementation

Contains Nonbinding Recommendations Draft - Not for Implementation

6. Analytical Methods for Identifying, Evaluating and Understanding Use-Related Hazards_________________________________________________________________ 15

Formative Evaluations ________________________________________________ 19

Clinical Validation Testing_______________________________________________ 27

Contains Nonbinding Recommendations Draft - Not for Implementation

Contains Nonbinding Recommendations Draft - Not for Implementation

Contains Nonbinding Recommendations Draft - Not for Implementation

Figure 2. Use-Related Hazards, Device Failure Hazards, and Their Intersection.

Figure 3: Addressing Use-Related Hazards in Risk Management.

Regulations, Guidance Documents, and Standards for HFE/UE

FDA Guidance Documents

Contains Nonbinding Recommendations Draft - Not for Implementation

National and International Standards

ANSI/AAMI/ISO 14971:2007 IEC 60601-1-8:2006

5. Device Users, Use Environments and User Interfaces

Contains Nonbinding Recommendations Draft - Not for Implementation

Device Use Environments

Device User Interfaces

Contains Nonbinding Recommendations Draft - Not for Implementation

Analytical Methods for Identifying, Evaluating and Understanding Use-Related Hazards

Identification of Known Problems

Analytical Approaches to Hazard Identification and Task Prioritization

Simulated Use Testing

Mitigation and Control of Use-Related Hazards

Design Verification Testing

10. Human Factors Validation Testing

10.1 Simulated Use Validation Testing

10.1.1 Tasks and Use Scenarios

10.2 Clinical Validation Testing

Contains Nonbinding Recommendations Draft - Not for Implementation

Contents Intended device users, uses, use environments, and training

Device user interface

Summary of known use problems

User task selection, characterization and prioritization

Summary of formative evaluations

Contains Nonbinding Recommendations Draft - Not for Implementation

Section 1: Intended Device Users, Uses, Use Environments, and Training

Section 2: Device User Interface

Section 3: Summary of Known Use Problems

Section 4: User Task Selection, Characterization and Prioritization

Section 5: Summary of Formative Evaluations

Section 6: Validation Testing

Contains Nonbinding Recommendations Draft - Not for Implementation

e.g., Virzi, 1992; Nielsen, 1993

Contains Nonbinding Recommendations Draft - Not for Implementation

References for Further Reading

You might also like