Electronic checklist support for disaster response

Krüger et al. Electronic Checklist Support for Disaster Response Electronic Checklist Support for Disaster Response Uwe Krüger Fabian Wucholt Institute of Computer Science Friedrich-Schiller-University Jena, Germany uwe.krueger@uni-jena.de Intercultural Business Communication Friedrich-Schiller-University Jena, Germany fabian.wucholt@uni-jena.de Clemens Beckstein Institute of Computer Science Friedrich-Schiller-University Jena, Germany clemens.beckstein@uni-jena.de ABSTRACT Requirements analysis of IT-support for rescue management showed that electronic checklist support is a vital function of any IT-based assistance system. Although checklists are a simple approach, their successful implementation and use depends on many factors. We nevertheless believe that Intelligent Electronic Checklist Support Systems (IECSS) are especially helpful for the (inter-) organizational cooperation in disaster scenarios like mass casualty incidents (MCIs). In this paper we describe why, when, and how electronic checklists can be used to coordinate the work of the geographically dispersed rescue forces. For this purpose we will have a look at safety-critical and complex tasks in aviation and medicine where checklists already are successfully used and try to profit from this experience for the MCI domain. Keywords Intelligent Electronic Checklist Support, Quality Management, Human Errors INTRODUCTION Studies in Human Factors Research have suggested that cognitive abilities decrease significantly as soon as the level of stress, mental and bodily fatigue, and emotional burdening increases. Task forces from authorities and organizations with safety responsibilities1 (in Germany called BOS) are subject to such negative influences in scenarios like mass casualty incidents (MCI), which may cause the occurrence of human errors: even apparently well-known work steps can be forgotten by the rescue forces or initiated several times (Reason, 1990). Task forces typically see themselves confronted with non-routine tasks which require fast, effective and far-sighted action. Though every MCI has its own unpredictable characteristics, there are basic steps which are common to all MCIs. In some areas (paper based) checklists (CLs) are used for controlling the correct execution of basic standard operation procedures (SOPs) steps. Requirements analyses of emergency management systems show that checklist support should be one of the core functionalities of any IT-based assistance system (see e.g., Lanfranchi and Ireson, 2009). Our work was motivated by the thesis: “No matter how expert you may be, well-designed checklists can improve outcomes” (Gawande, 2011). We therefore believe that an electronic checklist support system can significantly help the task forces to adhere to the standard-based and safety-critical work cycles (SOPs). Electronic checklists as we understand them are not about planning a fixed series of steps which have to be followed exactly by the rescue forces. Rather the task force should be able to act and decide flexibly at any time and have the opportunity to contribute their experiences to. Whether and when this help is needed, would depend on the discretion of the expert or the current workload during action. In this paper2 we suggest a transfer of the checklist paradigm from other safety-critical areas to the area of rescue organizations in MCI response. We first show how to successfully use checklists in safety-critical and com1 In the following called rescue organizations or short organizations. Our work was funded within the Federal Government's program Research for Civil Security (“rescue and protection of people”) by the German Federal Ministry of Education and Research (see: www.speedup.uni-jena.de). 2 Proceedings of the 9th International ISCRAM Conference – Vancouver, Canada, April 2012 L. Rothkrantz, J. Ristvej and Z. Franco, eds. Krüger et al. Electronic Checklist Support for Disaster Response plex work areas and describe first attempts of an electronic realization of paper based CLs (PCLs). Then we outline the role of checklists in rescue organizations. Next, we present the advantages and the potential of intelligent electronic checklist support systems (IECSS). We then show how to implement and integrate electronic checklists into the workflows of the rescue forces. MANAGING COMPLEXITY IN SAFETY-CRITICAL AREAS Safety critical areas are characterized by a complex work environment. Typical examples are aviation, nuclear industry or intensive care. Even small errors during critical process phases can then have fatal impacts. Anywhere where humans have to deal or handle with complex systems and situations, human errors are likely. In order to keep up with the critical work steps in a complex work environment, checklists are of value. In aviation, for example, “checklists are used to verify that certain critical procedural steps have been accomplished. Only procedural steps which, if omitted, would have a direct and adverse impact on normal operation are included” (Degani and Wiener, 2005). In the cockpit the application of electronic checklists (ECLs) was realized and evaluated for some types of aircrafts at the same time as modern “glass cockpits“ were introduced (see e.g., Boorman, 2001a). The experience gained from aviation, of course, is not exactly transferable to rescue organizations but, it nevertheless hints at basic principles regarding the design and the function of ECLs. An important by-product of the daily use of PCLs and ECLs in aviation are comprehensive observation results (see e.g., Degani and Wiener, 1990). These results have also revealed some problems in the interaction of humans and (electronic) checklists. PCLs (as well as ECLs) today are an integral part of the SOPs in cockpits. In recent years, PCLs are used increasingly in medicine (e.g., in intensive care units) as well and provide reminders to manage extreme complexity. They were evaluated to be exceedingly successful over a longer period of time by selected hospitals (see e.g., Gawande, 2011). Despite the impressive and positive effect of error minimization, paper based checklists are not considered helpful by all physicians. That is one reason why paper based checklists are not as common in hospital practice as in aviation. In the course of increasing computerization, the application of electronic checklist systems in clinical practice was discussed sporadically — primarily as a connection to the own clinical IT-systems. But it was shown, that due to the increasing complexity of such systems, their acceptance and usability typically decreases, causing the actual benefits of the checklist paradigm to become lost. CHECKLISTS FOR RESCUE ORGANIZATIONS Checklists are a highly discussed and controversial topic among experts of rescue organizations. Opinions differ to a large extent about what checklists exactly are and how and where to use them. We found that (in Germany) PCLs in operative-tactical areas occur only sporadically. If they exist, their application is investigated just a little or not at all. According to research in the human factor area, disadvantages in the use of checklists are also recognized (Grote, 2004). These includes: (1) a loss of flexibility in adapting to extraordinary situations, (2) a decrease of attention and a shrinking of the own “situation awareness” due to reliance on checklists, and (3) as well as a long-term reduction of problem-solving competence of checklist users, the so called “dependence problem”. In our studies we come to the conclusion that checklists could be a helpful support for two different user-groups in an MCI scenario: (1) Analogous to the operational areas mentioned above in safety-critical areas, checklists could be helpful for experts (i.e., special trained mission leaders), and primarily serve as a redundant protection level. (2) For less experienced forces an IECSS can be a useful help for coordination and standard-based task management. In an MCI situation it can happen, that non-experienced or untrained operation forces just do not exactly know what they have to consider and what exactly is expected from them. In the MCI context checklists rather represent an attempt of process control and shall give operation forces a guide to action, thus tell them what to do. User-group (1) has to be considered as critical because of the mentioned acceptance problems. A system which dictates what must be done (similar to a workflow) will be hardly accepted by experts of this community and probably leads to the rejection of the system itself. Because of the large number of variables, it also is not possible to provide a detailed workflow for every eventuality. Standard Operating Procedures versus Checklists Although often mentioned at the same time, “checklists” and “standard operation procedures” fall into two different categories. SOPs are standardized action processes, which represent a current best-practice of a particular working area. In every working environment SOPs can be identified with units of routine working steps. HowProceedings of the 9th International ISCRAM Conference – Vancouver, Canada, April 2012 L. Rothkrantz, J. Ristvej and Z. Franco, eds. Krüger et al. Electronic Checklist Support for Disaster Response ever, in many working areas SOPs are only implicit and passed on by learning processes of the work forces. For example, an experienced chief fire fighter knows what to do in an emergency and what the agreed and accepted standard procedures are in his fire department area. SOPs are a part of the organizational culture and sometimes are dictated by higher authorities or common regulations. Checklists for experts, however, primarily are auxiliary tools for remembering and controlling process steps. They usually do not represent every single step of a fullfledged SOP — they are only supposed to cover the most critical parts of a corresponding SOP. BENEFITS OF AN ELECTRONIC CHECKLIST SUPPORT SYSTEM Electronic checklist systems help to manage MCIs with a lower risk for human errors: they foster a more structured and situation-adjusted execution of the necessary steps and avoid the common problem of a missing situation overview. A carefully designed implementation of electronic checklists has many advantages over conventional paper based checklist: Surveillance and coordination of standard procedures: Each organization has its own vocabulary and its own structure of leadership. Since all organizations involved in an MCI rescue scenario are working together on one common problem, an overlap of work steps may occur at certain times and in some areas of operation. This can be illustrated by the example of an MCI with chemically contaminated patients. The fire brigade first has to decontaminate the wounded and only then can rescue services start to treat the patients. In order to appropriately manage those connecting points, the respective important work steps need to be identified, formalized and described in a way that allows supporting the synchronization with IT. Improve the self-contained measures: An IECSS can be used to collect and combine information from all the organizations involved — a feature that significantly enhances what communication researchers call collective situation awareness. With the help of such a system all involved rescue forcers can inform their colleges about all currently activated, running or completed checklists (or items). Automatic executing and monitoring: Connecting an IECSS to an existing information system, allows the system to automatically check certain items as finished once it recognizes that they are completed. For example, if a checklist item states that it is necessary to order a gear wagon for certain “dangerous goods“ (to find about the exact nature of those goods) then this item could be marked automatically as completed if another authority already placed such an order. Flexible and dynamic management structures: The content and design of a CL should match the experience and educational background of the user. For this purpose a system of CLs with content at a hierarchy of levels can be especially helpful. Hierarchical CLs can be used to present just the level of complexity the user currently is ready to take; even complex systems of CLs remain manageable: complex tasks can be summarized at higher levels or detailed by presenting lower levels as the situation or the operating force demands. Monitoring human tasks execution: Electronic CLs can help to reduce the chance for humans to make one of the following main mistakes when using paper based checklists: (1) forgetting what the current item is, and thereby inadvertently skipping an item; (2) skipping items due to interruptions and distractions; and (3) intentionally skipping an item and then forgetting to return to it (cf. Palmer and Degani, 1991). Automatic logging: An IECSS can automatically log all entries that the user checked off as being accomplished. These logs can then later be used to automatically produce a detailed documentation. They also provide the basis of a paper based backup for the case that the technical infrastructure becomes unavailable. Who noticed or edited what, and when can be recorded automatically. We are aware of the fact that this kind of surveillance may be perceived with discomfort and refusal by many rescue forces. APPLICATION ASPECTS AND REQUIREMENTS The SOP which belongs to a checklist usually is not explicitly available. Because no one can predict how an MCI may evolve. The operation forces only have a rough a priori idea about which working steps will be necessary to handle it and in which order. Hence, each rescue organization will have to create its own checklist philosophy, i.e., rules that govern how checklists are created, trained and applied in the organization. Our research suggests that the following four recommendations should be taken into account when organizations designs and implements (electronic) checklists: 1. Organizational aspect: It is necessary to exactly specify the user group of the checklist. Depending on the organization to which the checklist is addressed (police, fire brigade or emergency medical service); the design, the vocabulary used and maybe the execution strategy or working philosophy has to be customized. To ensure a standardized vocabulary for the formalization of checklist items, a formal specification of the terms and their relationships should be used (see, e.g., Wucholt et al., 2011). Proceedings of the 9th International ISCRAM Conference – Vancouver, Canada, April 2012 L. Rothkrantz, J. Ristvej and Z. Franco, eds. Krüger et al. Electronic Checklist Support for Disaster Response 2. Leading roles & personalization: Depending on the area of responsibility and the currently assigned role (e.g., chief operation manager), operation forces take over different tasks. As a consequence, they may be forced to assume leading roles, which are below their highest level of training or education. Depending on who is using it, a checklist should be tailored to the experience and education of this user. Further, CLs should be assigned to every potential role of every responsible mission leader. This includes even the first arriving rescue units. 3. Philosophy of use: How checklists are used varies from organization to organization. Their specific use is an outgrowth of each organization’s culture. Corporate culture includes many factors of the overall operational concept of the organization: traditional methods of operation, predefined work policies, management style, and delegation of responsibilities in the chain of command and control. Based on the experience from aviation there are two primary methods of running a checklist Challengeverification-response and Call-do-response (see Degani and Wiener, 2005). 4. Situation context: A checklist should also be adaptable to the current situation, i.e., the checklist system should intelligently select and present checklists as required by the situation. Triggers for this adaption may be factors like stress, environmental changes, motivation, acceptance or insecurity. A checklist should specify under which preconditions it has to be used. We are convinced that these recommendations can only be adequately realized in close cooperation between computer science and the humanities, and a specified focused to on (inter-) organizational culture issues (in our case Intercultural Business Communication). According to studies from other safety-critical areas, the application of checklists should be part of the safetyculture of the particular organization and must be trained in a suitable way. The better rescue organizations rate the advantage of using checklists in their domain, the better they accept those checklists for their daily work. This is especially true for attempts to get them to use electronic checklist systems (Hales and Pronovost, 2006). Concerning the above design aspects we identified three key requirements that should be satisfied by an IECSS: (1) the system should be easily configurable and customizable by the different rescue managers without special IT skills, (2) to handle complexity the system should support hierarchical CLs definitions in a flexible manner, and (3) the online operation of the system should be kept as simple as possible. FIRST PROTOTYPE Following these key requirements, we have developed a first prototype implementation to illustrate some significant features. Special data structures were invented that allow the operation forces to set up the organizational structure and the leading roles. In the current prototype users can define the CLs using a spreadsheet tool. This setup provided the starting point for the checklist design. At the heart of the system is a checklist-client, which kicks off the top level checklists for all the roles relevant to the mission. This top level checklist is displayed to the user once he assumes his role after having entered the mission scene. Once the top level checklist is chosen, the hierarchy will be generated automatically. Specify role Access to additional documents Main checklist instance Checking time Checked item Hide item Automatically generated checklist sub-hierarchy On-going time consuming background task Question mark button Figure 1. First prototype GUI of our IECSS follows the guidance: make it simple but flexible. There are usually two characteristics of an operational section structure, typically established when handling a mass causality incident: the corresponding command structure is hierarchical, and some task areas of the lower levels are optional. We suggest reflecting this by defining for every task area that is recognized to be relevant a Proceedings of the 9th International ISCRAM Conference – Vancouver, Canada, April 2012 L. Rothkrantz, J. Ristvej and Z. Franco, eds. Krüger et al. Electronic Checklist Support for Disaster Response checklist-template as the top level of the hierarchy for this task. Each checklist-template acts as a frame for all sub-checklists for a special working area responsibility and contains just items of a general nature. It is determined by the tasks (roles) relevant for the domain and must be determined by the rescue organization or the rescue forces themselves. Figure 1 shows the prototype of our checklist client as it may represent itself to the Chief Operation Manager, currently display a partially expanded and processed checklist hierarchy. Confirmed items marked “green” and annotated with the time they were checked. Only when all items of a sub-checklist are been confirmed, the root item will be marked “green” (check items) as well. Items which initiate a time consuming (background) task are marked “blue” until they are confirmed to have finished with success or failure. Items which have no relevance to the current operation can be hidden permanently with a click on the adjacent “cross”-button. On request hidden items can be brought back into attention. With the help of the question mark-buttons additional information for the respective item can be displayed. The display of stored, external resources (e.g., maps or figures) is possible by clicking the “gear”-button. This provides a convenient and context-related access to arbitrary additional information that may be relevant for handling the MCI. CONCLUSION AND FURTHER RESEARCH In this paper we outlined the role of checklists in rescue organizations (especially their relationship to SOPs) and assessed the potential of electronic checklists for the support of disaster response teams. As far as we know, the proposed basic concept of flexible linked electronic checklists is a new approach in the field of managing support for an MCI. For a system that automatically schedules and cancels checklists (-items), an appropriate monitoring system is necessary, which can rapidly react to changes in the operation area. For this purpose an information system is needed, which can collect and assess what is relevant in the current state of affairs. How this can be realized adequately is still a focus of our research as well as the questions how the system can automatically response to unexpected extern events. Initial feedback from users and researchers showed us the relevance of our approach but also revealed open problems with respect to user acceptance and a successful integration of the system into the existing IT infrastructure. Until now we have not yet realized all the potential benefits of electronic CLs. Nevertheless, further research will be aimed to question about how to realize inter-organizational CL-support and aspects about the field of cultural change by using checklists in everyday work processes. In addition, we need further studies regarding the achievement of user acceptance and user involvement in the definition of CLs for SOPs. REFERENCES 1. Degani, A. and Wiener, E. L. (1990): Human Factors of Flight-Deck Checklists: The Normal Checklist, NASA contractor report; NASA CR-177549, National Aeronautics & Space Administration, ARC. 2. Degani, A. and Wiener, E. L. (2005): Cockpit Checklists: Concepts, Design, and Use, Human Factors, 35(2), pp. i 28-43., San Jose State University. 3. Gawande A. (2010): The Checklist Manifesto: How to Get Things Right, New York, Metropolitan Books. 4. Grote Gudela (2004): Uncertainty management at the core of system design. Annual Reviews in Control, 28, 267-274. 5. Hales, B. M. and Pronovost P. J. (2006): The checklist - a tool for error management and performance improvement. In Journal of Critical Care, 21, S. 231-235. 6. Lanfranchi, V., Ireson, N. (2009): User requirements for a collective intelligence emergency response system. In: BCS-HCI '09: Proceedings of the 23rd British HCI Group Annual Conference on People and Computers, Swinton, 198-203, UK, British Computer Society. 7. Palmer, E. and Degani, A. (1991): Electronic checklists: Evaluation of two levels' of automation. Proceedings of the Sixth International Aviation Psychology Symposium (pp. 178-183). Columbus, Ohio. 8. Reason James (1990): Human Error. Cambridge, United Kingdom, Cambridge University Press. 9. Wucholt, F., Yildirim-Krannig, Y., Mähler, M., Krüger, U. and Beckstein C. (2011): Cultural Analysis and Formal Standardised Language - a Mass Casualty Incident Perspective. 8th International Conference on Information Systems for Crisis Response and Management, Lisbon, Portugal. 10. Palmer, E. and Degani, A. (1991): Electronic checklists: Evaluation of two levels' of automation. Proceedings of the Sixth International Aviation Psychology Symposium (pp. 178-183). Columbus, Ohio. Proceedings of the 9th International ISCRAM Conference – Vancouver, Canada, April 2012 L. Rothkrantz, J. Ristvej and Z. Franco, eds.