Ask Diana: A Keyword-Based Chatbot System for Water-Related Disaster Management

Abstract

This research developed a keyword-based chatbot system, Ask Diana, for water-related disaster management. Disaster management has been considered difficult and tedious due to the complex characteristics of disaster-related data. To deal with this problem, this research developed a chatbot system with a water-related disaster database, a user intent mechanism, and an intuitive mobile-device-based user interface. With such a system, users are able to access important data or information they need for decision making by directly asking the proposed chatbot or operating the image-based menus. The system was validated through a usability test and a six-month field test. The results demonstrated that Ask Diana can help related personnel access disaster data intuitively and develop corresponding response strategies efficiently.

1. Introduction

Effectively managing enormous data and precisely conveying that data to decision makers is considered a critical issue for disaster management, especially for water-related disasters. During the disaster response phase, the decision makers need to use observations, statistical data, and historical experience as references to form related strategies. The massiveness and diversity of disaster-related data make it difficult for the decision makers to accomplish their missions efficiently and accurately. For example, they may need to cross-reference dynamic observation data as well as data on static equipment preparation to prearrange flood prevention machines before a flood comes. To help decision-makers, many decision supporting systems for disaster management has been developed. However, the limited operating devices and the complex operation workflow have often constrained the performance of the system to help users deal with the enormous data.

Recently, applying Information Technology (IT) to develop decision support systems has become a trend. People can use IT to develop systems such as online websites, apps, or chatbots for decision support. Although, seemingly, utilizing IT for disaster management systems can bring many opportunities, the following issues should be resolved when implementing such technologies for the management of water-related disasters. Firstly, the system should contain a database, which contains sufficient data that the decision makers may need during the disaster response phase. Secondly, the system should have a mechanism to understand user intent that can analyze what data or information the user wants. Lastly, the system should have a friendly and intuitive user interface. During the emergency response phase, the decision maker needs to obtain the data accurately. A system with complex and tedious operation procedures may cut down the decision maker’s efficiency; on the other hand, a well-designed interface can help the user turn the data into useful information.

The objective of this research is to develop a decision support system using IT for water-related disasters. Although many decision supporting systems have already been developed recently, they are insufficient in gathering sufficient data, having an intuitive user experience, and revealing appropriate information. Therefore, in this research, a system with a database, an intuitive user interface, and a redesigned data revealing approach was developed for water-related disaster management.

2. Literature Review

Using data smartly and appropriately can greatly enhance efficiency and effectiveness in disaster management. However, due to the complex characteristics of disaster data, it is difficult for the related personnel to handle those data well. Therefore, using Information Technology (IT) to collect, process, and transfer disaster data has become a trend in disaster management. Recently, some research on using a chatbot system as a decision support system for disaster management has been conducted. The following literature review will sequentially describe the challenges of disaster data management, the applications of IT for disaster data management, and chatbot system examples.

2.1. Disaster Date Management

Disaster data management is considered an important role in disaster response phases. It is important to use the disaster data timely, accurately, and effectively. The characteristics of disaster data make it worthy of research and have been widely discussed in studies [1]. When disasters occur, the need to effectively handle enormous amounts of related data always follows [2]. A disaster event can be mainly divided into four phases: preparedness, response, recovery, and mitigation [3]. The disaster-related data can be used for prevention, advance alarms, early damage detection, and decision support. Using data properly during each phase of a disaster event can help the decision makers develop suitable strategies [4,5].

However, due to the characteristics of the disaster data, handling and applying the related data appropriately during disaster events is usually difficult for decision-makers. For instance, disaster data usually are not updated regularly, which makes the whole procedure of data update not only time-consuming but also labor-intensive [6]. Besides the data updating issue, the time sensitivity of disaster data is another challenge that needs to be coped with. In the disaster response period, decision makers may need data that should be updated dynamically without any delay to make decisions correctly and effectively [7]. Additionally, both intra- and inter-organization coordination is also a key point for disaster management because disaster data is typically provided from different organizations or different departments in an organization [8]. Thus, an integrated disaster data management system should be a system that allows for vertical and horizontal data flow [9].

In conclusion, a disaster data management system needs to integrate all related data, manage the data appropriately, and convey the data precisely. The system should also be able to deal with the characteristics of the disaster data: the massive amount of data, the time sensitivity, the vertical and horizontal coordination, and the abundant diversity.

2.2. Information Technology for Disaster Data Management

In order to provide decision makers or related personnel with adequate and correct information during the disaster response phases, studies implementing Information Technology (IT) have been conducted [10]. Recently, an evolution in the use of information has been triggered for storing and handling enormous data [11]. Such an evolution of technology brings many opportunities to the disaster data management. Researchers have tried to apply IT for disaster management, such as managing massive data through an integrated database, coping with spatial data through Geo technologies, or developing a decision support system through the Internet of Things (IoT) techniques.

For instance, the Hong Kong Observatory (HKO) constructed a database, which uses the PDF file format to display information on historical typhoon events that occurred between 1968–2016 [12]. Chen et al. (2016) developed a Lesson Learned Center, which provides disaster response reports and news of all the historical disaster events that happened in Taiwan for the decision makers’ reference [13]. As for dealing with the spatial data of disasters, Montoya developed an urban disaster data collection method based on remote sensing, global positioning systems, digital video, and geographic information systems [14]. Dominey-Howes conducted research that focused on analyzing the potential value of implementing historical documents and geological records of Tsunamis in the Aegean Sea Region of Greece for disaster management [15]. Chang et al. tried to utilize the state-of-art machine learning technology for an intelligent regional flood inundation warning system [16].

In addition to the effective design and completeness of a database, a decision support system that provides a channel for the user to access the data or information is another critical issue. Due to the development of IT, many decision support systems based on IT for disaster management have been developed. For instance, the spatial decision support systems (SDSS) with early warning, rapid response, and loss estimate functions for earthquake disaster management were discussed in Nyimbili’s research [17]. Papadopoulou et al. (2011) proposed a paper that aims to discuss the role of SyNaRMa (System for Natural Risk Management), a web-based GIS system for natural disaster management, as a tool to facilitate disaster management [18]. Irwin et al. proposed a decision support tool for estimating the resilience of urban systems [19].

Several decision support systems have been developed for water-related disaster. For instance, Tsai et al. (2017) proposed a qualitative information delivery system for elevating the efficiency of internal information delivery of the government agency [20]. T-search is a decision support system that uses a route matching algorithm to allow the user to search historical typhoon event data, such as response reports, typhoon information, news, and so on [21]. Horita et al. (2015) integrated the data from Wireless Sensor Network and Volunteered Geographic Information to develop a spatial decision support system for flood risk management [22]. PFRA is an assessment tool for pluvial flood risk. The tool collects data about land cover soils, microtopography, and projected hydro-meteorological conditions for identifying the spatial and temporal distribution of flood risks in newly developed areas [23]. Lin et al. (2018) developed a platform with a combination of artificial intelligence and cross-sourcing [24]. The system plays a role in filtering the enormous disaster response data during the disaster. By doing so, people can be involved in the disaster response activities and help the government to organize the response data.

Even though many decision support systems have been developed, the user interface and the data retrieval approach of these systems seem unintuitive and inconvenient for emergency operations. The user interfaces were usually designed for application with personal computers. Furthermore, the operation of the systems was designed for users who are well-trained in computer use. Therefore, studies have recently started to focus on developing systems based on mobile devices or tablets [25], such as chatbot systems.

2.3. Chatbot for Disaster Data Management

A chatbot is a conversational system that allows the user to interact with the machine through natural language [26]. Systems using text communication to do human-machine interaction were first developed by Weizenbaum in 1966. He developed a dialogue system, ELIZA, to imitate a psychotherapist [27]. Nowadays many chatbot systems with different purposes were developed [28]. For instance, Lu et al. (2006) developed a chatbot system as an online coach for students in an English learning environment. They integrated a Natural Language Processing (NLP) tool and an Artificial Intelligence Markup Language (AIML) to build a chatbot that was able to provide instant messages for students. They believed that using such an instant messaging-based chatbot could provide an intelligent learning environment for students [29].

With the advance of the mobile device and online social networking technology, applying a chatbot or dialogue-based system for emergency situations or disaster prevention has recently become a trend. Rescue is a chatbot that can help users quickly report an emergency situation and the user’s location to pre-set emergency contacts by tapping a few buttons on a mobile device [30]. Richter is a chatbot that contains educational materials, preparation materials, and resources for both earthquake and tsunami disasters. It allows a user to learn about and prepare for an earthquake or a tsunami by chatting with a machine [31]. 911bot is a Facebook Messenger-based application, which allows the user to report emergencies to relevant authorities. It provides an interface with image-based buttons so that the user can easily report the emergency by clicking image-buttons on Facebook Messenger [32]. In Taiwan, the National Science and Technology Center for Disaster Reduction (NCDR) cooperated with LINE Taiwan to develop an official LINE account. The LINE account is actually a chatbot system, which provides immediate alarms for earthquakes, floods, typhoons, etc. [33].

3. Research Objectives

This research aims to develop a decision supporting system for water-related disasters and consider Taiwan as the target area. The system was coined as Ask Diana, which can manage the enormous disaster-related data and convey the effective information to the decision maker precisely. The developed system should achieve the following goals:

• Ask Diana should have a database that contains sufficient data that meet the requirements of decision makers for water-related disasters.
• Ask Diana should offer a user intent mechanism that can automatically analyze what data or information the user wants.
• Ask Diana should provide an intuitive user interface with simple operating procedures for the decision maker to make timely decisions.

4. Ask Diana Methods

This research proposes a chatbot system, named Ask Diana, for the support of water-related disaster decision making. The name of the system is the concept of what it is able to do. The D of Diana represents “Disaster.” Diana is an elegant female name with a calm and bright meaning. “Ask Diana” means the user can ask whatever they want to know about a disaster and the system will answer calmly and professionally. Ask Diana utilizes Information Technology to manage the enormous data and to convey the data to decision makers. The chatbot system mainly contains three parts: a database for water-related disasters, a user intent understanding mechanism, and an intuitive user interface. The following sections will sequentially describe the system overview, the water-related disaster database, the user interface, and the user intent understanding mechanism.

4.1. System Overview

Ask Diana mainly consists of three modules: the water-related disaster database, the user intent understanding mechanism, and the user interface. First, the database gathers both the dynamic data and the static data that relates to water disasters. The data would be stored in the database and be managed and classified through a decision tree. Secondly, being a chatbot system, the user interface of Ask Diana was designed based on the characteristics of chatbots. Additionally, the data revealing method was also designed to convey effective information on a mobile device clearly. Lastly, the user intent understanding mechanism is to distinguish what data or information the user wants. This mechanism includes a fuzzy search algorithm to analyze the text that the user inputs and a keyword table to match the inputted keywords with the information stored in the database.

Figure 1 illustrates the system structure of Ask Diana. In Ask Diana, the user can search the data by inputting keywords or clicking the menus/buttons that Ask Diana provides. If the user uses menus to search the data, Ask Diana will match the input with the keyword table. On the other hand, if the user uses text to search the data, Ask Diana will first analyze the input through a developed fuzzy search algorithm and then match the results with the keyword table. The matching results of both input methods will be compared with the decision tree to see if what the user wants is the data or another menu to browse for more information in Ask Diana. If Ask Diana finds that the user wants another menu, it will then provide the user the menu for further searching. Otherwise, the system will enter the data processing phase and reveal the data as the search result.

4.2. Water-Related Disaster Database

As a decision supporting system for water-related disasters, Ask Diana should have a database that provides all the data that the decision makers need. To do that, five decision makers from the Taiwanese government were interviewed to discover what data they will need for decision making. Moreover, since the research team has been the government’s decision supporting staff for more than 11 years, this research also combined this experience and all the reports that have recorded, based on user demand, to define a decision tree. That decision tree may be used for managing and classifying all the data.

4.2.1. User Demands Definition

In order to define user demands, five decision makers of the Water Resource Agency (WRA) in Taiwan were interviewed. Two of the subjects (user codes W and C1) were deputy ministers of the WRA. One (user code C2) was the deputy chief engineer of the WRA. K was the director of the Water Hazard Mitigation Center of the WRA. The last one (user code Y) is the senior engineer of the Water Hazard Mitigation Center of the WRA.

Table 1. The results of the user demand interviews.

User Code	Position in WRA	Demands for Support System
W	Deputy minister	Weather observation data, statistical preparation data, disaster prevention operation data
C1	Deputy minister	Weather observation data, statistical preparation data
C2	Deputy chief engineer	Weather observation data, disaster prevention operation data
K	Director of the Water Hazard Mitigation Center	Weather observation data, historical experience data, international disaster data
Y	Senior engineer	Weather observation data, statistical preparation data, international disaster data

illustrates the results of the user demands interview. Since those decision makers were in charge of different tasks in disaster prevention, the data that they needed were slightly different. W, a deputy minister of the WRA, needed the decision supporting system to provide him weather observation data such as the rainfall data, statistical preparation data such as the equipment preparation status, and disaster prevention operation data such as meeting slides during the disaster. C, also a deputy minister of the WRA, needed weather observation data such as meteorograms and rainfall data and statistical preparation data such as drought resistance equipment. The deputy chief engineer of the WRA (user code C2) indicated that he wanted a decision support system that could include weather observation data and disaster prevention operation data for his daily tasks. Besides weather observation data, the director of the Water Hazard Mitigation Center (user code K) would also need historical experience data such as disaster event reports or news and international disaster information. The senior engineer (user code Y) mentioned that he would need weather observation data, statistical preparation data, and international disaster data.

After the interviews, the required data could be classified into five main categories: weather observation data, statistical preparation data, disaster prevention operation data, historical experience data, and international disaster data. Parts of these are real-time data that are provided by other agencies or institutions. Some are static data that have low update frequencies. For the real-time data from other institutions, Ask Diana should include application programming interfaces (APIs) to interface with those data. For the low update frequency data, a well-organized database to store and manage the data should be built.

4.2.2. Decision Tree

To classify and manage the enormous water-related disaster data, this research built a decision tree. A decision tree is a decision supporting tool that utilizes a tree-shaped model to perform classifications. In a decision tree, each node of the tree represents an attribute of the data; each leaf of the tree represents a single piece of data. The path from the root to the leaf represents the classification rule. Based on our more-than-eleven-year decision supporting experience, this research utilized data use phases, data use purposes, and data attributes to build the decision tree model for Ask Diana.

Figure 2 is the decision tree model of Ask Diana. This research first separated the data based on whether the data are weather related to classify the weather observation data. The rest of the data are categorized into disaster prevention data, disaster operation data, and disaster discovery data based on which phases they would be used during the disaster response period. The data for which their usage phase cannot be defined will be categorized as other data.

In the weather observation data, due to the data’s characteristics, they can be separated into the weather reports and meteorograms. The disaster prevention data can be classified into flood prevention and drought prevention. The disaster operation data can be classified into real-time alarm data, response operation data, and meeting files or records. The disaster recovery data can be classified into disaster recovery data, disaster response reports, and historical response experience.

By building the decision tree model, the water-related disaster data can be classified into five main categories and 10 sub-categories. The decision tree model could be utilized to manage the data. The user may be allowed to access the data from the root of the tree or from every node of the tree. Furthermore, menus for both the root and the nodes should be designed so that the user can access the data by clicking buttons in those menus.

4.3. User Interface

To provide decision makers an intuitive and easy operating procedure, a user interface based on mobile devices was designed. The interface of Ask Diana mainly contains three panels: data revealing panel, menu panel, and text input panel (Figure 3).

The data revealing panel is dialogue-based. The user input and data would all be displayed in this panel. Through a dialogue-based data revealing method, the user can access the data as if they are actually asking a person named Diana. Additionally, the data revealed in this panel will be visualized or summarized for the purpose of turning the data into effective information.

Ask Diana provides a menu panel so that the user can browse data by clicking the menu in the touch panel of the mobile device. Menus with buttons would be designed based on all the nodes on the decision tree. By doing so, the user is able to access data from the root to the leaves of the decision tree.

As a chatbot system, Ask Diana also provides a text input panel to allow the user to search the data by inputting keywords. The keywords inputted by the user would be analyzed by the user intent understanding mechanism. The mechanism would automatically match the user intent with the decision tree to check if the user is asking for another menu of the decision tree node or leaf.

4.4. Intent Understanding Mechanism

To analyze keywords that the user inputs, this research designed a user intent understanding mechanism. In the mechanism, this research developed a fuzzy search algorithm and constructed a keyword mapping table. The fuzzy search algorithm analyzes the user’s intent. The analysis result would be matched with the keyword mapping table to find suitable results for the user. After matching the user input with the mapping table, the system would determine whether the user wants to reach a node or leaf of the decision tree. If the user’s intent is to find a node, then Ask Diana would provide the user with a related menu for further operation. On the other hand, if the user wants a leaf, then the system would provide the corresponding data for the user.

4.4.1. Keyword Mapping Table

In order to match the user intent with the decision model, this research designed a handcrafted keyword mapping table. The keyword mapping table should contain all nodes and leaves in the decision tree. All data in the mapping table may have four values: data name, label, keyword, and data location. The data name is for indicating the name of the data like three-hour cumulative rainfall, pump distribution status, etc. The label value is used for determining whether the data represents a node (menu) or a leaf (data) in the decision tree. Each data would be given a main keyword. Taking the preparation status of large-scale pumps as an example, the main keyword may be “large-scaled pump distributions.” The data location should represent the data location in the database so that the system can find the corresponding data and display it to the user.

4.4.2. Fuzzy Search Algorithm

This research developed a fuzzy search algorithm to understand the user’s intent. The algorithm would match keywords from the user inputs with the keyword mapping table by calculating the similarity of strings. In the algorithm, due to the characteristics of the languages, a single word is regarded as a token for English inputs and a single character is used as a token for Chinese inputs. The similarity, $sim\left(d_j,q\right)$, is calculated by comparing the number of similar tokens in the input text and each keyword in the mapping table using Equation (1).

$$sim\left(d_j,q\right)=\frac{{{\displaystyle \sum }}_{i=1}^t{f}_{ij}}{t}.$$

(1)

In the equation, $d_j$ is the j-th main keyword from the keyword mapping table; q is the text inputted by user inputs; t is the number of tokens in the inputs; and $f_{ij}$ is the count of the appearance of the i-th token of the user input in the j-th main keyword from the mapping table. If the calculation result is 1.0, Ask Diana would provide the corresponding data to the user. For situations where the result is less than 1.0, a constant parameter k was set for comparison. The system would list all the data that have scores less than 1.0 and higher than k. If the calculation results are all less than k, then Ask Diana would tell the user to change their keywords.

Algorithm 1 illustrates the developed fuzzy search algorithm, FUZZY SEARCH. In the algorithm, the token array that the user inputs, Q, is the input and the corresponding data that meets the user intent, A, is the output. First, the token array Q is assigned to q; afterward, Q is processed by changing synonyms and removing pleonasms. For instance, WRA may refer to the Water Resource Agency. Words such as “I like to,” “Hey,” and “Hi” would be regarded as pleonasms and removed. Then, the similarity of q and each keyword in the mapping table would be calculated and stored in an array, sim. After calculating sim, the value would be used to determine what data the system should provide to the user. If there is any value in the sim that is equal to 1.0, then all the corresponding data would be regarded as data meeting the user’s intent. If there is no value in sim equal to 1.0, then those corresponding data that have values in sim higher than a constant value k would be regarded as the output. If there is no value higher than k, then Ask Diana would let the user know that there may be no corresponding data in the system.

Algorithm 1 Fuzzy search algorithm

FUZZY SEARCH

Input:Q: the text array that the user inputs

Output:A: the corresponding data that meet the user intent

1: while input text array, Q do

2:    q$\Leftarrow$ synonyms changed, and pleonasms removed from Q

3:    for j from 1 to N then     //N is the number of keywords in the mapping table

4:      $sim\left(j\right)\Leftarrow \frac{{{\displaystyle \sum }}_{i=1}^t{f}_{ij}}{t}$    //t is the number of tokens in q

5:    for l from 1 to N

6:      If there is $sim\left(l\right)=1.0$ in $sim\left(d_j,q\right)$ then

7:       A$\Leftarrow$ the l-th data in the mapping table

8:      else if there is sim(l) < 1.0 & sim(l) $\ge$ k then

9:        A$\Leftarrow$ the l-th data in the mapping table

10:       else

11:       A$\Leftarrow$ “no corresponding data in the system”

12:       end if

13:      return A

14: end while

5. Implementations

The Ask Diana method was implemented by considering Taiwan as the target area and the Water Resource Agency (WRA) as the target user. Due to the location and weather, the country suffers from typhoons and floods annually. A total of 387 typhoons occurred in Taiwan from 1986 to 2018 [34], which make it urgent to have a state-of-art decision support system.

For the implementation, the Water Resource Agency of Taiwan was considered as the target user to construct the decision tree and the water-related database. In the Water Resources Agency, the decision makers are now used to using the LINE Group to deliver information and give orders to their subordinates. Therefore, the user interface was designed based on the specifications of mobile devices and LINE. The user intent understanding mechanism was realized using hypertext preprocessor (PHP) scripts.

The system was developed based on PHP scripts and utilized LINE, a commercial messaging app, as the platform. The following sections will sequentially describe the developed Ask Diana and its operating procedures.

5.1. Water-Related Disaster Database for Ask Diana

For the water-related disaster database, this research collected data based on the results of the interviews. Ask Diana collects a total of 142 kinds of data containing weather observation data, statistical preparation data, disaster prevention operation data, historical experience data, and international disaster data. All the data are from National Taiwan University (NTU), the Central Weather Bureau (CWB) of Taiwan, WRA, and open data from the government of Taiwan (Figure 4). The data was stored in the database as PDF files, text data, images, and URIs. In order to dynamically parse the data from different agencies, three APIs have been developed. The APIs are in charge of parsing the dynamic data such as rainfall data from CWB, river level alarms from the WRA, and earthquake reports from the government’s open data.

In the water-related disaster database module, a decision tree was built for managing the massive amounts of data. The decision tree contains a total of 212 nodes including 142 leaves. This research used the phases of a disaster event that the data would be used for to categorize them into five categories: weather observation data, disaster prevention data, disaster operation data, disaster recovery data, and other data.

Table 2. The data distribution in the decision tree.

Main Category	Number of Leaves (Data)	Number of Branches (Menus)	Number of Totally Nodes (Data + Menus)
Weather observation	33	15	48
Disaster prevention	10	8	18
Disaster operation	44	20	64
Disaster recovery	10	6	16
Other	45	21	66
Total	142	70	212

shows the data distributions in the decision tree. Weather observation data contains 33 kinds of data in 48 nodes. Disaster prevention data contains 10 kinds of data in 18 nodes. Disaster operation data contains 44 kinds of data in 64 nodes. Disaster recovery data contains 10 kinds of data in 16 nodes. Other data contains 45 kinds of data in 66 nodes.

5.2. User Intent Understanding Mechanism for Ask Diana

The developed user intent understanding mechanism was implemented on Ask Diana through PHP scripts. The mechanism contains the keyword table and the fuzzy search algorithm. For the keyword table, 212 keywords were built in the table based on the decision tree of Ask Diana. Since Ask Diana was implemented in Taiwan, the language system is Chinese. For the fuzzy search algorithm, this research regarded each character as a single token instead of using a single word, because there is no space in a sentence to segment words in Chinese. The constant value k in the fuzzy search algorithm was set to 0.5.

The PHP scripts were built on an AWS server. The LINE Messenger API, which is provided by LINE, will send user inputs to Ask Diana’s server whenever the user inputs text on their mobile device. When the server gets the user inputs, the mechanism will start to analyze the input text, and then send the suitable data back to the LINE Messenger API to show them to the user. If only one data in the mapping table get a similarity score of 1.0, Ask Diana returns the related data to the user. Due to the restrictions of LINE, if more than one data item gets a similarity score of 1.0, then Ask Diana will return four related data for the user to choose from. If none of the data has a similarity score of 1.0, the system returns the top four data items with similarity scores that are higher than 0.5 for the user to choose from (Figure 5).

5.3. User Interface for Ask Diana

In order to let the user browse Ask Diana’s data by clicking buttons on the mobile device, this research designed three kinds of image-based menus based on the LINE Messenger API’s specifications: main menu, image-map menu, and carousel menu (Figure 6). This research designed menus for each node of the decision tree. With the designed menus, the user may access data from the root of the decision tree to the leaf. Additionally, when the user clicks the image-based buttons, Ask Diana will not only provide the related data or menu, but it will also return the keywords of the data or menu. Therefore, the user can learn the keyword(s) of the data or menu so that they can simply key-in the keyword(s) to access the data or menu instead of clicking the buttons step-by-step.

In addition to the image-based menu, this research also post-processed 23 kinds of data with a data visualization technique. Ask Diana is a chatbot system built on mobile devices. The original data revealing approach of those 23 kinds of data was designed for personal computer users. They usually contain too many words or unclear images with the wrong size such that the user may not be able to acquire important information efficiently using a mobile device. To deal with this issue, the way to reveal those 23 kinds of data was redesigned by utilizing simplified figures, icons, maps, and tables (Figure 7). Ask Diana will return those data as images so that the user can not only acquire the important information quickly but also forward the data to others on LINE easily.

6. Validation

To verify the accessibility of Ask Diana in water-related disaster decision support, a usability test and a field test were conducted. In the usability test, five scenarios were designed for real decision support staff at the Water Hazard Mitigation Center of Taiwan to test if Ask Diana can be implemented for their responsibilities. For the field test, the research team have cooperated with the WRA and have implemented Ask Diana for actual water-related disaster response activities since 1 January 2018.

6.1. Usability Test

For the usability test, five scenarios with five tasks were designed. Ten subjects participated in the usability test. The time necessary for the subject to finish each task was recorded as the test result. The following sections will describe both the designed usability test and the results of the usability test.

6.1.1. Design of Usability Test

Five scenarios were designed: four for test functions of Ask Diana and one for user’s feedback.

Table 3. Designed tasks for usability test.

Scenarios	Task
Scenario 1: Become a LINE friend with Ask Diana	Use the QR code or input the LINE ID (@askdiana) of Ask Diana to add Ask Diana as a LINE friend.
Scenario 2: Check the current drought resistance abilities of the WRA.	According to the Central Weather Bureau’s reports, the expected rain this winter may be less than previous years. The media is now very concerned about the current water situation. Use Ask Diana to check the water storage rate of the Zengwen Reservoir and the preparedness of artificial rain boosters.
Scenario 3: Check if it is necessary to open the second leveled central flood response team.	Due to the continuous increase of the Northeast Monsoon, the director of the WRA asks his decision support team to judge if the WRA should open the second level central flood response team. Use Ask Diana to find the opening standards and check if the current situation matches the opening standards.
Scenario 4: Check the future rainfall situations.	The Central Weather Bureau continuously issues heavy rain reports. Due to the increasing of the Northeast Monsoon, an extremely heavy rainfall may occur in the next days. Use Ask Diana to check the rainfall situations in the following days using QPF.
Scenario 5: Other suggestions for Ask Diana.	In order to enable Ask Diana to provide disaster prevention information more effectively during the disaster prevention and response phase, please share the experience of today’s test and provide your suggestions for us.

shows the content of each scenario. First, the user was asked to add Ask Diana as their LINE friend by inputting the LINE ID or scanning the QR code of Ask Diana. Secondly, the user was instructed to check the drought resistance abilities of the WRA. Scenario 3 was designed to test if the user was able to use Ask Diana to check the operation standards and to help them make related response strategies. The user was asked to use Ask Diana to link to another website, QPF, which provides future rainfall forecasting results, for scenario 4. Finally, in Scenario 5, the subjects were asked to write down their suggestions for Ask Diana.

Ten subjects were involved in the usability test. These 10 subjects were all staff at the Water Hazard Mitigation Center of the WRA. They were all in charge of decision supporting work during the disaster prevention and response period. Whether the subject passed the task or not and the time required for the subject to finish the task was recorded. In addition, the suggestions that the users provided were also recorded and analyzed.

6.1.2. Usability Test Results

After the test, the results were analyzed in terms of two indexes: the operating time and the pass rate.

Table 4. The usability test results.

Scenarios	Operating Time (secs)										Ave. (mins)	Pass Rate (%)
	A	B	C	D	E	F	G	H	I	J
Scenario 1	20	20	15	10	30	60	60	177	0 *	0 *	0.7	100
Scenario 2	120	240	204	221	210	510	Fail	385	120	60	3.8	90
Scenario 3	280	Fail	Fail	337	210	Fail	Fail	Fail	Fail	Fail	4.6	30
Scenario 4	540	480	318	380	480	450	Fail	677	242	460	7.5	90
Total (secs)	960	740	537	948	930	1020	60	1239	362	520
Total (mins)	16.0	12.3	9.0	15.8	15.5	17.0	1.0	20.7	6.0	8.7	12.2

* I and J had added Ask Diana as their LINE friends before the usability test.

illustrates the usability test results. In the tests, the subjects spent 12.2 minutes on average to complete all the tasks. As for the pass rate, all subjects were able to finish Scenario 1, 90% subjects were able to finish Scenario 2 and Scenario 4, and only 30% subjects were able to finish Scenario 3.

The results showed that most subjects were able to successfully finish Scenario 1, 2, and 4, except for Subject G. Subject G failed to finish Scenarios 2, 3, and 4. According to the suggestions that Subject G gave, he was not familiar with smart devices, such as mobile phones or tablets. Therefore, the tasks were too difficult for him to finish. For Scenario 3, among the 10 subjects, only three were able to finish this scenario. The reason may be that Ask Diana could only provide one piece of information at a time. Task 3 was designed to test if the user was able to use Ask Diana to look for multiple pieces of information for their decision support. Based on the test logs, the users tried to use Ask Diana to get multiple pieces of information at one time. However, since Ask Diana could only find one piece of information at a time, asking for multiple pieces of information may cause the system to fail to get the related data for the user.

For Scenario 5, all subjects were asked to write down their suggestions for Ask Diana. The remarks could be organized into three main suggestions:

• It will be more useful if Ask Diana can deal with Natural Language. Ask Diana used a keyword mapping table to understand the user’s intent. This caused the understanding process to have poor performance when the user input was long and colloquial.
• Ask Diana should be able to provide multiple pieces of information at one time. Instead of retrieving all the requested information simultaneously, the users had to split the input into fundamental queries to retrieve one part of the information at a time. This would cause unnecessary time wastage and even failure when users try to look for multiple pieces of information using Ask Diana.
• The data revealing approach should be modified to match the specifications of mobile devices. Ask Diana provided some data by giving the URL of the original provider. However, some of the provider’s websites did not have a Responsive Web Design (RWD) for mobile devices. This may cause the user to have trouble reading details of data on mobile devices (Figure 8).

6.2. Field Test

For the purpose of a real case study, the research team cooperated with the government of Taiwan (Water Hazard Mitigation Center) and have implemented Ask Diana for actual disaster response activities for six months (1 January 2018 to 28 June 2018). All the user logs were recorded for further analysis. Based on the user logs, 381 users have added Ask Diana as their LINE friend for water-related disaster response activities. It has already collected 12,078 records from actual users. Figure 9 illustrates the cumulative usage graph of Ask Diana. In the figure, two obvious slope changes occurred in early April and early June. The first slope change occurred at the beginning of the flood season of Taiwan. The second slope change occurred when the 0611 flood emergency response event started. According to the user logs of Ask Diana, the decision makers and the decision support staff of the WRA would frequently use Ask Diana to look for weather observation data and disaster prevention data. Additionally, both the director of the WRA and the director of the Water Hazard Mitigation Center of the WRA, the two main decision-makers in the WRA, used Ask Diana to get water-related disaster data every day.

In addition to Ask Diana’s large and daily fixed usage, the results of the case study also revealed that it mainly has three advantages for effective response activities.

• Ask Diana successfully elevates the accessibility of water-related disaster data. Ask Diana provides a database, which contains a variety of water-related data. With such a database, the user can easily access disaster data for decision making by simply inputting the keywords of the data.
• Ask Diana is able to improve the efficiency of data transformation during the disaster response period. In the past, when decision makers needed disaster data, they would ask their support staff to collect and organize information for them and wasted much time waiting for support staff responses. Being a chatbot system, Ask Diana is able to provide effective data or information to decision makers immediately.
• Ask Diana provides a platform that allows the user to access water-related disaster information easily. Built on LINE, a popular commercial communication platform in Taiwan, the user does not need to spend time learning how to operate a new platform. Instead, they only need to add Ask Diana as their LINE friend and ask Diana for data or information they want. Based on the observations during the field test, the operation process is more intuitive than the agency’s original system, which made the personnel have more willing to use Ask Diana. Furthermore, revealing data on LINE allows the user to easily forward the data to whomever they want for further discussion.

7. Discussions

In this research, a water-related disaster chatbot for decision support was developed. Through a usability test and a six-month field test, the developed system was validated to greatly contribute to disaster decision support. However, some limitations still exist when using Ask Diana for disaster response.

7.1. Contributions

This research proposed a chatbot system for water-related disaster decision support and made efforts in the following aspects:

• Sufficient data for water-related disaster decision support: By conducting expert interviews, this research have designed a database with a decision tree for water-related disaster. Without such a database, staff from related agencies need to collect data from different resources. This would be a time-consuming and labor-intensive process. However, with Ask Diana, decision-makers are able to get all the data or information they need immediately for decision making through a chatbot system.
• Highly efficient data transfer: As a chatbot system with a user intent understanding mechanism, Ask Diana allows the user to get data by inputting keywords. In addition, Ask Diana is built on a commercial communication platform, users can easily forward the data or information they get to whomever they want to for further discussion. With Ask Diana, the process of data transfer during the emergency response period can be simplified.
• An intuitive platform on mobile devices: A user interface was designed based on mobile device specifications for Ask Diana. In the past, decision-makers or their decision support staff needed to access the data they wanted using a personal computer or laptop. They usually needed to be well trained to operate the system. With Ask Diana, the user is able to operate the chatbot by using image-based buttons on their mobile device. The user also does not need to spend time learning a new system.

7.2. Limitations

Even though the developed planning method can help the user access the disaster data easily, the following limitations need to be overcome.

• No Natural Language Process (NLP) algorithm: Ask Diana currently utilizes a fuzzy search and keyword mapping table for user intent understanding. This leads to poor performance in the case of long inputs or natural language inputs. Additionally, instead of retrieving all the requested information simultaneously, the users have to split the input into fundamental queries to retrieve one part of the information at a time.
• Data revealing approach for data with URLs: Ask Diana now reveals some data by giving the original website URL for the user. However, some of the websites did not have the Responsive Web Design for mobile devices. Data with such issues would be shown in an inappropriate way on mobile devices. This may make it hard for the user to precisely retrieve the information they want.
• Labor-intensive system maintenance: Ask Diana uses a handcrafted keyword mapping table to achieve the user intent understanding. The mapping table now contains more than 200 keywords. The process of maintaining such a large table is labor-intensive and time-consuming. It also makes the task of improving the system performance difficult due to a lack of flexibility in terms of extending the mapping table. Moreover, if the user wants to change the system from Chinese into English version, then they may need to rebuild the entire keyword mapping table.

8. Conclusions

This research proposes a keyword based chatbot system, named Ask Diana, for water-related disaster management. Ask Diana contains a water-related disaster database, a user intent understanding mechanism, and a mobile device-based user interface. The database collects sufficient data for water-related disaster decision support so that decision-makers can get all data or information they need on Ask Diana instead of searching within different resources. A fuzzy search algorithm and keyword mapping table are developed for the understanding of user intents. With such a mechanism, users are able to access the data in Ask Diana by inputting pre-set keywords. Ask Diana is built on a commercial communication platform with a redesigned user interface so that the system can be easily and quickly implemented in the field. A usability test, which involved 10 subjects, validated that Ask Diana was able to facilitate the disaster prevention personnel’s access to efficient and intuitive data or information for decision making. In addition, a six-month field test was conducted by cooperating with the government of Taiwan. The test results demonstrated that Ask Diana has three main advantages for water-related disaster management. First, Ask Diana successfully elevates the accessibility of water-related disaster data. Second, it is able to improve the efficiency of data transformation during the disaster response period. Finally, it provides an intuitive platform that allows the user to access water-related disaster information easily. In conclusion, this research developed a keyword-based chatbot system for water-related disaster management. The system can help both the decision makers and their decision support staff to access disaster data and develop corresponding response strategies. However, Ask Diana can only play the role of decision supporting, that provides the needed and sufficient information for humans to conduct the decision.