Proceedings of the Human Factors and Ergonomics Society ... Annual Meeting, Sep 1, 2015
Eye tracking analysis can be used to evaluate human performance. However, it is a challenging tas... more Eye tracking analysis can be used to evaluate human performance. However, it is a challenging task to map eye tracking data to multi-element moving objects if the objects take different complex shapes and dynamically move within the display. We propose algorithms that approximate multi-element moving objects into simpler areas of interest (AOIs). Based on minimum AOIs required to represent objects, three AOI types (rectangle, circle, and triangle) were created that maintains the same sizes regardless of object shapes. The algorithms were applied to a task of observing a simulated air traffic control display. Two types of eye fixation data were used for evaluation: Simulated data using uniform distribution and human participant data. The number of eye fixations differed significantly among AOI types when simulated data were used (p < .001) but were insignificant when human data were used (p = .94). We discuss how eye tracking data may be affected by AOI types, and we offer suggestions to improve simulated eye fixations and AOI designs for multi-element moving objects. This pilot study provides ground work for better AOI design and possibly for automating the development of accurate AOIs for multi-element moving objects.
SPE Annual Technical Conference and Exhibition, 2017
The contribution of Human Factors, in complex commercial and contractual environments like the oi... more The contribution of Human Factors, in complex commercial and contractual environments like the oil and gas industry, is fully-fledged and admitted. With the understanding that there were cascading failures due to human errors, the root cause analysis of accidents reveals that many of them were preventable. The Human Factors has not been accentuated thoroughly in this industry, and, traditionally, the focus has been on personnel knowledge and competence. In addition, lack of emphasis on Human Factors in academia as a backbone of the industry could be another reason for this negligence. Therefore, the proper academic training, which thoroughly emphasizes the non-technical skills (NTSs) in conjunction with Human Factors, is lacking in the curriculum of complex engineering programs like petroleum. For the first time, at the University of Oklahoma, the Human Factors in Oil and Gas Operations class was offered. The offered course fulfilled the mission of training petroleum engineering stu...
Offshore drilling operations consist of complex and high-risk processes. Lack of situational awar... more Offshore drilling operations consist of complex and high-risk processes. Lack of situational awareness in drilling operations has become an important human factor issue that causes safety accidents. Prolonged work shifts and fatigue are some of the crucial issues that impact performance. Eye tracking technology can be used to distinguish the degree of awareness or alertness of participants that might be related to fatigue or onsite distractions. Oculomotor activity can be used to obtain visual cues that can quantify the drilling operators’ situational awareness that might enable us to develop warning alarms to alert the driller. Such systems can help reduce accidents and save non-productive time. In this paper, we used eye tracking techniques to differentiate the situational awareness between a representative expert and a group of novices using a scenario-based Virtual Reality Drilling Simulator. Significant visual oculomotor activity differences were identified between the expert a...
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2016
Visualization approaches for eye movement analysis suffers from two limitations: (1) inability to... more Visualization approaches for eye movement analysis suffers from two limitations: (1) inability to handle the stochasticity of both number and position of moving areas of interests (AOIs), and (2) absence of quantitative metrics to analyze eye movement data. We adapted the directed weighted network (DWN) and associated “centrality” metrics to support the visualization of the complex eye movement data. A case study was performed using a realistic air traffic control task environment. Promising results were found as we were able identify important targets (aircraft) interrogated by an air traffic controller based on different time frames. This case study serves as a foundation to develop effective data visualization methods and quantitative metrics for analyzing complex eye movements for a multi-element tracking task.
2021 4th International Conference on Bio-Engineering for Smart Technologies (BioSMART), 2021
A dynamic task refers to a task in which the state of the system can dynamically change when a us... more A dynamic task refers to a task in which the state of the system can dynamically change when a user interacts with the system's components. For example, when an air traffic controller detects aircraft on converging flight paths, the controller can select from multiple altitude, heading and speed clearances to maintain safe separation between them. Some clearance options for those two aircraft, however, may lead to losses of separation with other aircraft. One viable non-intrusive approach to characterize a user's interaction with a system is through real-time analysis of eye movements at the time when the state of the system is changing. The presentation of data from such analyses could be an effective way to enhance user training techniques. In this article, we provide a framework of how to analyze eye-tracking data to identify useful characteristics along with associated algorithms, followed by a simple case study to validate our framework.
The Federal Aviation Administration (FAA) forecasted a 1.4% annual increase in the US air traffic... more The Federal Aviation Administration (FAA) forecasted a 1.4% annual increase in the US air traffic volume; from currently 43.2 million aircraft to 60.3 million by 2040 (FAA, 2018). However, the currently available number of expert air traffic control specialists (ATCSs) might not be sufficient to handle the anticipated increase of air traffic volume. Additionally, the current training completion time of the air traffic controllers takes many years of intensive training (Hampton, 2016). Therefore, the FAA has been trying to find ways to efficiently train the FAA Academy candidates. One of the critical tasks of ATCSs is to detect and mitigate possible aircraft conflicts (i.e. possible collisions) through visually scanning the radar screen. The ATCSs look for conflicting situations between aircraft pairs (or groups) to resolve it and guide them to their destination in a safe/timely manner. Thus, the ATCS’s task involves a significant amount of visual scanning of the radar display and, s...
Due to the high cost of training new air traffic control specialists (ATCSs) and the expected inc... more Due to the high cost of training new air traffic control specialists (ATCSs) and the expected increase of new hires in the near future, we need less costly methods to effectively train them. One of the approaches is to utilize expert ATCSs’ visual scanning characteristics as a learning method; however, there are many issues that need to be addressed when we try to visualize and/or aggregate complex raw eye movement data (i.e. eye fixations, durations, and saccades) into meaningful patterns. In this paper, we provide a compact and conceptual review of how the eye movement data visualization methods have been evolving. In addition, we further explain the concepts of recently advancing data visualization approaches to analyze the eye movements of ATCSs in order to better evaluate human performance or train novices.
Due to the high cost of training new air traffic controllers, we need less costly ways to effecti... more Due to the high cost of training new air traffic controllers, we need less costly ways to effectively train them, particularly in the light of the bubble of new controllers expected to enter the FAA workforce in the near future. If we can better understand how veteran controllers visually search and control multiple aircraft under different situations, then we could improve the training procedures to reduce the time and cost for training new controllers. An experiment was conducted to characterize and classify veteran controllers’ visual search patterns and aircraft control strategies using a high-fidelity simulator. Eye tracking data and verbal responses were analyzed together to discover dominant strategies. The results show promise on using veteran controllers’ strategies to support training of novices, especially FAA Academy candidates.
Universal Design for Learning (UDL) seeks to reach all students through accessible and flexible c... more Universal Design for Learning (UDL) seeks to reach all students through accessible and flexible curriculum that matches their learning pathways and increases their ability to perceive, understand, and actively apply new skills to various applications. A cohesive review of the pivotal UDL approaches is provided along with how those pivotal approaches can be further implemented into not only K-12 STEM education but also higher education. UDL applications in the general curriculum will be expounded upon and illustrated with practical approaches instructors can take to achieve UDL in their classrooms. Strategies for effective implementation and recommendations for the future of UDL will be discussed, specifically in the context of FAA operational training.
Pilot fatigue is a critical reason for aviation accidents related to human errors. Human-related ... more Pilot fatigue is a critical reason for aviation accidents related to human errors. Human-related accidents might be reduced if the pilots’ eye movement measures can be leveraged to predict fatigue. Eye tracking can be a non-intrusive viable approach that does not require the pilots to pause their current task, and the device does not need to be in direct contact with the pilots. In this study, the positive or negative correlations among the psychomotor vigilance test (PVT) measures (i.e., reaction times, number of false alarms, and number of lapses) and eye movement measures (i.e., pupil size, eye fixation number, eye fixation duration, visual entropy) were investigated. Then, fatigue predictive models were developed to predict fatigue using eye movement measures identified through forward and backward stepwise regressions. The proposed approach was implemented in a simulated short-haul multiphase flight mission involving novice and expert pilots. The results showed that the correla...
The Federal Aviation Administration (FAA) forecasted substantial increase in the US air traffic v... more The Federal Aviation Administration (FAA) forecasted substantial increase in the US air traffic volume creating a high demand in Air Traffic Control Specialists (ATCSs). Training times and passing rates for ATCSs might be improved if expert ATCSs’ eye movement (EM) characteristics can be utilized to support effective training. However, effective EM visualization is difficult for a dynamic task (e.g. aircraft conflict detection and mitigation) that includes interrogating multi-element targets that are dynamically moving, appearing, disappearing, and overlapping within a display. To address the issues, a dynamic network-based approach is introduced that integrates adapted visualizations (i.e. time-frame networks and normalized dot/bar plots) with measures used in network science (i.e. indegree, closeness, and betweenness) to provide in-depth EM analysis. The proposed approach was applied in an aircraft conflict task using a high-fidelity simulator; employing the use of veteran ATCSs a...
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2020
Virtual reality (VR) enable us to train in a safe environment using computer-generated simulation... more Virtual reality (VR) enable us to train in a safe environment using computer-generated simulations. One such simulated environment is the Deepwater Horizon operation, and VR enables us to evaluate trainees’ and operators’ situation awareness (SA) in a non-hazardous environment. One unobtrusive and viable SA evaluation method might be the use of eye movements, specifically the time-ordered visual scan paths. In this research, we investigated how SA can be associated with visual scan paths in an anomaly detection task within the oil drilling rig virtual reality simulator. The results show that the trainees having lower SA tended to create random visual scan paths whereas the trainees having higher SA tended to create concentrated and refined visual scan paths. The results show promise in developing timely intervention methods through analyzing the visual scan path characteristics of the trainees.
Proceedings of the Human Factors and Ergonomics Society ... Annual Meeting, Sep 1, 2015
Eye tracking analysis can be used to evaluate human performance. However, it is a challenging tas... more Eye tracking analysis can be used to evaluate human performance. However, it is a challenging task to map eye tracking data to multi-element moving objects if the objects take different complex shapes and dynamically move within the display. We propose algorithms that approximate multi-element moving objects into simpler areas of interest (AOIs). Based on minimum AOIs required to represent objects, three AOI types (rectangle, circle, and triangle) were created that maintains the same sizes regardless of object shapes. The algorithms were applied to a task of observing a simulated air traffic control display. Two types of eye fixation data were used for evaluation: Simulated data using uniform distribution and human participant data. The number of eye fixations differed significantly among AOI types when simulated data were used (p < .001) but were insignificant when human data were used (p = .94). We discuss how eye tracking data may be affected by AOI types, and we offer suggestions to improve simulated eye fixations and AOI designs for multi-element moving objects. This pilot study provides ground work for better AOI design and possibly for automating the development of accurate AOIs for multi-element moving objects.
SPE Annual Technical Conference and Exhibition, 2017
The contribution of Human Factors, in complex commercial and contractual environments like the oi... more The contribution of Human Factors, in complex commercial and contractual environments like the oil and gas industry, is fully-fledged and admitted. With the understanding that there were cascading failures due to human errors, the root cause analysis of accidents reveals that many of them were preventable. The Human Factors has not been accentuated thoroughly in this industry, and, traditionally, the focus has been on personnel knowledge and competence. In addition, lack of emphasis on Human Factors in academia as a backbone of the industry could be another reason for this negligence. Therefore, the proper academic training, which thoroughly emphasizes the non-technical skills (NTSs) in conjunction with Human Factors, is lacking in the curriculum of complex engineering programs like petroleum. For the first time, at the University of Oklahoma, the Human Factors in Oil and Gas Operations class was offered. The offered course fulfilled the mission of training petroleum engineering stu...
Offshore drilling operations consist of complex and high-risk processes. Lack of situational awar... more Offshore drilling operations consist of complex and high-risk processes. Lack of situational awareness in drilling operations has become an important human factor issue that causes safety accidents. Prolonged work shifts and fatigue are some of the crucial issues that impact performance. Eye tracking technology can be used to distinguish the degree of awareness or alertness of participants that might be related to fatigue or onsite distractions. Oculomotor activity can be used to obtain visual cues that can quantify the drilling operators’ situational awareness that might enable us to develop warning alarms to alert the driller. Such systems can help reduce accidents and save non-productive time. In this paper, we used eye tracking techniques to differentiate the situational awareness between a representative expert and a group of novices using a scenario-based Virtual Reality Drilling Simulator. Significant visual oculomotor activity differences were identified between the expert a...
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2016
Visualization approaches for eye movement analysis suffers from two limitations: (1) inability to... more Visualization approaches for eye movement analysis suffers from two limitations: (1) inability to handle the stochasticity of both number and position of moving areas of interests (AOIs), and (2) absence of quantitative metrics to analyze eye movement data. We adapted the directed weighted network (DWN) and associated “centrality” metrics to support the visualization of the complex eye movement data. A case study was performed using a realistic air traffic control task environment. Promising results were found as we were able identify important targets (aircraft) interrogated by an air traffic controller based on different time frames. This case study serves as a foundation to develop effective data visualization methods and quantitative metrics for analyzing complex eye movements for a multi-element tracking task.
2021 4th International Conference on Bio-Engineering for Smart Technologies (BioSMART), 2021
A dynamic task refers to a task in which the state of the system can dynamically change when a us... more A dynamic task refers to a task in which the state of the system can dynamically change when a user interacts with the system's components. For example, when an air traffic controller detects aircraft on converging flight paths, the controller can select from multiple altitude, heading and speed clearances to maintain safe separation between them. Some clearance options for those two aircraft, however, may lead to losses of separation with other aircraft. One viable non-intrusive approach to characterize a user's interaction with a system is through real-time analysis of eye movements at the time when the state of the system is changing. The presentation of data from such analyses could be an effective way to enhance user training techniques. In this article, we provide a framework of how to analyze eye-tracking data to identify useful characteristics along with associated algorithms, followed by a simple case study to validate our framework.
The Federal Aviation Administration (FAA) forecasted a 1.4% annual increase in the US air traffic... more The Federal Aviation Administration (FAA) forecasted a 1.4% annual increase in the US air traffic volume; from currently 43.2 million aircraft to 60.3 million by 2040 (FAA, 2018). However, the currently available number of expert air traffic control specialists (ATCSs) might not be sufficient to handle the anticipated increase of air traffic volume. Additionally, the current training completion time of the air traffic controllers takes many years of intensive training (Hampton, 2016). Therefore, the FAA has been trying to find ways to efficiently train the FAA Academy candidates. One of the critical tasks of ATCSs is to detect and mitigate possible aircraft conflicts (i.e. possible collisions) through visually scanning the radar screen. The ATCSs look for conflicting situations between aircraft pairs (or groups) to resolve it and guide them to their destination in a safe/timely manner. Thus, the ATCS’s task involves a significant amount of visual scanning of the radar display and, s...
Due to the high cost of training new air traffic control specialists (ATCSs) and the expected inc... more Due to the high cost of training new air traffic control specialists (ATCSs) and the expected increase of new hires in the near future, we need less costly methods to effectively train them. One of the approaches is to utilize expert ATCSs’ visual scanning characteristics as a learning method; however, there are many issues that need to be addressed when we try to visualize and/or aggregate complex raw eye movement data (i.e. eye fixations, durations, and saccades) into meaningful patterns. In this paper, we provide a compact and conceptual review of how the eye movement data visualization methods have been evolving. In addition, we further explain the concepts of recently advancing data visualization approaches to analyze the eye movements of ATCSs in order to better evaluate human performance or train novices.
Due to the high cost of training new air traffic controllers, we need less costly ways to effecti... more Due to the high cost of training new air traffic controllers, we need less costly ways to effectively train them, particularly in the light of the bubble of new controllers expected to enter the FAA workforce in the near future. If we can better understand how veteran controllers visually search and control multiple aircraft under different situations, then we could improve the training procedures to reduce the time and cost for training new controllers. An experiment was conducted to characterize and classify veteran controllers’ visual search patterns and aircraft control strategies using a high-fidelity simulator. Eye tracking data and verbal responses were analyzed together to discover dominant strategies. The results show promise on using veteran controllers’ strategies to support training of novices, especially FAA Academy candidates.
Universal Design for Learning (UDL) seeks to reach all students through accessible and flexible c... more Universal Design for Learning (UDL) seeks to reach all students through accessible and flexible curriculum that matches their learning pathways and increases their ability to perceive, understand, and actively apply new skills to various applications. A cohesive review of the pivotal UDL approaches is provided along with how those pivotal approaches can be further implemented into not only K-12 STEM education but also higher education. UDL applications in the general curriculum will be expounded upon and illustrated with practical approaches instructors can take to achieve UDL in their classrooms. Strategies for effective implementation and recommendations for the future of UDL will be discussed, specifically in the context of FAA operational training.
Pilot fatigue is a critical reason for aviation accidents related to human errors. Human-related ... more Pilot fatigue is a critical reason for aviation accidents related to human errors. Human-related accidents might be reduced if the pilots’ eye movement measures can be leveraged to predict fatigue. Eye tracking can be a non-intrusive viable approach that does not require the pilots to pause their current task, and the device does not need to be in direct contact with the pilots. In this study, the positive or negative correlations among the psychomotor vigilance test (PVT) measures (i.e., reaction times, number of false alarms, and number of lapses) and eye movement measures (i.e., pupil size, eye fixation number, eye fixation duration, visual entropy) were investigated. Then, fatigue predictive models were developed to predict fatigue using eye movement measures identified through forward and backward stepwise regressions. The proposed approach was implemented in a simulated short-haul multiphase flight mission involving novice and expert pilots. The results showed that the correla...
The Federal Aviation Administration (FAA) forecasted substantial increase in the US air traffic v... more The Federal Aviation Administration (FAA) forecasted substantial increase in the US air traffic volume creating a high demand in Air Traffic Control Specialists (ATCSs). Training times and passing rates for ATCSs might be improved if expert ATCSs’ eye movement (EM) characteristics can be utilized to support effective training. However, effective EM visualization is difficult for a dynamic task (e.g. aircraft conflict detection and mitigation) that includes interrogating multi-element targets that are dynamically moving, appearing, disappearing, and overlapping within a display. To address the issues, a dynamic network-based approach is introduced that integrates adapted visualizations (i.e. time-frame networks and normalized dot/bar plots) with measures used in network science (i.e. indegree, closeness, and betweenness) to provide in-depth EM analysis. The proposed approach was applied in an aircraft conflict task using a high-fidelity simulator; employing the use of veteran ATCSs a...
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2020
Virtual reality (VR) enable us to train in a safe environment using computer-generated simulation... more Virtual reality (VR) enable us to train in a safe environment using computer-generated simulations. One such simulated environment is the Deepwater Horizon operation, and VR enables us to evaluate trainees’ and operators’ situation awareness (SA) in a non-hazardous environment. One unobtrusive and viable SA evaluation method might be the use of eye movements, specifically the time-ordered visual scan paths. In this research, we investigated how SA can be associated with visual scan paths in an anomaly detection task within the oil drilling rig virtual reality simulator. The results show that the trainees having lower SA tended to create random visual scan paths whereas the trainees having higher SA tended to create concentrated and refined visual scan paths. The results show promise in developing timely intervention methods through analyzing the visual scan path characteristics of the trainees.
Uploads
Papers by Ziho Kang