Eye Tracking Research in Cartography: Looking into the Future

Abstract

Eye tracking has been served as one of the most objective and valuable tools towards the examination of both map perceptual and cognitive processes. The aim of the present article is to concisely present the contribution of eye tracking research in cartography, indicating the existing literature, as well as the current research trends in the examined domain. The authors discuss the existing challenges and provide their perspectives about the future outlook of cartographic eye tracking experimentation by reporting specific key approaches that could be integrated.

1. Introduction

A typical cartographic communication model consists of three main parts; cartographer, map, and map user [1]. According to this model, cartographers encode geographic information in order to provide cartographic products which work as “vehicles” that deliver cartographic information to map users. In practice, the process of encoding involves the selection of a representative part of the real world and its depiction using a specific graphic language (cartographic symbolization). This language is supported by specific design tools, including visual [2], dynamic [3], and sound [4] variables. The application of the design variables has resulted in different types of cartographic products, such as static, animated, and multimedia maps. Moreover, interactivity constitutes one of the main characteristics of the maps that are usually distributed on the Internet [5]. The third component of the cartographic communication model involves map users, as well as the process of decoding the transferred information delivered by the map. The examination of map users’ responses to map design is a critical question in cartographic research [6]. This statement is also clear considering several categories of map readers/users (e.g., according to different levels of their expertise).

Over the last decades, the cartographic community has recognized the importance of examining and involving theories, models, and methodological methods developed in the research field related to visual perception and cognition, in cartographic research [7,8,9,10,11,12]. Hence, the adaptation of well-established experimental techniques, already implemented in similar research fields such as psychology and neuroscience, gave the opportunity to cartographers to examine different aspects related to the basic components of cartographic communication. The process of transferring psychological research to cartography is based on the simple assumption that treats maps as the experimental stimulus and the map perception process as the reaction during its observation [13]. The examination of different aspects related to parameters, which are involved in map perception, plays a significant role in the cartographic design process. Considering that maps are abstract and artificial design products, the outcomes produced by experimental cartographic studies could be implemented directly towards the production of effective and efficient maps.

The aim of the present article is to briefly report the contribution and the current trends of eye-tracking technology in cartographic research, as well as to identify and report the authors’ point of view towards the future perspectives in the examined field. In the sections below, existing research review studies, related to eye tracking techniques and eye movement analyses methods implemented in cartography, are reported. The examined domains of the existing experimental studies are also described briefly. Moreover, current research and perspectives are discussed in order to highlight the future outlook for cartographic experimental research.

2. Review Studies Related to Eye Tracking Methods in Cartographic Research

The influence of eye-tracking technology in cartography is clear after considering the amount of published research works since the first experimental studies appeared in the field [14,15]. The first review study on the applications of eye movement analyses in cartography and related fields, such as psychology, was provided by Steinke [16]. Back in the 1990s and 2000s, this work was quite important and fruitful, especially for the researchers who wanted to perform experimental research in the field of map perception and cognition, since it established the first framework highlighting, and at the same time, the existing trends in the field. Just a few years ago (in 2017), Kiefer et al. [17] delivered an overview of the eye-tracking applications in the research domains of spatial cognition, GIS, and cartography, identifying the existing trends and challenges in these fields. Krassanakis and Cybulski [18] extended the aforementioned overview by focusing on the research panorama of eye-tracking research in cartography during the decade 2009–2018. Krassanakis and Cybulski [18] identified five main categories that cartographic eye-tracking studies could be classified into, including cartographic symbolization and design principles, comparing 2D and 3D representations, map users’ expertise, several other applications related to map perception processes, as well as eye tracking analysis tools and methods provided by cartographic community. Additionally, in previous work, some initial indications about the future perspectives in the field were also highlighted.

3. Current Trends

Considering the research studies that have appeared since 2019, eye tracking serves as an objective tool for the examination of different aspects related to the map perception process. The reported trends in the current paragraph are based on the examination of the existing literature (since 2019), which was studied after searching in well-established academic search engines, such as Scopus and Google Scholar. As examples, research studies which aim to examine the influence of map labels [19] and map legends [20] in the map perception process, can be mentioned. Moreover, previous years’ eye tracking has been used as the main experimental method for the evaluation of the effectiveness of specific types of maps (e.g., [21,22,23]) or different map-based graphical user interfaces (e.g., [24]) working either on typical or smaller (i.e., smartphones) displays [25], under different types of tasks (e.g., [26,27]) and conditions. Substantial contributions have also been presented in the domains of geography map-use education [28,29] and in geovisualization tools’ learnability effects [30]. Additionally, recent research studies appearing in the field deal with more theoretical concepts related to the map symbolization process as well as the effectiveness of the fundamental element of map design used either in classic static representations [31] or in modern animated maps [32].

As also noticed in the review article provided by Krassanakis and Cybulski [18], recent cartographic experimental studies combine eye tracking with other techniques, such as electroencephalography (EEG) [33] or expert judgment procedures and questionnaire analyses (e.g., [34]). Furthermore, the great experience gained from cartographers in eye-tracking methods can also be proved considering the delivered tools to scientific and research community (e.g., [35,36]). Finally, eye tracking has been utilized as an experimental tool towards the evaluation and the comparison between virtual reality (VR)/augmented reality (AR) and desktop-based environments [37,38]. In VR, eye tracking is also used to determine to what extent cartographic elements (such as a navigational mini-map) determine visual attention in several translocation tasks [39].

Another trend in cartographic research is the search for efficient visual strategies during specific map-based tasks. One of the first research works in this field was the study by Çöltekin et al. [40]. This methodological approach contributed to a deeper understanding of how people perform visual interaction strategies combined with inference and decision-making in pattern recognition. However, studying visual behavior while using the maps’ graphical user interface contributed to highlighting differences in visual strategies over time [30,41,42].

4. Future Perspectives

Eye-tracking research gave great opportunities to cartographers towards examining theories and models developed in the fields of vision, visual perception, and visual cognition, under map usage and map-based task conditions. Additionally, the quick development of eye-tracking equipment contributed substantially to the performance of several types of experimental studies, since required procedures (e.g., participants’ calibration) can be adapted easier under different conditions (i.e., experimental set-ups). Concurrently, the accuracy of eye-tracking devices can also be considered more than adequate for cartographic examination processes. However, a huge challenge is associated with the improvement of the overall usability (mainly in terms of their flexibility and accuracy) of low-cost eye-tracking devices. This can also be proved considering existing research studies [43,44,45] delivered by the cartographic community towards this goal.

A further step involves the online shift of eye-tracking experimentation. This step requires the overall improvement of the ordinal webcam-based eye tracking implemented approaches such as these described in related fields (see, e.g., the work described in [46]). The development of ordinal webcam-based eye tracking methods could substantially contribute to the more effective examination of different parameters related to several types of cartographic products as well as to graphical user interfaces (GUIs) connected to them. Moreover, considering that today the vast majority of cartographic products are distributed as web maps, the utilization of objective tools that are able to capture the visual behavior online meets several opportunities and challenges [47]. The potential of shifting “lab-based” experiments online gives the opportunity to expand the overall number of participants. However, this also raises some questions regarding the validity of the experimental process. The most important problem is that the experimental process is not fully supervised by the experimental operator. Previous studies have shown that lab participants are generally more accurate than online participants [48]. Additionally, it is not always feasible to retain constant experimental conditions in online studies. Hence, experimental design must be adaptive to different parameters including lighting conditions, Internet connection properties, hardware and software setup, etc. However, in this way, cartographic experiments will become more realistic, examining the visual behavior under more natural situations. The cartographic community is well informed about the technological solutions and software related to the conducting of online eye-tracking studies [49,50]. As also pointed out by Murali and Çöltekin [49], the biggest problem of existing online eye-tracking solutions is connected to the accuracy and precision of eye-tracking data. New approaches, which use artificial intelligence (AI), seem to improve the overall validity of the collected gaze data [49]. However, this is an open challenge that needs further examination and interdisciplinary research.

Taking into account that eye-tracking devices can capture binocular gaze data at high (up to 2000 Hz) recording frequencies, the amount of collected data during a typical experiment is huge. Hence, the modern analysis approaches must be automatized to avoid possible mistakes. Several existing tools (e.g., [51,52,53]), including software for similar techniques that can be combined with eye tracking (e.g., mouse tracking [54]) developed by cartographers could help researchers to program the experimental analyses of the collected gaze data.

Cartography and eye-tracking perspectives are related to the use of eye-movement recording technology in VR headsets. Existing equipment (HTC Vive Pro Eye) allows the integration of virtual reality and geospatial data, which is the crucial aspect of the multimedia map. The immersive VR environment supports the possibility of exploring terrain visualization and eye tracking enables capturing users’ gazes and comparing visual behavior with the real world [55].

The development and distribution of—dedicated to cartographic stimuli—eye tracking databases can also play a critical role in future approaches in the field. It is important to consider that this practice is typical in other domains related to computer science and visual attention modelling (see e.g., EyeTrackUAV [56] and EyeTrackUAV2 [57] eye-tracking datasets). The role of such datasets is twofold. Firstly, they can serve as objective ground truths towards visual attention modeling and prediction processes. Secondly, it gives the opportunity to other researchers to extend existing studies promoting an open science approach at the same time.

Eye tracking also has some limitations. Eye-tracking research relies on the assumption that what the users fixate on is related to processing. It is defined as the eye-mind hypothesis (EMH) [58]. This assumption has certain limitations. As noted by Schindler and Lilienthal [59], in solving geometrical problems, fixations patterns were not confirmed by the EMH. This limitation can be overcome, and gaze data could correspond to the attentional processing index, as long as the visual stimuli in cartographic research is relevant to the actual task [60]. Besides, cartographic symbols communicate certain facts about reality and might play a role in the retrieval from memory process. Nevertheless, in some cases, modeling of this retrieval process results as being independent of eye movements [61]. Consequently, cartographic research adopting eye tracking faces challenges in the field of psychology. Overcoming these challenges has to include other methods that would complement existing research. Such a complementary approach should involve usability measures, e.g., think-aloud protocols which have already been applied in evaluation of cartographic products [62,63]. However, recent studies have shown that using these think-aloud protocols combined with eye tracking is also problematic. Applying the concurrent think-aloud method (CTA), which is simultaneous with recording eye movement, results in a distortion of eye-tracking data. Therefore, Prokop et al. [64] suggest the retrospective think-aloud (RTA) method as being more suitable for eye-tracking research. It provides independent data on user behavior, and at the same time, does not influence the duration of individual fixations.

Another limitation identified early on is that lighting conditions must be appropriate since the eye tracker uses infrared light [65]. Eye-tracking research requires considerable time resources. Usually, during an experiment, the eye tracker records the eye movement of one user. Before starting the experimental procedure, there could be problems, e.g., with calibrating the participant, which makes the whole process even longer. In cases where visual calibration is not possible, Harrar et al. [66] proposed nonvisual calibration methods. Even if these alternative calibration methods turned out to be similarly precise to standard calibration, methods such as replacement calibration led to significantly smaller errors than alternative methods. This approach provides the possibility to perform eye-tracking research with blind and visually impaired people. The common approach is to present up to nine markers during eye-tracker calibration. Therefore, to obtain more data in the calibration procedure, researchers might use smooth pursuit or vestibulo-ocular reflex while using symbolic regression [67]. In a smooth calibration procedure, the participant follows only one marker, which moves along a predefined path and at a constant speed. The second approach allows participants to observe a marker while doing head movements. Both procedures offer advantages over the nine-point calibration, such as continuous data.

A major challenge for cartographic research is the recording of eye movement data together with additional signals from different sources. These co-recordings include EEG, galvanic skin response (GSR), or functional magnetic resonance imaging (fMRI). However, each method has an optimal setup, which might not be optimal in combination with other methods. Movement while EEG recording might result in artifact-biased data [68]. Consequently, combining different approaches and recording other signals require expertise in each field and is more challenging for the experimental operator to prepare a map-based task. Therefore, the combination of, e.g., eye tracking and EEG during a VR navigation task poses serious research limitations. Another serious limitation of the usage of eye tracking in cartographic research is that it records only foveal vision. Peripheral vision also receives information, and movement in this area (e.g., in dynamic maps) has a significant impact on visual attention.

Cartographic research using eye tracking can benefit from relatively novel paradigms, e.g., critical cartography or post-representational cartography [69]. The deconstructive approach to map design can be verified and confronted with the classic mapping approach. The subjective approach of mapmakers in processes such as the choice of projection, symbolization, data categorization, or scale, can be visually assessed by various groups of users. This theoretical framework emphasizes how the map is produced and how it works in the world. Therefore, the practical implications of these approaches combined with eye-tracking methodology could result in a new perspective questioning the traditional scientific discourse [70]. Respectively, the future challenge is to critically evaluate if the eye-tracking experiment is an appropriate method for more complex, cognitively more demanding tasks, such as spatial-pattern and trends identification.

5. Concluding Remarks

Undoubtedly, eye-tracking technology has substantially contributed to the experimental cartographic research. Considering the existing applications and the current trends discussed in the framework of the present article, we can state that the cartographic community has gained critical experience in eye-tracking methods over the last decades. This expertise could serve as a great asset towards the adaptation of modern cartographic studies in existing challenges. These challenges could mainly be connected to online (webcam-based) eye tracking, the automatization of the experimental analysis, the incorporation of VR/AR technologies, the open distribution of cartographic oriented eye tracking datasets, the concept of EMH, more effective processes for participants’ calibration, as well as the combination of methods and techniques aimed at cartographic experimentation.