Augmented Reality Applications for Learning Geography in Primary Education

Abstract

Augmented Reality is an emerging educational technology that has the potential to provide innovative methods of teaching and create engaging learning experiences. Augmented Reality applications implementing game-based design could enrich education by increasing motivation and engagement and enabling better learning outcomes. Similarly, Augmented Reality, in the context of Geography, could enhance the learning process and the user experience through the visualization of the content and a better understanding of abstract concepts. Therefore, in this study, (a) three specially designed Augmented Reality applications are described for teaching Geography in the fifth and sixth grades, and (b) an extensive usability evaluation study is reported using the three applications. Teachers (N = 6) and pupils (N = 43) from the fifth and sixth grades, as well as computer science students (N = 43) participated to assess the usability of the proposed Augmented Reality apps. The results were positive since the proposed Augmented Reality apps provided high-level usability. Finally, they revealed that there was acceptance for the Augmented Reality technology by all participants and a willingness to be incorporated into the teaching process.

1. Introduction

Augmented Reality (AR) has been progressively advanced and widely used nowadays in many fields, including that of education. AR blends the real world with virtual reality, as it augments the real environment with digital elements [1] and allows manipulation, exploration, creation, and interaction with virtual artifacts [2]. Due to its nature, AR can easily visualize abstract concepts and phenomena as well as invisible objects [3]. Therefore, AR could provide the learner with a deeper learning experience through active engagement and seamless interaction with the 3D objects. Additionally, the integration of AR applications into the educational process indicates an increase in the learner’s performance, motivation, satisfaction, and creativity [4]. As a result, AR could have huge potential, providing new ways of teaching and learning.

Despite the rising use of AR technology, the research studies that integrate AR into education are relatively few in the field of sciences, such as history [5,6], physics [7,8,9,10], astronomy [11,12], chemistry [13,14], biology [15], mathematics [3,16], programming/ computational thinking [17], and geography [2,18,19]. For example, in the subject of Geography, on the one hand, the traditional way of learning places the learner as an information receiver [19]. On the other hand, a game-based learning approach, which implements learning activities with game principles, such as appropriate and constructive feedback and interactivity, can promote the learner’s engagement [20]. Therefore, AR with game-based principles could complement the traditional way where the level of interaction is very low, enhancing the learner’s experience, and improving the retention of information [21] as well as the learning outcome [22].

Taking into account the aforementioned, the current work implements three AR applications in the subject of Geography in the fifth and sixth grades of Greek primary schools. The proposed AR applications include game-based design principles, such as providing meaningful and guided feedback as well as promoting interaction. In addition, an experiment was conducted to evaluate the AR applications. Teachers and pupils of fifth and sixth grades, as well as computer science students, participated in the experiment. They evaluated the AR applications on usability and how game-like they were. Finally, it is worth mentioning, that this study is based on previous research work [10], in which an indicative platform for teaching different subjects in all the classes of primary and secondary school was proposed. Within this context, the proposed three AR applications will extend the current version of the platform by adding a second subject to that of Geography.

The rest of the paper is organized as follows. In Section 2, a literature review is presented regarding the utilization of AR in education and specifically in the subject of Geography. The architecture of the AR Geography applications along with the analytical description of each of the apps is depicted in Section 3. In Section 4, the pilot experiment along with the procedure and material is presented, and in Section 5, the results are discussed. Finally, Section 6 concludes the paper, and presents the limitations and future work.

2. Literature Review

AR is used at all stages of education from primary school to university [23]. Beginning with primary school, a few studies have been conducted to explore AR use in Geography [2,18,19,24]. More specifically, Salazar et al. [18] proposed an AR mobile application for learning the solar system to fifth-grade students at primary school. They conducted an experiment to investigate whether the AR application can positively influence students. Overall, 27 students participated, and the results indicated a significant increase in learning outcomes, user satisfaction, and interest.

Herpich et al. [19] developed a mobile AR game for teaching Geography in the sixth grade of elementary school. The AR game focused on the concept of orientation, where the students could explore the Amazon jungle through user missions, secret location trails, and puzzles. Gamification techniques were also applied, such as ranking, rewards, etc. The sample of this activity included 75 students, who found the AR game attractive, fun, and satisfactory. Therefore, the authors concluded that properly incorporating AR technology into the teaching process of geography-related topics can be a constructive and effective method.

In research conducted by Rellia [24], a mobile AR application was proposed to teach English as a foreign language through content, which was Geography. The participants were 46 fourth-grade pupils, in total, from primary school. Two separate groups were taught Geography in English, the first one in the traditional way and the experimental group with the usage of an AR Geography application. The AR application featured 3D models, animation, animals, plants, etc. The results of the experiment were satisfactory and showed that AR application had a positive impact on pupils as well as helped them to improve their learning outcomes.

Ramnarain-Seetohul et al. [2] implemented an AR application to support learning in the subjects of Geography and History in primary school. The AR application was regarding the Naval Battle of Grand-Port on Mauritius, and the learner could navigate to the battle using AR markers. The AR application was evaluated by 10 teachers who found the application usable and effective, and agreed that such an application could be a useful teaching aid.

Advancing to secondary school, Xiao et al. [22] proposed an AR application called “Starry Sky Exploration—Eight Planets in the Solar System” which was addressed to secondary schools in China for the Geography curriculum. The AR application introduced the students to the eight planets of the solar system. In the study, 36 students were involved, and the use of AR application with experiential learning demonstrated that students increased learning effectiveness, satisfaction, motivation, and enhanced interest in the learning method.

Schnürer et al. [25], proposed an AR application that augments the printed school atlas with a digital map. The 3D content includes 3D visualizations, animations, etc. Overall, 58 students participated from different classes and different secondary schools. Students easily navigated to the augmented maps, found them interesting, and were influenced in a sustainable way.

Vahldick et al. [26] also used augmented books as an extension of printed textbooks. A mobile AR application that modeled maps and other characteristics of the African continent was developed. The findings showed that the average score of 28 students of the ninth grade increased and they were more motivated.

Apart from K-12 education, studies have been conducted on tertiary education as well. For example, Xefteris et al. [27] proposed a mixed-reality learning environment, which included robots and a tangible interface where an AR interactive map was displayed for learning geography, history, and computational thinking. Overall, 24 undergraduate students participated and positively evaluated the framework, concluding that the learning outcomes improved as well as the learning process was fun and enjoyable.

Turan et al. [28] developed a set of 15 AR applications for teaching Geography to first-year university students. The study included 95 students divided into two groups: control and experiment. The research revealed that AR reduces students’ cognitive load levels while increasing their achievements and motivation, concluding that AR technology is beneficial also for higher education.

However, according to Shuaili et al. [29], the efficiency of using AR in Geography needs to be studied further. They conducted an experiment including 64 students of the 10th grade, divided into the control and the experimental group. During the experiment, the experimental group used an AR Geography application, and the control group continued with traditional textbook education. The findings revealed that there was an increase in the motivation and interaction of students using the AR technology, but there were no major variations in the learning outcomes, implying that the research should be expanded.

Summarizing, incorporating AR technology into education can be advantageous in enhancing students’ experience as well as improving motivation and knowledge. In addition, the use of new learning styles enriches the teaching and learning process in the domain of Geography, since abstract concepts and phenomena can now be visualized and perceived by students. Despite the benefits, AR in the specific domain of Geography has to be further examined to investigate whether it is beneficial or not.

3. AR Geography Applications

The educational AR apps were specially designed, including game-based elements. They were also based on the Greek curriculum for teaching Geography in the fifth and sixth grades. Three AR apps were developed in total: one for the fifth grade and two for the sixth grade. The reason why there are two apps for the sixth grade is that both of them refer to the planet Earth, but the first AR app (see Section 3.2) focuses on the population, density, etc., while the second (see Section 3.3) on the solar system, atmosphere, etc.

All three AR apps have the same architecture and can be downloaded from the links provided in the section “Supplementary Material”. More analytically, the pupil has the opportunity to play the app for a better understanding of the educational material with the help and guidance of his/her instructors (teachers). By selecting an app, the instructions are presented briefly, explaining what the pupil must do to complete the game. Then, the camera of the device is activated, and s/he has to place in front of the camera a specific image from the school textbook (Figure 1). After tracking the appropriate image target (image or QR Code), the real image of the textbook is augmented with 3D virtual objects, and the user’s AR experience and learning begin by playing the game. Each game differs from the other, offering different functions and capabilities.

Several technologies were combined in order for the games to successfully function. All three AR apps were developed in the Unity Game Engine [30] using the C# programming language [31]. The Vuforia Engine [32], which is the most used platform for AR development, was also used to track and recognize images in real-time as well as to superimpose 3D virtual objects onto real ones. Some of the assets were imported from the Unity Asset Store [33], while others were created using Blender [34].

3.1. AR App for 5th Grade: Greece

The first AR app is addressed to fifth-grade pupils of Greek primary school and is designed for teaching Geography and specifically for learning about Greece. When the app starts, the main screen appears including three buttons (Figure 2).

Each one is responsible for a separate function. The left button displays general info about the game, the right button closes the application, and the middle one opens the main menu with the games. When the main menu is clicked, seven buttons appear, one for each game that corresponds to a chapter of the textbook. After a game is selected, the pupil has to constantly scan the QR Code through the camera, which is in the corresponding chapter of the textbook. As soon as the scan is done, the augmented environment appears on the book. At the end of each game, the result of the pupil’s performance is displayed.

The first game includes the material from the book chapter “Types of maps” (Figure 3). At the beginning of the game, the instructions are presented to the pupil in order to understand the procedure that s/he has to follow to complete the game. Up to this point, the pupil has two options–either to return to the main menu or to start playing the game. By selecting the game, the camera of the device turns on, giving the opportunity to scan the QR Code that exists in the textbook in the corresponding chapter. By scanning the QR Code, the augmented objects are revealed. The game resembles a 3D classroom, which consists of a school board and three different types of maps. On the school board, the questions are displayed in random order each time. The question appears and the pupil has to select the correct type of map. Then, the next question is shown on the school board. This process is repeated for all the questions. After answering all of them, the game presents to the pupil his/her score. After discussing the results with the teacher, s/he can return to the main menu by pressing the button at the bottom of the screen.

The second game refers to the “Orientation” chapter (Figure 4). The goal of this game is to make the pupils understand the basic elements of orientation and how a person can navigate in a place. In the first scene, the pupil has to read the instructions to complete this game. After entering the main game, the pupil scans the QR Code and the game environment is displayed, showing a scene with a variety of 3D objects that are placed in different positions. The pupil should carefully read each of the sentences and choose the appropriate answer to fill in the sentence. This is achieved by selecting one of the four buttons that appear at the bottom of the screen. The sentences are displayed in a random order each time. With the completion of the game, the total score of the pupil is shown.

The next game is called “Coasts of Greece” and aims to teach pupils the coastal features (Figure 5). Firstly, the game instructions are presented and then the scanning of the QR Code is required. Soon after, different coastal features (e.g., peninsulas, bays, capes, etc.) are presented in sequence through a 3D representation of the map of Greece.

Thanks to a button at the top of the screen, the pupil can explore the 3D coastal features by rotating them up to 360°. As s/he explores each one of them separately, s/he selects which one of the buttons that appears on the side of the screen, corresponds to the specific coastal feature. When an answer is selected, the existing button disappears and a text referring to some information about the coastal feature is displayed. After closing the text, the next button becomes active, waiting to be matched with the correct feature. After everything is answered, the summary is presented to the pupil for discussion with the teacher.

The fourth game is about the “Islands of Greece” (Figure 6). The first thing that the pupil must do is to read the instructions. After pressing the button at the bottom of the screen, an arrow appears pointing to an island complex from which the game will start. After that, the camera of the device is activated to scan the QR Code. Then, the 3D model of this island complex is revealed along with four buttons that contain some letters. Next, a sign appears upon each island, containing a few letters from the name of the island. The goal is to fill in the names of the Greek islands with the given letters. With the completion of all the names of an island complex, all the signs of the islands are displayed, offering the pupils the ability to observe and discover all the islands with their names. The game will resume when the pupil presses the button at the top of the screen. Then, a new island complex is presented, and the pupil must follow the same steps to complete the process. At the end, a summary is shown indicating the end of this game.

The next game is called “Mountains of Greece” (Figure 7). The goal is to teach the pupils the heights and locations of the largest mountains in Greece. First, the illustrated instructions appear. As the game begins, the camera turns on to scan the QR Code. After that, the first mountain is represented through a 3D model and four buttons appear on each side. From the left side, the pupil selects the button that refers to the height of the corresponding mountain, while from the right side, s/he selects the button that refers to the location of the corresponding mountain. When a button is selected, the correct answer is shown on a board at the back of the scene. Additionally, feedback is given by changing the colors of the buttons. Red corresponds to the wrong answer, while green corresponds to the right one. After selecting both buttons correctly, additional information appears on the board. In the end, a review of this game is shown on the board.

The sixth game is about the “Climate of Greece” (Figure 8). The goal is to inform about weather phenomena and data that retrieved from a weather website [openweathermap.org (accessed on 20 August 2020)]. Once again, the instructions appear and then the pupil must scan the QR Code from the textbook. After that, four signs depicting weather phenomena appear above the map. The pupil clicks them, one by one, and fills in the sentences through the options that are provided by the buttons below. After completing a sentence correctly, the sign is locked. When all the signs are locked, a button appears that activates a board, which contains a variety of weather data. This information with the help of the teacher can be discussed in the class, resulting in meaningful conclusions. Then, the game is completed by selecting the button at the bottom of the screen.

The last game is the “Geographical distribution of the population” (Figure 9). This game aims to provide a logical cause of the phenomenon of urbanization in Modern Greek society. The pupil reads the game instructions and then when s/he scans the QR Code, the 3D map of Greece is visualized with some signs on it. At the bottom of the screen, there are six icons with a number that represents the size of a population. By choosing a sign, the game requests the pupil to drag and drop the more suitable answer of an area population, based on his/her personal beliefs and critical thinking about the correlation with the geographical location (mountainous area, island, cities, etc.). By selecting the right choice, some reasons why the population in the specific area has this size are presented. The above procedure is followed for all the signs on the map. After that, the conclusion appears along with a button that leads to the main menu.

3.2. AR App for 6th Grade: Earth as a Living Space for Humans

The second AR app is designed and developed for the sixth grade of Greek primary school and focuses more on learning the popularity, density, etc., of the Earth. The introductory screen includes a play button that leads to the main menu consisting of four more buttons, each one corresponding to a specific chapter of the textbook (Figure 10).

In the main menu, there is also a back button (arrow) at the top right of the screen that closes the menu and returns to the introductory screen. This AR app uses both interactive and non-interactive activities where the pupil has to unravel by scanning images from the textbook.

The first game represents the chapter “Earth as a Celestial Body” and contains three different activities/mini games. Firstly, in every activity/mini game apart from the navigation menu (on the top left) of the screen, which helps the pupil to navigate into the application, there is an info button on the top right that indicates which images of the textbook have to be scanned to reveal the AR content (Figure 11). Therefore, in the first activity, the camera is activated, and the pupil scans the target image. Then, a video of the rotation of the Earth is presented. There is also a 3D clipboard that has information about the Earth, such as the shape, movement, and duration of both the planet’s rotation and orbit around the Sun, etc. This AR activity is not interactive.

In the second activity (Figure 12), the pupil scans the corresponding image and the Earth’s orbit around the Sun in 3D appears. The pupil can move around and watch the Earth orbiting around the Sun. Like the previous video, this is also not interactive.

The third mini game refers to the position of the planets of the solar system (Figure 13). When the image is scanned, a message pops up mentioning the instructions of the game. The 3D model of the Sun, along with a shelf that contains all the planets, appears in front of the pupil. The goal of the mini game is to choose the correct planet from the closest to the farthest from the Sun. Whenever a correct planet is selected, the 3D model of this planet disappears from the shelf and slowly appears orbiting around the sun. According to the number of right or wrong choices, the mini game keeps a score. After all the planets have been chosen in the correct order, a message pops up with the final score. Then, the shelf disappears, leaving on the screen the Sun and the eight planets orbiting around it.

The second game refers to the chapter “Natural Environment” and includes two mini-games. In the first mini game, after scanning the appropriate image, the 3D model of the world map appears, consisting of different colors on each ocean so that they stand out (Figure 14). Additionally, the ship is the travel method for this game, which stands at a platform (port). The goal of this game is that the pupil learns the locations of the oceans and some interesting facts about each one of them.

First, the pupil must choose the destination from a dropdown list. The options are Pacific East, Pacific West, Atlantic, Indian, and Arctic. Then, by clicking the travel button, which is next to the dropdown list, the ship starts sailing towards the destination that is chosen by the pupil. When the ship reaches the desired destination, a panel shows up with information about this ocean. In addition, a button “Return to port” appears on the bottom of the screen, and by pressing it, the existing panel disappears, and the ship starts traveling from its current position back to the position of the port.

The next mini game is a quiz that gives the pupil the opportunity to test his/her knowledge (Figure 15). After scanning the image, random questions from a pre-set list appear on a school board. The list contains 17 questions in total, but in every quiz, 10 of them are randomly selected. The pupil chooses the correct answer between three or four options, depending on the question. On the bottom right of the screen, the score gets updated in real-time whenever the pupil answers a question. After answering all questions, a panel pops up providing the final score and a relative feedback message. The pupil has the opportunity to retry the test with different questions by pressing the “try again” button on the bottom left which appears simultaneously with the feedback.

The third game is from the chapter “Earth as a place of Human Life” and consists of three activities/mini games. In the first activity, after scanning the corresponding image, a 3D world map augments the real map which is located in the textbook (Figure 16). It is color coded to represent the population density (people per km²) for each country. On the bottom left of the screen, the pupil can only see the colors that represent each population range in millions (Light Green is 1–50, Yellow is 50–100, Blue is 100–200, Orange is 200–300, Red is 300+ and White is countries with no data), since it is not an interactive activity.

The second activity is kind of similar to the previous one (Figure 17). It is a 3D world map that represents the population of continents throughout the years. The data are drawn from a table that exists in the textbook stating the year and all the continent’s population at this time. In this activity, there is also a slider with which the pupil changes the currently selected year and sees the colors of the map change based on the population at the time. Again, on the bottom left of the screen, the pupil can see the colors that represent each population range in millions (White is 0–400, Light Green is 400–800, Yellow is 800–1200, Blue is 1200–2000, Orange is 2000–3000, Red is 3000).

The last mini game of the third game is a quiz (Figure 18). There is a list of 14 questions, out of which 10 are drawn randomly each time. The differentiation of this quiz from the previous one is that the answers are represented by 3D objects. There are four possible answers for each question and the pupil must select one of them. After the pupil answers all 10 questions, a panel pops up containing feedback depending on the final score.

Finally, the fourth game is based on the chapter “Continents” (Figure 19). After scanning the image from the textbook, the 3D model of the world map is revealed along with an air balloon. The purpose is to learn useful information about some countries. First, the dropdown list contains 14 countries. The pupil chooses one of these countries and then clicks on the “Travel” button, next to the dropdown list, to see the air balloon travel to the chosen destination. Once the air balloon reaches the destination, useful information about the chosen country is displayed on a panel, such as the flag of the country, the capital, the population, and a photo collage of some of its most well-known sights. Then, the same process can be repeated for another country.

3.3. AR App for 6th grade: Earth as a Celestial Body

The third AR app also refers to the pupils of the sixth grade, but it focuses more on teaching them the Earth. When the pupil opens the app, the main menu appears, which includes some general instructions on how to play the game (Figure 20). Below the instructions, there are seven buttons, each of which is a different game that corresponds to a specific chapter of the school textbook, according to the Greek curriculum.

The first game is called the “Orbit and rotation of the Earth” (Figure 20). When the pupil selects this game, the camera is activated, and s/he has to scan the target image that exists in the corresponding chapter of the textbook. Then, the augmented 3D objects, which are the Earth and Sun, are placed over the textbook. Firstly, a 3D animation of the Earth’s orbit around the Sun is shown, and afterward, a relevant question is displayed that the pupil has to answer by selecting one of the three buttons that are below the question. After choosing one, the pupil moves on to the next activity. Similarly, a 3D animation where the Earth rotates around itself appears and then the pupil selects one of the three answers (buttons) to answer the question. If the answer is either correct or not, appropriate audio and visual feedback is given to the pupil. The main buttons that are on the bottom of the screen during the whole game are a reset button, which restarts the game, and a menu button that leads to the main menu.

The second game refers to the “Poles, equator, parallel circles, and meridians of the Earth” (Figure 21). The camera turns on and the pupil scans the corresponding image from the textbook. The 3D model of the Earth appears, and the pupil is ready to play the game. The aim is to answer the questions one by one by selecting one of the three buttons that appear below each question. For example, one question could be what the horizontal or the vertical red ring around the Earth represents. The pupil selects each time one of the three answers and moves on to the next one. Similar to the first game, audio and visual feedback is provided to the pupil.

The third game contains material from the book chapter “Earth’s Axis and Rotation—Day and Night” (Figure 22). The pupil selects this game from the main menu and scans the corresponding image. The 3D model of the Earth appears on the screen. Additionally, a Greek flag is placed where Greece is located, and the Sun illuminates the Earth. The purpose of the game is to rotate the Earth so that night becomes day in Greece. Once the Earth is rotated correctly, the game is over, and the rotation feature is disabled. Then, a tick appears on the screen, followed by fireworks and a sound indicating to the pupil that the game is over. Additionally, there is an info button that consists of the game instructions and some relative information.

The next game is about “Our Solar System” (Figure 23). The game is selected by the pupil and after scanning the image from the textbook the game starts. Firstly, a 3D representation of our system appears, including the Sun in the center and all the planets in a row on the right. There are also some concentric circles (rings) which represent the distance of each planet from the Sun. Each circle has an available position in which the pupil must place the correct planet. Additionally, there is a dropdown list located at the top left of the screen, where the pupil can select the planet for which s/he wants to learn more, and the relative information appears at the bottom. As long as all the planets are at the correct distance from the Sun, the game is completed, and feedback is given to the pupil.

The fifth game refers to the “Atmosphere” (Figure 24). The same initial steps are required from the pupil to start the game, which are to select the specific game and scan the correct target image from the textbook. Then, the 3D model of the Earth appears on the screen along with five round semi-transparent objects around it, representing the layers of the atmosphere. To improve the aesthetics of the game, when the camera tracks the image, the animation starts, and when the animation stops, all the 3D objects are placed in their positions to start the game. The goal is to place the layers of the Earth’s atmosphere in the correct order. When the pupil selects a specific layer from the dropdown list, the corresponding information is displayed. In the case of placing a layer correctly, it overlays the Earth. Otherwise, the game does not allow the pupil to place the layer that s/he has chosen and tries again. In both cases, appropriate visual and audio feedback is provided.

The sixth game is about the “Regions of Vegetation” (Figure 25). After scanning the image, three 3D models of trees from different regions, as well as three round objects representing a different region, are displayed. The purpose of the game is to match the trees to the correct region. Matching the trees to the corresponding region correctly results in the game’s ending. Additionally, there is a dropdown list on the top left of the screen, and by selecting one of the three regions, information about each one is presented below.

The last game includes “Sights and Historical Monuments of Europe” (Figure 26). The appropriate image from the school textbook is scanned and a 3D model of a well-known monument appears on the screen. The purpose of the game is for the pupil to find out which country each monument is from. The pupil selects one of the three buttons that are displayed on the screen. When the right answer is selected, some information about the specific monument is presented as well as a “next” button, where clicking brings up the next monument and the next question. The game consists of four questions and the pupil has unlimited attempts. In case of a wrong choice, appropriate feedback is given. After the pupil answers all questions, the game is completed.

4. Method

4.1. Participants

Four experiments were conducted to evaluate the three proposed AR Geography apps to investigate their usability and their playful nature. The first three took place at three different primary schools in the region of Ptolemaida (Greece), where 43 pupils and six (6) teachers from the fifth and sixth grades participated in total (see

Table 1. Participants.

	5th Grade AR App: Greece	6th Grade AR app: Earth as a Living Space for Humans	6th Grade AR App: Earth as a Celestial Body	Total
Teachers	2	2	2	6
Pupils	15	14	14	43
Comp. Science Students	43	43	43	43
Total	60	59	59

). In the last one, 43 undergraduate students who attended the course of Human-Computer Interaction (HCI) in the Department of Information and Electronic Engineering at the International Hellenic University (Greece) had the right to participate. It is worth mentioning that pupils participated in the experiment with the role of end-user, teachers with the role of users who will integrate such technology into their teaching process, and computer science students with the role of game developers. Finally, all the participants were informed about the experiment, consented (in the case of pupils, consent was given by their parents), and participated voluntarily.

4.2. Procedure and Material

The three experiments that took place in the classrooms of three different primary schools lasted 45 min, while the duration of the experiment conducted in the university lecture hall was 90 min since university students had to evaluate all three AR apps (Figure 27). Regarding the first three experiments, pupils were randomly assigned to dyads, and with the help of the researchers, the AR app was installed on schools’ ICT equipment such as Android mobile devices. Then, the instructors (teachers) gave a relevant introductory stimulus. Next, the pupils opened the AR Geography game, and according to the game’s specifications, they had to scan either a QR Code or an image. After scanning, they played the game and, in the end, appropriate feedback was given to them according to their performance. Regarding the 5th grade AR app, the game that was chosen was the “Types of maps” where the pupils had to answer a quiz with random questions that refer to the different types of maps. For the 6th grade AR app: Earth as a living space for humans, the game “Continents” was chosen where an air balloon travels to different countries, and information regarding these countries is displayed. Finally, for the last AR app for the 6th grade: Earth as a celestial body, the game “Atmosphere” was chosen, where the pupils had to place the layers of the atmosphere in the correct order. At the end, pupils had the opportunity to discuss what they have learned with their instructors (teachers).

The procedure of the experiment that was conducted in the university lecture hall was the same, with the difference that there was neither introductory stimulus, nor instructors, and that university students played and evaluated all three AR apps, one after the other.

After each experiment was completed, a questionnaire was given to all participants in printed form to evaluate the corresponding AR app. The questionnaire, also known as the System Usability Scale (SUS) is a state-of-the-art method in the user interface design field [35] and includes 10 questions to evaluate the usability of apps. The questionnaire uses a Likert scale of 1 (Strongly Disagree) to 5 (Strongly Agree) and the Greek version of it was used [36]. To calculate the overall SUS score, firstly the total score for all odd-numbered questions is added and then 5 is subtracted from the total to get (X). Next, the total score for all even-numbered questions is added and then that total is subtracted from 25 to get (Y). Finally, the total score of the new values (X + Y) is added and multiplied by 2.5 to convert the original scores of 0–40 to 0–100. Following these steps, the scores of participants for each question are converted to a number. Though the scores are 0–100, these are not percentages and should be considered only in terms of their percentile ranking. Based on the literature, SUS scores above 70 are considered above average and are acceptable, and anything below 70 is below average. According to the grading scale, SUS scores that are below 60 are considered as an “F”, between 60 and 69 as a “D”, between 70 and 79 as a “C”, between 80 and 89 as a “B”, and 90 and above as an “A” [37,38,39].

In addition, one more question was added to measure whether the proposed AR Geography apps look like games or not. Finally, apart from the questionnaire, data were also collected through personal observation occurred by the researchers throughout participants’ interaction with the AR apps.

5. Results and Discussion

5.1. Evaluation of Quantitative Method

To measure the usability of the three AR Geography apps, a System Usability Scale (SUS) was performed.

Table 2. Usability scores.

	5th Grade AR App: Greece	6th Grade AR App: Earth as a Living Space for Humans	6th Grade AR App: Earth as a Celestial Body
Teachers	73.75 (SD 12.37)	82.50 (SD 3.68)	73.75 (SD 5.30)
Pupils	70.50 (SD 15.98)	70.00 (SD 14.61)	76.07 (SD 9.69)
Comp. ScienceStudents	82.26 (SD 11.47)	85.87 (SD 12.12)	77.44 (SD 18.54)
Total *	79.04 (SD 13.57)	81.99 (SD 14.15)	76.99 (SD 16.47)

* Total refers to the total SUS score when the total sample of each AR app is considered.

presents the SUS scores per user group (teachers, pupils, and computer science students) and per AR app, which are acceptable since they are above 70 [37]. It is worth mentioning that the AR app for the 6th grade: Earth as a living space for humans obtained the highest SUS scores by teachers (82.50) and computer science students (81.99), while it obtained the least satisfactory scores from the pupils (70.00). A possible explanation for this might be that the “Continents” game that was chosen for the experiment, focused more on how to present useful information in an understandable and playful way–features that seem to be important for teachers and computer science students as game developers–rather than the interaction–a design feature that seems to be important for the pupils’ engagement.

Additionally, the total SUS scores are very satisfactory since all the total scores are above 75. The total SUS score per AR app refers to the SUS score that is calculated from the total sample. For example, in the case of the AR app for the 5th grade, the total sample consisted of 60 participants, who rated the specific AR app with a total SUS score of 79.04.

More specifically,

Table 3. Mean values and standard deviations of participants’ responses for the AR app for 5th grade: Greece.

Item	English Version of SUS	Pupil Responses Mean (Std)	Teacher Responses Mean (Std)	Comp. Science Students Mean (Std)
Q1	I think that I would like to use this system frequently.	4.07 (1.03)	4.50 (0.71)	3.95 (0.87)
Q2	I found the system unnecessarily complex.	2.00 (0.93)	2.50 (0.71)	1.44 (0.55)
Q3	I thought that the system was easy to use.	4.00 (1.00)	4.00 (0.00)	4.23 (0.75)
Q4	I think that I would need the support of a technical person to be able to use this system.	2.40 (1.24)	3.50 (0.71)	1.65 (1.04)
Q5	I found that the various functions in this system were well integrated.	4.27 (0.88)	4.50 (0.71)	4.14 (0.71)
Q6	I thought that there was too much inconsistency in this system.	2.40 (1.06)	2.50 (2.12)	1.77 (0.95)
Q7	I would imagine that most people would learn to use this system very quickly.	3.73 (0.96)	4.50 (0.71)	4.51 (0.51)
Q8	I found the system very cumbersome to use.	2.13 (1.41)	1.50 (0.71)	1.46 (0.80)
Q9	I felt very confident using the system.	3.60 (0.99)	4.00 (0.00)	4.05 (0.87)
Q10	I needed to learn a lot of things before I could get going with this system.	2.53 (1.25)	2.00 (0.00)	1.65 (0.78)

,

Table 4. Mean values and standard deviations of participants’ responses for the AR app for 6th grade: Earth as a living space for humans.

Item	English Version of SUS	Pupil Responses Mean (Std)	Teacher Responses Mean (Std)	Comp. Science Students Mean (Std)
Q1	I think that I would like to use this system frequently.	3.43 (1.09)	4.50 (0.71)	4.00 (0.90)
Q2	I found the system unnecessarily complex.	2.57 (1.28)	1.00 (0.00)	1.39 (0.73)
Q3	I thought that the system was easy to use.	4.14 (0.86)	4.50 (0.71)	4.39 (0.62)
Q4	I think that I would need the support of a technical person to be able to use this system.	2.79 (1.19)	3.50 (0.71)	1.28 (0.50)
Q5	I found that the various functions in this system were well integrated.	3.93 (0.92)	4.50 (0.71)	4.26 (0.85)
Q6	I thought that there was too much inconsistency in this system.	2.14 (1.03)	1.00 (0.00)	1.56 (0.80)
Q7	I would imagine that most people would learn to use this system very quickly.	4.29 (0.62)	4.50 (0.71)	4.39 (0.62)
Q8	I found the system very cumbersome to use.	1.64 (0.84)	1.00 (0.00)	1.46 (0.70)
Q9	I felt very confident using the system.	3.64 (0.84)	4.50 (0.71)	4.42 (0.59)
Q10	I needed to learn a lot of things before I could get going with this system.	2.29 (0.99)	3.00 (1.41)	1.42 (0.66)

and

Table 5. Mean values and standard deviations of participants’ responses for the AR app for 6th grade: Earth as a celestial body.

Item	English Version of SUS	Pupil Responses Mean (Std)	Teacher Responses Mean (Std)	Comp. Science Students Mean (Std)
Q1	I think that I would like to use this system frequently.	3.57 (1.16)	4.50 (0.71)	3.77 (1.00)
Q2	I found the system unnecessarily complex.	1.86 (1.03)	2.50 (2.12)	1.79 (0.91)
Q3	I thought that the system was easy to use.	4.36 (0.63)	4.00 (0.00)	3.93 (0.94)
Q4	I think that I would need the support of a technical person to be able to use this system.	2.21 (0.80)	3.50 (0.71)	1.74 (1.05)
Q5	I found that the various functions in this system were well integrated.	4.14 (0.66)	4.50 (0.71)	4.14 (0.74)
Q6	I thought that there was too much inconsistency in this system.	1.71 (1.07)	1.50 (0.71)	1.67 (0.99)
Q7	I would imagine that most people would learn to use this system very quickly.	4.14 (0.86)	4.00 (0.00)	4.02 (0.96)
Q8	I found the system very cumbersome to use.	1.43 (0.65)	1.00 (0.00)	1.79 (0.96)
Q9	I felt very confident using the system.	3.57 (1.09)	4.00 (0.00)	3.91 (0.92)
Q10	I needed to learn a lot of things before I could get going with this system.	2.14 (1.17)	3.00 (1.41)	1.79 (0.91)

present the mean values and standard deviations of the responses that participants rated for each AR Geography app. Regarding the AR app for the 5th grade (see Table 3), the majority of the participants (pupils, teachers, and university students) would like to use this app frequently and they found the app neither complex nor cumbersome to use.

For the AR app for 6th grade: Earth as a living space for humans (see Table 4), the teacher–in contrast to the pupils and computers science students–supported that they would need help from a technician to use the system as well as they might need to learn a lot of things before teachers will be able to use it. This might be explained by the fact that one of the teachers did not have a satisfactory level of expertise and familiarity with such technology. In addition to this, another explanation might be the limited sample, which consists of only two teachers.

As far as the AR app for 6th grade: Earth as a celestial body is concerned (see Table 5), the majority of the participants agreed that the AR app is easy to use and, in general, they felt confident using it, since the AR app did not present any inconsistency.

Table 6. Mean values and standard deviations of the statement “It looks like a game”.

	5th Grade AR App: Greece	6th Grade AR App: Earth as a Living Space for Humans	6th Grade AR App: Earth as a Celestial Body
Teachers	4.00 (SD 0.00)	5.00 (SD 0.00)	5.00 (SD 0.00)
Pupils	3.93 (SD 0.70)	3.71 (SD 0.65)	4.07 (SD 0.92)
Comp. Science Students	4.53 (SD 0.55)	4.37 (SD 0.72)	4.40 (SD 0.85)

presents the results regarding the extra question that was added to the questionnaire and referred to whether the proposed AR apps look like games (see Table 6). The participants highly rated all three AR Geography apps as game-like. More specifically, the results in the AR app for 6th grade: Earth as a living space for humans, are in line with the previous findings, as pupils rated it with the least mean value, in relation to the rest AR apps, while teachers and university students with the highest ones.

Taking into account the previous results, all three AR apps obtained satisfactory ratings regarding usability, since the majority of the participants claimed that AR apps are easy to use and felt confident while using them. This could be due to the fact that the majority of the participants, including pupils, are familiar with such devices and technologies–a fact that made the experiment easier to be accomplished. Finally, participants supported that the proposed AR apps are playful because their purpose is to learn through a game. These conclusions are coherent with the literature review, which found that AR games in the domain of Geography are attractive, fun, and satisfactory [19], enhancing the learner’s experience [21].

5.2. Evaluation of Qualitative Method

All four experiments were conducted successfully, and the procedure was followed smoothly. Based on researchers’ personal observations, pupils were very excited, and after completing the experiment, they requested to play another AR game. Another point that researchers noted was that two (2) out of six (6) teachers did not have the requisite computer literacy to support the experiment as instructors, a condition that might have caused some problems. However, because of the pupils’ familiarity with the technology, which is due to the AR games such as Pokémon GO, they successfully completed the experiment without their motivation being reduced. Even in the case of a teacher who had the expertise and attempted to advise the pupils on how to use the AR app, this attempt was rejected by the pupils who, as digital natives, did not require any specific instructions.

Teachers’ acceptance of the AR technology was wide since they were impressed by the impact the AR apps had on pupils. It should be mentioned that, according to teachers’ comments, the average score of correct answers of pupils while using the AR app was higher compared to the traditional textbook. Furthermore, teachers found AR apps attractive and motivational. As a result, all teachers, including those who did not have the same level of computer literacy, expressed interest in AR technology and willingness to integrate it as an auxiliary tool into the teaching process. In addition, they claimed that AR could also be expanded in other domains, not only in Geography, since it could be a useful auxiliary tool for the teaching process.

Finally, regarding the computer science students, the majority were pleased with the design and development of the AR apps, however, they mentioned some improvements–such as having the exit button in a visible place on all screens and having less text so as to be more readable.

6. Conclusions and Future Work

Leveraging the aforementioned, the current study investigates the usability of three proposed AR apps for teaching Geography in the 5th and 6th grades of primary school. The results that were extracted from the SUS test and observations were very promising. All user groups (pupils, teachers, and computer science students) found the AR apps satisfactory and highly usable. Specifically, pupils were excited about the experiment, since they found that AR apps are easy to use, attractive, and playful. Furthermore, teachers stated that AR technology could be a very useful tool and they would like to know how to incorporate such technology into their teaching process. However, some of them were not familiar with such technology and agreed that they could need help from a technician, but this fact did not affect their acceptance of the proposed AR apps. Finally, computer science students found the design appealing, even though they mentioned some improvements. To summarise, there was excitement and acceptance for AR technology, as all the participants used the proposed AR-based applications without hesitation. Overall, average pupils’ performance, according to teachers’ comments, was higher when compared to the traditional assessment methods. These conclusions are in accordance with the literature review, which indicated that AR technology can be motivational [4,26,28], interesting [22,25], fun [19], and can enhance learning outcomes [24,29].

Nevertheless, the proposed AR Geography apps have some limitations that should be taken into consideration in future work. Firstly, some design issues must be improved, such as some buttons must be visible throughout the game and the text has to be reduced to be more readable to the users. Secondly, it is worth mentioning that the computer science students were the same who evaluated all three AR apps. Therefore, the case in which students might have been influenced by the previous AR apps while evaluating one AR app was not investigated in this work and would be interesting to study in future work. Additionally, the innovation of the proposed AR apps does not lie in the design and implementation, but in the fact that they are applied in a field–that of teaching Geography in primary school–which according to the literature review has a limited number of research studies to investigate the usefulness of such technology. Moreover, since it is preliminary work, larger-scale experiments could be very useful to extract more valuable conclusions. Finally, future plans will include the investigation of whether the learning outcomes and experience of the user will be increased or not through long-term studies, resulting in a more robust AR methodology for teaching Geography.