Keywords

1 Introduction

The rapid development of technology and the rapid growth of information society are changing the paradigm of education. Virtual reality (VR) is used in various fields such as games, education and training, and medical technology development [1]. The scope of use of VR education is increasing, and it is expanding to various fields used in our daily life. Virtual reality is widely used not only for general education, but also for job training in industrial fields, medical, military, and enterprises. In particular, it is attracting attention as an experiential content that provides a new learning experience by integrating virtual reality and artificial intelligence(AI) technology into educational content. It is possible to make an experiential education that can be experienced directly by moving to the place where it is difficult to escape from the limitations of space and observing the process of learning principles being executed. The learner actively participates in learning through critical thinking and imagination through experiential learning in virtual space [2, 3]. In the implementation of learning content, it will be necessary to provide a natural interaction between the user and the system by utilizing virtual reality to induce immersion in the learner and to lead a learning motivation and active learning effect.

Users interact with the system through the user interface (UI) and are crucial for systems that require intuitive interactions such as virtual reality. The primary purpose of UI design in the virtual reality will be to allow the user to naturally interact with the content and use it easily and conveniently. Virtual reality has the advantage of providing an immersive feeling through the realistic interaction between the user and the system and feeling the presence that can feel as if they are in the virtual world [4]. In particular, it maximizes the immersion of the user by supporting the user’s natural interaction with the computer-generated environment. To induce the immersion of the user, a UI that can naturally be explored and interacted as in reality is required.

The purpose of this study is to analyze the components of the diegetic UI that can be applied to the VR environment and to suggest the utilization method of the VR content that can induce the sense of immersion. In this paper, we introduce our VR learning program for the constellation that can enhance the user’s sense of immersion using the diegetic UI in VR environment.

2 Design Approach

In this project, we aim to expand students’ curiosity about science and their scientific imagination by developing VR content that utilizes interesting space science materials that can be observed in everyday life. We implemented the exploration of space in the form of experience related to the constellations of the universe through the natural interaction with the user using virtual reality. We have made general scientific common sense about the universe and Greek myths related to constellations into interesting VR content in conjunction with storytelling. The player experiences the stories of the constellations by exploring the constellations of the universe, starting with an event that teleports into space from an astronomical observatory where he accidentally enters. Figure 1 shows the learning model of a constellation.

Fig. 1.
figure 1

The learning model of constellation

Full size image

3 Diegetic User Interface

The diegetic user interface is the part of the scene and fiction of the virtual world [5]. Diegetic elements exist within the virtual world, and the player can directly manipulate the elements that make up the virtual world through the avatar’s perspective. It is to configure the interface so that the player can interact with all of the internal elements such as characters, backgrounds, props, and sounds that exist in the virtual world that can be seen or heard. There are many virtual reality research results show that it is more efficient to create an interface with diegetic elements existing in the virtual world than a non-diegetic user interface represented by a typical GUI [6,7,8].

In this project, we classified the diegetic UI which is mediated as an element in the VR environment into the place, agency and sensory.

3.1 Place

A place is a diegetic UI element that creates new relationships and experiences between the player and the virtual world, and the realm can expand to serve as a role to create a new level of experience for the player. The astronomical observatory is the place where players start in this project and is a familiar place in reality. Players can interact and navigate by picking up, touching, and interacting with objects in space. From the moment the HMD is worn, the player is completely blocked from viewing and enters a virtual world different from that seen in reality. The player’s first experience starts with a place that suggests how the story will unfold in the future. That is, it provides the user with a preparation step to immerse himself in the story. The universe is the second place where player can explore the constellations. By providing an experience of a fantastic place that is not commonly experienced in everyday life, it inspires players and allows them to become immersed in the real world. As a diegetic UI mediated as a place, it allows players to go beyond fantasy and reality to recognize that they have entered the virtual world rather than reality and become a subject in the virtual world. Besides, if we organize a place according to the progress of the story, we can make players experience dramatic inspiration. Figure 2 shows the diegetic UI mediated as a place in this project.

Fig. 2.
figure 2

Screenshots of diegetic UI mediated as a place

Full size image

3.2 Agency

According to a locus of control, we classify an agency as an agent and an avatar [9]. The player controls the avatar to become the subject of the story, and also helps the story progress from the agent. The avatar gives the player the freedom to move around and observe the view and space from the first-person point of view so that the player becomes an imaginary person and feels immersed. It is essential to make the player immersive and interact in harmony with the virtual environment as if acting in reality. Through these experiences, you can make the player the subject of the story, experience the narrative structure, empathize with the situation, and induce empathy. Agent refers to a character or object controlled by a computer system, which means an animal, robot, or human in a virtual environment. An agent is the AI of the control panel in this project, and the player starts from the beginning by listening to the voice of the AI coming from the speaker of the control panel. Although it is the face and voice of the AI appearing on the screen attached to the controller, it makes the story smooth and improves the understanding of the situation. Even when moving from the observatory to outer space, the voice of AI continues to be heard and acts as an auditory medium connecting the real world with the fantasy world. Figure 3 shows the diegetic UI as an agent in this project.

Fig. 3.
figure 3

Screenshot of diegetic UI mediated as an agent

Full size image

3.3 Sensory

Sensory elements used to induce natural interaction and effective responses to the player. In virtual reality, players can refine their experiences by viewing, playing, and feeling perception cues related to their experiences. The main elements of perceptual immersion were visual and auditory. The visual and auditory elements focus the player’s attention and provide clues. As a visual element in this project, a flickering effect was created on the button of the control panel to draw the attention of the player. It also provided clues for progress by faintly flashing the constellation shape when the player points to outer space. The visual elements generated effects on the objects that the player can interact with and drew the player’s attention and provided clues to proceed with the story.

As the auditory element, the focus is on the moment of reaction that occurs in the direct interaction process between the user and the virtual environment, and it is produced with a diegetic sound existing in the virtual world so that heterogeneity does not occur. Perceptual immersion can improve through the synchronization of visual and auditory. When the player selects a constellation, the picture of the constellation overlaps and appears. Then, the agent told Greek myths related to the constellations to enhance the learning effect with interesting stories about the constellations. The agent that appears from the beginning of the story, although not visually expressed, accompanies the player and provides the necessary information. Also, the player can feel the social presence of interacting with a computer-generated agent and interacting with someone in the virtual world. Figure 4 shows the diegetic UI as a visual effect in this project.

Fig. 4.
figure 4

The visual effect of constellation display

Full size image

The diegetic UI mediated as sensory induces direct interaction with the objects necessary for the story progression and conveys information to the player.

4 Conclusion and Future Work

In this study, we analyzed the diegetic UI that can be applied to the virtual reality environment and suggested a method to induce immersion. When we create a UI in virtual reality, we try to exclude the creation of a GUI that exists outside the virtual world and induce immersion as a diegetic UI that allows the user to interact naturally with all components in the virtual world. We classified the diegetic UI that is mediated as an element in the VR environment into the following three types. First, the diegetic UI mediated as a place can provide an immersion with a realistic experience as a friendly place where we can visit easily even in ordinary times or a fantastic experience as an imaginary place that is hard to experience in reality. Second, the diegetic UI mediated as an agency plays a role in virtual reality in which the character becomes the subject of the story or helps the story progress in events that occur in the narrative. Third, the diegetic UI mediated as sensory is not limited to audiovisual as a means of interaction in virtual reality, but it enhances immersion by increasing interaction by utilizing all human senses. As visual cues and auditory cues that exist in virtual reality, it serves as a clue to the player and helps the story progress.

As future work, we plan to utilize the tactile sense to maximize the user’s immersion.