Systematic Review of Comparative Studies of the Impact of Realism in Immersive Virtual Experiences

Some early definitions of Virtual Reality (VR) include: “Virtual Reality is electronic simulations of environments experienced via head-mounted eye goggles and wired clothing enabling the end-user to interact in realistic three-dimensional situations,” by Coates [16], and “Virtual Reality is an alternate world filled with computer-generated images that respond to human movements. These simulated environments are usually visited with the aid of an expensive data suit which features stereophonic video goggles and fiber-optic data gloves,” by Greenbaum [32]. A more recent definition of VR from Fuchs et al. [26] is: “Virtual reality is a scientific and technical domain that uses computer science and behavioural interfaces to simulate in a virtual world the behaviour of 3D entities, which interact in real-time with each other and with one or more users in pseudo-natural Immersion via sensorimotor channels.” In this work, we consider Fuchs et al. definition. However, because our work focuses on the higher end of immersion, it is constrained to include stereoscopy devices such as HMD or CAVEs. IVEs can originate from different immersive technology beyond VR. Thus, we also considered the Mixed Reality (MR) spectrum. Considering the virtuality continuum [76], MR consists of merging virtual and real-world stimuli [26, 76]. AR consists of overlaying virtual elements over the real world, positioning it closer to the real-world end of the continuum, while VR is on the opposite side of the continuum (purely virtual). MR, therefore, encompasses everything in between, including the known Augmented Reality (AR).

The sense of presence seems to be one of the main results of virtual experiences. It has been used in the literature as a metric to evaluate the user experience and consequently the virtual environments, guiding future research. Several authors defined presence in different ways throughout the literature. Gibson [28] referred to presence as “the feeling of being in an environment”; Witmer and Singer [124] considered presence as being the “the subjective experience of being in one place or environment, even when one is physically situated in another”; Slater and Wilbur [112] defined presence as “a state of consciousness, the (psychological) sense of being in the virtual environment.” According to the authors, users who feel higher levels of presence in a virtual experience are also more prone to behave similarly to how they would in a real situation. Harter et al. [37] consider that the same autonomic reactions users have on real situations may also happen in virtual experiences as long as there is the suspension of disbelief (even though users know the virtual experience is not real, they suspend disbelief). David et al. [86] in his work about VR training with firefighters, considers that this mental suspension of disbelief is related to higher levels of presence. These results suggest that presence is particularly crucial in VR training scenarios, requiring users to behave in the IVE and real life in the most similar way possible. However, note that realistic behavior is not presence but rather a sign of it [104].

Several conceptualizations of presence have emerged throughout the literature. An explanation of several of those can be found in Lombard and Ditton’s work [60]. Lombard et al. introduce presence as realism: “a medium can produce seemingly accurate representations of objects, events, and people—representations that look, sound, and/or feel like the “real” thing” [60]. The authors also argue that this concept can be confused with “social realism,” which refers to the extent to which a media is plausible by reflecting events that could occur in a non-mediated world. Although presence as realism can include “social realism,” it also includes “perceptual realism.” For example, in a sci-fi scene, some events are unlikely to happen in reality, and therefore the social presence can be low. Nevertheless, if people and objects behave as expected in the same scene, then perceptual realism would have a high value even if the social presence is low.

In 2009, Mel Slater [105] introduced two constructs: “place illusion” (PI) and “plausibility illusion” (PsI). PI is considered the “strong illusion of being in a place despite the sure knowledge that you are not there,” which is equivalent to what is usually called presence. The author considers PI as being binary; either there is an illusion or not, there cannot be a partial illusion. PsI is the “correlations between external events not directly caused by the participant and his/her sensations (both exteroceptive and interoceptive).” PsI does not require physical realism as people can still show signs of anxiety and other emotions depending on what is happening in the virtual experience, even though virtual realism is low. An example is the Stanley Milgram obedience experiment [77], where users would show signs of anxiety when causing pain to a non-realistic virtual avatar [107]. According to Mel Slater, if we experience both PI and PsI, then we are more likely to respond as if the situation was real.

Although presence is closely related to realism, Baños et al. [3] argue that reality judgment (“the belief that our experiences are real”) is a construct different from presence. This means that users can consider an IVE real even if they do not feel any sense of presence and vice versa. Under this definition, Skarbez et al. [103] further suggest that reality judgment is synonymous with PsI.

The term immersion in the context of VR is often confused with presence, being used sometimes interchangeably [103]. According to Slater et al. [112], immersion refers to the technology (objective characteristics of the systems that make a virtual experience possible) giving an inclusive (extent to which the user is isolated from physical reality), extensive (number of sensory stimuli being used), surrounding (field of view), and vivid (richness, information content, resolution, and quality of the displays) experience to users. The authors [112] also state that immersion requires a virtual body as it is part of the perceived environment and the one that is doing the perceiving. This immersion definition can also be referred to as “perceptual immersion” [6]. In their work from 2009, Slater et al. [105] further extended the immersion definition, characterizing it by the supported sensorimotor contingencies (SCs). SCs are actions we perform to perceive the environment around us (e.g., looking around, crouching, bending down, jumping). Valid sensorimotor actions are actions allowed by the system, resulting in the images being updated accordingly. Valid effectual actions are actions users can take that result in changes in the environments. Both sets of actions are called valid actions, which users can perform, resulting in changes in their perception or modifications to the virtual environment. For example, if users try to bend down, but the HMD only allows for rotational movements, then the image will not be updated accordingly; therefore, this action would not be a valid sensorimotor action. Similarly, if the user tries to grasp a virtual object but there is no hand tracking, this action would not be a valid effectual action. Slater et al. [105] state that an ideal immersive system should be able to simulate a non-immersive system fully. For example, an immersive system comprising an HMD, could simulate a non-immersive system such as a desktop monitor, mouse, and keyboard. Still, the other way around would not be possible. Note that Slater defines immersion as purely objective and dependent on the physical properties of the system.

A contrasting definition of immersion comes from Witmer and Singer [124]. They define immersion as “a psychological state characterised by perceiving oneself to be enveloped by, included in, and interacting with an environment that provides a continuous stream of stimuli and experiences.” This definition can also be called “psychological immersion” [61]. We consider Mel Slater’s immersion definition for this works’ scope because it clearly separates presence (subjective feeling) from immersion (objective characteristics of the system).

Several definitions have been given to fidelity. Franzluebbers and Johnsen [24] considered fidelity as how well the object physical properties are replicated. Meyer et al. [75] state that fidelity is a measure of how well the simulation represents the real world. To objectively evaluate fidelity, the authors affirm that a “referent” is required—“an abstract description of the real world that defines reality in a level of detail and format that makes a meaningful evaluation possible.” Because the real world is too complex to be compared, a referent would define reality through an abstraction of the real world, to which the simulation system can be compared to and evaluated.

Bowman and McMahan [20] discuss that realism comes from high-fidelity sensory stimuli in successful immersive VR applications. McMahan et al. [72] divide a VR environments’ overall fidelity into display fidelity (exactness in which the real-world stimuli is represented) and interaction fidelity (exactness that the real-world interaction can be reproduced). The authors also concluded that both constructs are important to determine performance, presence, engagement, and usability.

Alexander et al. [2] define fidelity as how well the real world is emulated. The author also defines various subcategories of fidelity, which can be divided into three subcategories. Physical Fidelity refers to how the simulation is physically similar to the real environment being replicated. However, a virtual environment can look similar to the real one until we try to interact with it. Functional Fidelity defines how similar the functionality of the virtual environment is to the real one. Last, Alexander et al. [2] describes Psychological Fidelity, which, as the name describes, defines how close the simulation can replicate the psychological factors that happen in the direct real environment.

Schricker et al.’s [99] definition is similar to Meyer et al. [75], as they describe fidelity as the extent to which a simulation represents its referent. The referent is an abstraction that is as close to the real world as possible, with no concern for design issues or resource limitations while a model usually will not be as close to the real world because it must make these considerations. However, the model is what it’s actually developed, and therefore, will not be as close to the real world, because it must consider the inherent limitations.

On the fidelity referent topic, Roza et al. [97] considers it as a “formal specification of all knowledge about reality plus indicators to determine the uncertainty levels and quality of this knowledge to judge the confidence level of this referent data.” The authors state that for Fidelity to be measured it must consider a common referent to be compared to.

Similarly to Schricker et al. [99], Meyer et al. [75], Roza et al. [97], and Hughes and Rolek [41] define fidelity as how well the attributes and behaviors of the referent are reproduced. Regarding fidelity measurement, Vincenzi et al. [83] describe two main methods to do it. We should note that fidelity for his work context is related to simulations (e.g., airplane cockpits). One method to measure fidelity can be a mathematical calculation of how many identical elements are shared between the real and virtual worlds. A higher simulation fidelity is then the result of a greater number of shared identical elements. The other methods refer to the trainee’s performance. By evaluating the transfer of knowledge through comparing his/her performance in the simulation with the real world, it is possible to measure fidelity indirectly.

Coherence is defined by Skarbez et al. [102] as “the set of objectively reasonable circumstances that the scenario can demonstrate without introducing objectively unreasonable circumstances.” The authors link coherence to PsI, as both depend on the users’ expectations and past experiences and highly depend on the specific virtual scenario and/or software. It does not require the virtual experience to replicate the real world. The authors further divide coherence in physical coherence (laws of physics) and narrative coherence (how virtual agents and scenario meets the users’ expectations as they know it from everyday life) which seems to overlap with Lombard et al.’s perceptual realism [60]. Skarbez et al. [103] also discuss that coherence can not be purely objective, as it will always depend on the user’s past knowledge and experiences. However, they consider that it would be helpful to consider as if it was objective, because developers cannot control the user’s prior knowledge. Still, they can control whether or not the virtual experience events are internally consistent. Furthermore, developers can further minimise how dependent coherence is on their prior knowledge by hinting the user to expect certain behaviors.

The term authenticity is considered by Gilbert [29] as how well the virtual experience is within the expectations of the user consciously and unconsciously. Malliet et al. [66] referred to authenticity as a constituent of perceived video realism, which was based on Hall’s [35] typicality (events and behaviors related to everyday life) and plausibility (events and behaviors that can potentially occur in the real world) notions. Malliet et al. [66] consider the sense of realism as “authenticity” where it “relates to the belief gamers have in a game designer’s intention and ability to express something real.” For example, the simulation of human emotion or the virtual characters and storylines of humanness is considered the authenticity construct. Skarbez et al. [103] discussed how Gilbert’s authenticity is similar to coherence but distinct as it assumes the virtual experience is trying to replicate the real world on the contrary to coherence.

Credibility is considered by Boucaud et al. [10] as the degree to which the user felt the virtual agent behaved in a human-like way. The scope of his study was about social touch in human-agent interaction. Building on their definition, the more overall meaning of credibility would be if the virtual world behaves similarly to the real world, which ultimately is equivalent to Gilbert’s authenticity and similar to the definition of Malliet et al. Bishop et al. [7] stated that the virtual experience would become more credible as data, computer display environments, and haptic feedback improve. Although the authors did not directly define credibility, it seems to be a function of the system capabilities (immersion/fidelity). Gonçalves et al. [30] used a dictionary definition to define credibility as “the quality of inspiring belief,” which is necessary to develop a sense of presence. This definition is equivalent to coherence and PsI, where there are no assumptions that the virtual environment is trying to replicate reality, but instead if the events and the environment itself follows its internal logic and meets the users’ expectations for the given context.

In the scope of visual stimulus, Chalmers and Ferko [13] considered realism in real-time the “holy grail” of computer graphics. VR systems aim at providing users with both realism and real-time update of visual stimuli. Creating a single realistic computer rendered image uses fewer resources than producing the same level of realism at a high and constant frame rate required by IVEs. There is a tradeoff between quality and system performance. Usually, realism is reduced to achieve decent performance. Authors state that no one can precisely define what realism is. They introduce a way of considering realism in virtual environments—Levels of Realism (LoR). This is the level necessary to achieve the same experience in the virtual environment as in the real environment. These can be particularly useful in training simulations to guarantee that trainees do not adopt different strategies just because they are in a virtual environment, which would ultimately harm their real-world performance in the same task. The authors also introduce the concept of believable realism. This is achieved by introducing imperfection in the virtual environment. Usually, computer-rendered images look intact and clean, which does not happen in the real world. Stains, dust, scratches, and other imperfections can affect the sense of perceived realism [62].

Ferwerda [23] defined three types of realism in computer graphics: physical (“image provides the same visual stimulation as the scene”), photorealism (“image produces the same visual response as the scene”), and functional realism (“image provides the same visual information as the scene”). Physical realism requires that virtual scenes be objectively the same as the real scene. Such would require the geometry, textures, lighting, and so on, to be done with extreme accuracy. A photorealistic image is an image indistinguishable from a real photograph of a real scene. Note that a photograph is a representation of reality and not reality itself. Introducing certain artefacts present in real photographs in computer-generated ones can actually increase its photorealism [62]. Functional realism is an image that provides meaningful information about objects’ properties, allowing users to perform real tasks. An example would be instructions to put together pieces of furniture.

Chalmers and Ferko [13], using physics and believability as “dimensions,” classify realism into four quadrants: Is Believable (IB), Not Believable (NB), Is Physics (IP), Not Physics (NP). For example, a VR system for training should fall in the IP quadrant, as it should introduce real-world physics, but it can (IB) or not be believable (NB). Ferwerda’s Functional realism and Uncanny Valley Avatares (explained ahead) are placed into NBIP, because they are physically correct but are not believable. Photographs and Photorealism are placed in IBIP, whereas VR is placed between IBIP and NBIP.

Perroud et al. [88] consider five acceptations of realism:

Furthermore, Hoorn et al. [40] give another realism definition stating that if the simulation’s goal is achieved, then the simulation was realistic enough. They mention that there is a lack of objective methods to evaluate physiological realism. They proposed a model to evaluate physiological realism using a driving simulation context considering the human visual system skills. The authors point out that the model needs to go through more experimentation to validate the scoring system fully.

According to Slater et al. [108], visual realism can be divided into geometric realism (how similar the virtual object is to its real counterpart) and illumination realism (how realistic the lighting is). Their study concluded that a higher sense of presence is more likely with real-time ray tracing than with ray casting. In the follow-up study, Yu et al. [129] concluded that the previous results were due to dynamic shadows and reflections. Also, that illumination quality did not affect presence, but global illumination did result in greater plausibility. Such did affect participant responses leaving the authors to conclude that it is worth introducing global illumination. Hvass et al. [42] conducted a study comparing low vs. high visual realism (through polygon count and texture resolution). The results suggested that a stronger feeling of presence develops when participants are exposed to heightened visual realism. Furthermore, self-reports and physiological measures also indicated a stronger fear response in higher levels of realism, which leads authors to conclude that such may reveal a stronger presence. The sense of presence seems to be a well-established metric and a relevant factor in evaluating perceived realism. Studies seem to indicate its increase usually leads to a higher sense of presence. However, there are cases where realism might not lead to presence and where realism without presence is enough. Bowman and McMahan [20] discussed one example: the oil and gas industry, where users need to perceive complex 3D structures to make the right decisions, not requiring a high feeling of presence but rather a high level of realism. Another possible issue with higher realism is the Uncanny Valley [82]—the emotional response of a person over an avatar reassembling a real human being. Widely used across the robotics and computer graphics fields, it explains the phenomena where users’ feeling of empathy increases with the degree that an avatar reassembles an actual human until this feeling shifts abruptly to revulsion in the so-called “valley.” However, if the reassembling keeps increasing, then the feeling of empathy returns. When designing virtual experiences with human-like elements, such as virtual agents or self-avatars, the uncanny valley might negatively affect the virtual experience if their realism is brought up to the point where it is eerily similar to a human being. In these cases, it could be better to use a less-realistic avatar or virtual agent.

Considering author definitions of realism, we theorize that realism can be both subjective and objective. Subjective realism depends on how users perceive the environment. This means that for the same system and experience, users can perceive different levels of realism, depending on several variables such as past life experiences and physiological differences. However, objective realism is how well the system (hardware and software) provides the same stimuli as the real world, whether users perceive it as real or not. There is a consensus that fidelity defines how a system can replicate the real-world experience. However, because reality is too complex, authors use a “referent” (an abstraction of reality) to make a comparison between the virtual and real experience possible. Slater’s definition of immersion is closely related to fidelity. However, we distinguish between both terms where immersion is the system potential to provide a realistic virtual experience (defined by the supported SCs), and fidelity is how well that system is used in this regard. For example, a highly immersive system could allow full-body tracking with extreme precision, but if the IVE application only makes use of head tracking with a bad software implementation, then such application will be of lower fidelity compared to an application where full-body tracking is used together with a highly optimized software implementation within the same immersive system. Following this definition, fidelity is therefore equal to objective realism.

Ferwerda’s physical realism is oriented to the visual system. Nevertheless, if we extend its definition to encompass the rest of the human senses—the virtual scene produces the same stimuli as the real counterpart scene—then it is similar to our definition of objective realism. Likewise, Perroud et al.’s dimensions of physiologic realism and realistic construction of the virtual world [88] also fit within our notion of objective realism. To objectively evalute an IVE realism, the scene must be possible to be replicated in the real world. For example, the realism of a partial fantasy world (an experience that can only be partially reproduced in the real world) could be objectively evaluated only in the parts shared between the real world (e.g., gravity or light behavior). The rest of the fantasy elements (e.g., humans being able to fly on their own) cannot be replicated in the real world and, therefore, have no realism is zero, even though they can still be coherent/credible within the internal logic of the experience and provide a sense of presence.

Gilbert’s [29] authenticity, Skarbez et al.’s [102, 103] coherence, Slater’s PsI [105], Lombard et al.’s [60] perceptual realism, and Perroud et al.’s psychological realism [88] are closely related to subjective realism—how well the virtual experience meets expectations of users of what they consider to be real. Subjective realism can therefore be high even if the IVE does not replicate the real world. This means that users can perceive a fantasy world to be realistic even if it depicts events that would be impossible to happen in real life, as long as they make sense within the logic of the scenario context. As per Gilbert’s definition, the exception being authenticity makes assumptions that the IVE is trying to replicate the real world.

In short, we divide realism into subjective and objective realism, which we will use throughout the rest of the article. Subjective realism considers how the users’ expectations are met and ultimately how they perceive the experience as real. Objective realism considers how close to reality the virtual experience was, discarding whether or not the users thought it was real or not. It is objectively defined by the system characteristics and how they are used.

We found different terms and definitions used to describe realism; thus, to avoid possible misinterpretations between different studies, we reviewed the terms and definitions from this systematic review of selected papers.

The eligibility criteria (Table 1) allow identifying related studies that can answer the RQ and discard studies that do not. Because the criteria are defined before the document search, the chance of bias is lowered [48]. The criteria were defined to restrict the number of documents to the most relevant ones to answer the RQ.

Four well-established databases were included in the search phase: IEEE Xplore, Elsevier Scopus, Clarivate World of Science, and ACM Digital Library. The research team considered that these four databases cover all the venues where relevant studies for the systematic review scope are published. The search was conducted through Titles/Abstracts/Keywords. The query was built with the research questions in mind. Because the survey focuses on immersive VR, AR, and MR, the search term “virtual reality,” “augmented reality,” and “mixed reality” had to be found together with one of the following terms: immersive, HMD, variations of head-mounted, CAVE, or headset. Related terms had to be included to narrow the obtained results of the search and orient it towards realism. After several iterations, the final version of the query was the following:

The search was conducted on April 6, 2020, and had no time range limitations. A total of 1,287 documents were added to the screening phase. In addition, a search in additional sources was done to verify if any more documents to be included were not picked up in the selected databases’ search. As a result, the search identified and added 13 more documents, resulting in a total of 1,300.

From the initial 1,300 documents, 490 were duplicates, resulting in 810 unique works for the screening phase. Papers with the same title would be further investigated as possible duplicates, and if the abstract was identical, it was considered a duplicate. In the case of similar abstracts, the full-text analysis would reveal if the studies were duplicates or not. The screening was performed individually by three researchers in an unblinded standard way. It consists of reading the abstracts to determine which ones are relevant and which ones are not. Two researchers read each abstract, and if there was a consensus between researchers, the document was accepted or rejected from further analysis. In case there was no consensus, a third researcher was assigned to decide by the majority. The screening phase excluded 425 documents (through exclusion criteria), resulting in 385 documents for full-text reading.

The resulting papers underwent a full-text assessment guided by the exclusion criteria to assess their eligibility. From the 385 documents left from the screening phase, 306 were further excluded through exclusion criteria. The papers that fulfilled the eligibility criteria were processed to extract all the data necessary to answer the RQs and conduct a quality assessment. Thus, from the initial 1,300 documents, 79 were deemed fit for data extraction for the scope of this systematic review.

After reading the full-text documents, data was gathered through predefined forms. The variables included were aimed at answering each research question (Tabel 2). Data records were extracted to answer RQ1, the author’s terms and definition (if available). For RQ2, the dependent and independent variables will be considered as well as the results and study highlights. The authors’ study limitations and future work will be considered to discuss this systematic review results better.

The document quality assessment allows for an objective evaluation of how reliable studies are and to what extent they can be trusted. During the full-text analysis of the documents, a quality assessment was conducted. The scoring system was similar to the approaches by Connolly [17], Feng [22], and Melo et al. [74]. Similarly to the study selection, and to avoid bias, the scoring was conducted by two researchers. If in consensus, then the rating would be closed; if not, then a third reviewer would moderate the scoring and provide closure. The scoring system consists of three items rated from 1 (lowest) to 3 (highest). The final score is constituted by the sum of the scores from the three items. The items cover the type of document, sample size, and how robust and relevant the methodology was for the study itself:

Type of document: conference papers get one point, papers published in the third quartile or inferior journals get two points, papers published in second- or first-quartile journals get three points (quartiles at publication time).

Sample size: sample sizes lower than 6 per group get one point, between 6 to 11 get two points, higher than 11 get three points (sample size recommendation based on Macefield [64], considering the lower end of the baseline range of problem discovery group size and higher end of the baseline range comparative study baseline range).

Methodology: evaluated through the analysis of the instruments, materials and procedures, limitations and possibility of biases, the extent to which the study is valid and replicable. Studies that present severe issues (e.g., issues than can compromise the entirety of the results without being presented as limitations or flaws in describing the methodology to the point that cannot be replicated) in this matter get one point. Studies that present issues that do not critically affect the trustworthiness of the results (e.g., flaws that can affect the results but are well documented as study limitations) get two points, studies that only present minor issues that do not compromise the results (all studies that did not fall into the previous categories) get three points.

The mean quality score from all 79 documents considered for full-text was 6.22 points, with a standard deviation of 1.54 (Figure 1(b). Documents with a score equal to or higher than seven are considered high quality. As such, 34 documents (43%) are considered high quality and provide an overall high-quality peer-review, sample count, and robust methodology. There is a limitation in the quality score field “document type,” because some conferences can have higher-quality documents than some journals. This limitation, however, is mitigated through the other two quality scores (methodology and sample). If the paper is of high quality, then it will still be considered high quality even if at a conference (1 point from conference plus 6 points from methodology and sample). Since several works are considered high quality (almost half), we conclude that the work seems to be well supported and in a mature phase. We also conclude that because of the quality research being done around realism in IVEs, both authors and journals/conferences seem to recognize this research field’s potential.

Due to the lack of existing taxonomies that can cover such a wide range of factors, it was decided to create one for this study. The taxonomy proposed allowed us to integrate every variable found throughout the studies (within the scope of the study) into categories. The categorization of the dependent variables (regarding the user experience) can be found in Table 3 and independent variables (from now on called Realism Factors, corresponding to objective realism) in Table 4. User Experience (Task Satisfaction) and User Performance (Effectiveness/Efficiency) factors are based on the satisfaction and effectiveness/efficiency metrics, respectively, from ISO/IEC 9126-4 recommended usability metrics [1].

Although we did our best to segment each independent and dependent variable in different Realism Factors and categories, in specific contexts, some variables can be ambiguous and may fit in more than one Realism Factor or category (in part the result of the absence of a proper taxonomy). In these situations, the researchers discuss where the variable would fit the best according to the study context. For example, changing the tangible device that the participant uses to interact in the IVE could both fit in IVE System (Interaction) as well as IVE Content (Haptic) or IVE Content (System). However, if the study is oriented to research interaction (e.g., studying different levels at which the user can provoke changes in the virtual environment) and not the haptic feedback effect (e.g., compare different haptic devices/modes while the extent to which the users can provoke changes in the virtual environment remains the same), then the study would be placed in the IVE System (Interaction). This way, we avoid increasing complexity by simultaneously having the same variable in different factors/categories. The Realism Factors are divided into two main categories:

Both IVE Content and IVE System have sub-categories that were created to accommodate the studies’ independent variables. This categorization enabled us to systematize the data better and find patterns in what was studied and yet have to be researched. During the analysis, the meaning of the variables and what they measured was taken into account when fitting them into a category.

Inside of each Realism Factor there are different levels of objective realism that are compared. Usually, authors state which level (or condition) is the most realistic. However, some authors do not give this information; in these situations, the research team considered the most realistic condition the one that better mimics (or has a better potential of mimicking) real-life situations. Based on this methodology, we report that a given factor’s Better/Worse level of realism had a Positive/ Negative/Mixed/Neutral impact in a given dependent category. Note that Positive/Negative does not necessarily refer to higher or lower scores, as there are variables where higher scores could be interpreted as worse (and vice versa). Note that an increase or decrease of scores in a given dependent variable does not necessarily mean that a given Realism Factor directly affected that same variable but rather indirectly by influencing an unconsidered variable, which by its turn influenced the one being measured. The results had to be significantly different for the impact to be considered Positive or Negative. For qualitative studies, the results were based on the authors’ conclusions.

Consider the following example: A study investigates how different levels of visual detail can influence the sense of presence, user memory recall, and cybersickness [36]. In this case, the visual detail is included in the Realism Factor of IVE Content (Visual - Environment), as it refers to a virtual environment’s visual content. The sense of presence is included within the User Experience (Presence) category. The memory recall is considered a metric of User Performance (Effectiveness/Efficiency), and therefore it is included in this dependent category, while cybersickness is included in Physiological Responses. In the case that presence scores increases in the higher visual detail condition while memory recall and cybersickness are not influenced, we report it as follows: Better IVE Content (Visual - Environment) resulted in a Positive impact in User Experience (Presence) and a Neutral impact in User Performance (Effectiveness/Efficiency) and Physiological Responses.

There are also situations where authors do not isolate all variables, resulting in conditions that differ in more than one factor (e.g., Condition A has better lighting and sounds, whereas Condition B has worse lighting and no sounds). Therefore the independent variable cannot be categorized solely in one Realism Factor. In these cases, we report the following: Better/Worse IVE System (Camera) + IVE Content (Audio) resulted in a positive/neutral/negative/mixed/indeterminate result.

Another situation that can occur is when a particular Realism Factor has no impact (Neutral) by itself on the experience, but the interaction with other Realism Factors has. In these situations, we report it as follows: Better/Worse IVE System (Camera) x IVE Content (Audio) results in a positive/ neutral/negative/mixed/indeterminate result. In this same situation, there is the possibility of impacting the experience when the level of objective realism of a giving factor is low and the level of objective realism of other factors is high. It would be erroneous to state that a better level of realism in both factors would result in a specific impact. In these cases, we report that IVE System (Camera) + IVE Content (Audio) results in a positive/neutral/negative/mixed/indeterminate result.

The Realism Factor’s impact within the same dependent category on user’s experience was also categorized into five possible outcomes:

There is a difficulty in synthesizing the types of impact on physiological responses. For example, an increase in heart rate might be neither positive nor negative, just a manifestation of the human body to the stimuli received in the IVE. However, in some contexts, we may attribute an increase of heart rate as positive or negative if the virtual experience depicted manages to replicate the same physiological reaction of a real analogous situation. In a hypothetical scenario where a user would experience a very stressful event (where a heart rate increase would be expected in a similar real-life situation), we consider it a positive impact if the heart rate increases. In conclusion, the type of impact registered at physiological responses will be highly context-dependent. Similarly to physiological responses, user behavior’s also presents a challenge to synthesize. Participants have different backgrounds and life experiences and personalities (between other variables) that can lead them to behave differently to the same stimuli. However, some behaviors (such as proxemics [50]) can still be studied. In the case of User Behavior, the type of impact will be considered positive if the user reacts as expected (i.e., similarly to a real-world condition) as per the authors’ conclusions. However, if the authors do not conclude whether user behavior was closer or not to reality, then our research team would analyze the context of the study and decide the impact type based on how we expected the user to behave under the same conditions in the real world.

The first RQ is centered on understanding which terms authors used (regarding realism and similar terms) and how they define them. Considering all documents that fit the criteria of the systematic review, there is a clear use of the terms realism (50 documents \(-\)53%) and fidelity (39 documents \(-\)42%) over credibility (2 documents \(-\)2%), coherence (1 document \(-\)1%), congruent (1%), and believable (1 document \(-\)1%).

Note that some authors used more than one term. The terms that the authors are defining are the ones considered for the analysis. If no definition is given, then all terms used are considered (e.g., a document may contain realism and fidelity, but no definitions, the document is considered for both realism and fidelity count). Considering only documents that contain definitions (34 documents), we have: 18 documents (53%) defining fidelity, 13 documents (38%) defining realism, 2 documents (6%) defining credibility, and 1 document (3%) defining coherence. All these data and definitions can be visualized in the supplementary material.

Almost all authors presented their specific views on the subject, resulting in different approaches. Some authors only defined one dimension of the term they used, others divided it into several dimensions, and some considered only one dimension of the term (e.g., visual realism [109] or visual fidelity [31]) and further divided it into sub-dimensions.

Regarding authors who use the term realism, one took on a more objective approach [54]—“Realism is the Immersion, which is the ‘the objective level of sensory fidelity produced by a IVE System”’; another a more subjective approach [18]: “closeness between the viewer’s perception of the virtual environment and an identical real one”; and others considered both aspects [88] when dividing it into sub-dimensions. Here, we can verify the importance of dividing realism in objective and subjective to avoid misinterpretations of the same across multiple studies. The objective approach by Laha et al. [54] and Perroud et al. [88], “Realistic looking” and “Realistic construction of the virtual world” and “Physiologic realism” constructs, are closely related to our notion of objective realism. However, we do distinguish in which immersion is not realism but rather the capabilities of the system within which a realistic virtual experience could take place. Conti et al. [18] and Perroud et al. [88] “Psychological realism” approaches are similar to our notion of subjective realism. However, on the contrary to Conti et al. [18], we do not assume if the virtual experience tries to replicate the real world, as users might consider a virtual experience realistic even if it depicts events impossible to happen in the real world as long as they are coherent [102, 103] within the rules of the environment.

Regarding fidelity, the majority of authors define it objectively to how well a virtual experience replicates a real analogous one [5, 19, 24, 31, 36, 38, 58, 72, 89, 101, 131], while one author takes on a more subjective approach [68] by implying perceptual fidelity as “not necessarily equivalent to physical simulation.” We can conclude that fidelity is in fact the same as our objective realism definition, independent of users, except to Mania et al. [68] “perceptual fidelity,” which fits the subjective realism where users might consider a virtual experience real even if it does not properly simulate the real world.

Overall, there seems to be a mixed-use of the terms realism and fidelity, mainly the fact that some authors refer to realism as either objective or subjective or the same as other terms (which could also be subjective or objective for other authors), whereas some further divide realism in subjective and objective dimensions. For example, Slater et al. [109] divided visual realism into two components, geometric realism and illumination realism, where the latter one is defined as “the fidelity of the lighting model.” Rogers et al. [95] also seem to consider fidelity and realism as synonyms “fidelity (also naturalism, or realism)” using them interchangeably by defining display fidelity as “sensory realism, referring mainly to auditory and visual qualities” and interaction fidelity as “action realism, i.e., the degree of exactness with which user actions in VR resemble real-world actions in terms of biomechanical similarity, input, and control.”

Very few authors used and defined credibility. Boucaud et al. [10] defined it as, “We understand credibility here as the degree to which the participant feels the agent behaved itself in an adequate human-like way”, and another used a dictionary definition [30]: “defined by Merriam-Webster is the quality of inspiring belief.” Both definitions are equivalent to coherence [102], which are, by their turn, similar to our notion of subjective realism.

Only one author considered coherence defining it as [102]— “the set of objectively reasonable circumstances that can be demonstrated by the scenario without introducing objectively unreasonable circumstances”—while also stating it is different from fidelity—“Coherence as a construct is related to, but distinguishable from, fidelity,” which was already discussed in the introduction of this article.

We must note that we did not find works that based their definitions on terminology. Most of the authors included and defined certain dimensions/components that were of interest in the context of their studies. Therefore, they do not show us the whole picture. Although this systematic review only considered IVEs under the constraints of RQ2 criteria, each author seems to provide us different definitions, where although some similarities, may confuse readers. To add to this possible confusion, some authors do not provide definitions. These situations could be compared to the confounding use of immersion [124] and presence [106], where authors were using one’s definition instead of the other, which could originate confusion, mainly when no definitions are given.

Only 27 documents (34%) presented the authors’ definition of the term used. This suggests that the authors considered the definition as general culture. In these cases, the term’s definition should be considered as the one present in the dictionary, which sometimes might not be the definition authors considered for their paper.

Even though several terms with different definitions were used, when considering the ones that address the subjective side, they all come together regarding “how the IVE meet the users’ expectations of what is real within the internal rules of the virtual scene.” Different people have different realities due to cultural background, physiological, and/or other differences. For example, a person unfamiliar with airplanes might find a hypothetical flight simulator as real. However, an aircraft pilot may find the same flight simulator as unreal. This would be due to differences in both subjects’ life experiences. There are also mental illnesses with psychoses that can shift the perception of reality. There are also situations where unrealistic experiences (such as the Stanley Milgram experiment [77] replicated in VR [107]) might still be considered real despite being known that they are not. This is known as PsI [105], which could also be considered the same as reality judgment [103]), coherence, and credibility.

Then, we have the objective side of the definitions, which commonly means “how a system was able to replicate the real-world conditions,” independent of the users’ perception. Due to these circumstances, a unified definition of realism is important.

After listing every variable from data extraction, the research team found patterns in both independent and dependent variables, allowing us to categorize them into several groups. Results from three studies could not be categorized (check supplementary material) and were excluded from RQ2 statistics, resulting in 76 documents being considered. A detailed description of the impact of each Realism Factor per document can be found in the supplementary materials. Overall, the impact of realism was positive (Figure 2): 64% positive, 27% neutral, 3% negative, 4% mixed results, and 2% were indeterminate.

This systematic review aimed at investigating, in an exploratory way, the terms and definitions being used in the literature to define realism in IVEs (RQ1) and the impact of objective realism in the user’s virtual experiences (RQ2).

RQ1 results led us to conclude that there are two mainly used terms: realism and fidelity. Overall, we verified several authors confound use of realism-related terms, while some did not even provide a definition. Therefore, a table was created describing each term and its definition (if present) (see supplementary material). We have proposed segmentation of realism into subjective and objective realism for the scope of this study. Objective realism defines how well a virtual experience replicates the real world, determined by what the immersive system allows and how well we take advantage of it. Subjective realism relates to how users perceived the virtual experience as being real, whether the virtual experience depicts a scenario possible to happen in the real world or not.

RQ2 results indicated that objective realism had an overall positive impact and very low negative and mixed impacts. Realism had the highest positive impact on embodiment and less on physiological responses. Only three dependent categories were negatively affected by objective realism (although with very low occurrences count): physiological responses, user performance, and perceived environment realism. The three less researched dependent categories (with equal or less than 10 occurrences) were embodiment, virtual agents, and user behavior. We suggest further research on these less-evaluated factors to increase the validity of the results further.

Regarding the Realism Factors, the least independently studied ones (with equal or less than 10 occurrences) were IVE System (Audio), IVE System (Physics), and IVE Content (Scent). Therefore, we suggest further studies to verify the impact of such Realism Factors in IVEs, especially the IVE Content (Scent), where no occurrences were found. A table was created denoting all the results found for each document (see supplementary material).

Methodology or/and equipment limitations were found to be contributors to the negative impacts registered. Examples of the first one were methodologies that facilitated the users’ performance by simplifying the scene with less-objective realism or methodologies that promoted a higher risk of simulator sickness. Hardware limitations contributed to an increase in motion sickness symptoms. We consider the following points as the main takeaways from this study: