3.3.1 Search engines

We conducted the sample search within three different scientific databases: Ebscohost (EBSCO), Elsevier's Scopus and Web of Science maintained by Clarivate Analytics. These three were chosen based on evaluations in academic articles (Adriaanse and Rensleigh, 2013), search modification and recording possibilities of each search engine.¹ The revision of search engines was vital, since they limit and direct possible search results and thereby the conclusions of the whole review. For instance, Google Scholar is a good source for grey literature, such as conference papers, academic books, and other publications, adding depth to reviews (Haddaway et al., 2015; Paez, 2017). We excluded grey literature from this review, so Google Scholar was not used.

3.3.2 Search criteria

The review considered solely peer-reviewed journal articles in English. This was the first step for verifying the quality of included articles (Dawidowicz, 2010). Search criteria were modified multiple times, as multiple details affected the search results. Essentially, the three search engines were chosen to achieve as comprehensive search results as possible. The fluidity of XR as a concept resulted to tens of thousands of articles, which needed more targeting. The search criteria were modified to emphasize practically oriented research on actual use of extended reality in working life. Conclusively, three different keyword groups were formed.

The first keyword set (virtual reality OR augmented reality OR extended reality OR mixed reality) covered the technology itself including all XR variations from virtual, augmented and mixed to extended reality. Search with abbreviations, such as “MR”, were tested but it did not significantly change the search results, acquired with full words. In some cases, abbreviations added irrelevant articles into search results, as especially “AR” can be simply a part of a word. Moreover, we maintained the word “reality” with its counterpart extended, virtual, augmented, or mixed, although there are multiple variations. For instance, Mikropoulos and Natsis (2011) use the term virtual environment in their review, but contemplated, in fact, on same issues when virtual reality medium is in question. Cave automatic environments (CAVE) were not included in the keyword set as we presumed research articles within our scope would use some form of XR in addition to CAVE in their topics, keywords, abstract or title.

The second keyword set (participat* OR collaborati* OR coordinati* OR co-operati*) was formed in order to concentrate on the actual use of extended reality in working life settings requiring and/or transforming collaboration and cooperation. We were not interested in how XR transforms individual working tasks, but how it potentially transforms overall collaboration between people and other technological and material objects. Since our aim was to collect research on actual working life collaboration, we took special interest in social and educational research and looked for how technology shapes and changes how the actual work is organized and conducted (Orlikowski, 2007). Therefore, we searched for articles where social sciences were contributing to XR research in different research fields or as a part of multidisciplinary research. Therefore, these keywords (participat* OR collaborati* OR coordinati* OR co-operati*) representing quite wide scope of interest were addressed to this category to obtain as wide range of articles as possible.

The third keyword set (work* OR organization* OR organisation* OR occupation*) aimed to limit the search results specifically to working life context and to organizational transformations. This was one of the most challenging part of the sample search process. The research represented by selected articles did not need to be specified as work or organizational research as long as the context for XR application was working life. Conclusively, this keyword set included somewhat different terms (work, organization, occupation).

4.1.1 Application areas of XR

The analysed articles as well as the application areas are specified in Table 1. We identified three main application areas of XR in the articles: design (13), remote collaboration (6) and training (5). In addition to the three themes, category ‘other' includes two cases: Haavik (2016) inquired AR tools in surgeons' work and Barreau et al. (2015) covered a VR experience of an 18^th century merchant ship built for historical analysis. We excluded medical area from our search, as uses of XR in medical field constitute a specific, own field of research with its specific questions and emphases (see e.g. Davids et al. (2020)). Haavik (2016) is an exception which was included in this review since it met the search criteria and focuses specifically on work practices of surgeons rather than medical uses of XR.

Under the category of design, urban planning was well represented covering five out of thirteen articles (Bratteteig and Wagner, 2012; Dembski et al., 2020; Sanchez-Sepulveda et al., 2019; Wagner, 2012; Wang and Chen, 2009). Two articles (Bratteteig and Wagner ,2012; Wagner ,2012) analysed the same three case studies utilizing MR set-up called MRTent for participatory design of different urban areas in different countries. In addition to five urban planning articles, six articles were in automation, engineering, and construction (AEC) sector. For instance, XR enabled collaborative design of a hospital building (Maftei and Harty, 2015) and worked as an enhanced communication tool between designers and customers overall in AEC (Shen et al., 2013). Hallbeck et al. (2010) inquired the use of XR in an efficient workplace design in five different industrial settings. Aldabra et al. (2020) used VR in design workshops for refugee camps.

XR was applied for remote collaboration in six cases. Remote collaboration cases were in remote work meetings (Da Silva and Garcia, 2013; Hayashida ,2018; Oprean et al., 2018), design-related tasks (Narasimha et al., 2019) and in security services, such as by military personnel or security guards (Lukosch et al., 2015; Nilsson et al., 2011). The studies of Da Silva and Garcia (2013), Hayashida (2018), and Oprean et al. (2018) concentrated especially on the spatial experience that can be shared and provided remotely. The studies support the idea that shared spatial experience strengthen the feeling of presence and thus may support the more general feeling of shared social experience.

The five articles considering training included both individual training scenarios (Anastassova and Burkhardt, 2009; Nykänen et al., 2020) and team training (Bertram et al., 2015; Nilsson et al., 2011; Passos et al., 2017). The sub-area of team training was partly overlapping with remote collaboration, as team members often were in remote locations. Bertram et al. (2015) noted that while knowledge transfer was measurably the same with training in physical and virtual environments, trainees were more motivated in physical environments. The research of Nykänen et al. (2020) found the perceived self-safety motivation higher in VR-based training versus lecture-based option. While this would suggest the realistic physical scenario to be most efficient and VR-based training better than mere lectures, the differing research settings and manners of measurement do not justify strict comparison.

Save View full size Expand inline Collapse inline

Table 1. Analysed articles and application areas of XR.

As noted in the description of data collection an analysis, we needed to stretch our original requirements for long-term real-life uses of XR. Although technological development has been significant and possibilities of XR keep widening, the practical uses still seem rare based on our results.

4.1.2 Potential added value of extended reality in working life

We found two ways in which XR technologies are at the moment contributing into working life in practical settings compared to potential technological promises of them. First, extended reality can potentially enable scenarios otherwise inaccessible or unreachable. The reason for inaccessibility may exist due to costs or danger of effort (Bertram et al., 2015). Safety was especially emphasized in the training of security personnel (Lukosch et al., 2015; Passos et al., 2017), construction workers (Nykänen et al., 2020), and underground coalminers (Grabowski and Jankowski, 2015). As another example of increased safety, Haavik (2016) inquired unspecified AR tools use in surgical operations to access to internal body parts with minimum disruption. In addition, accessing previously inaccessible could mean accessing space-located knowledge in real time (Anastassova and Burkhardt, 2009). For instance, to Barreau et al. (2015), the technical realization of historical and naval information on virtual sailing enabled historical analysis from multiple aspects, i.e. general living conditions and social actions.

Second, the articles stressed the potential of XR to improve already existing practices. For instance, it may offer additional possibilities “as a tool for advice and support in stressful situations” (Lukosch et al., 2015, p.636). In addition, XR can offer more adept visual content for the design process (Bratteteig and Wagner, 2012; Gül, 2018; Maftei and Harty, 2015; Shen et al., 2013) by enabling rapid modification of a design model and multiple revisions. This may ease already existing practices or, by becoming more cost-efficient, enable it in some cases, where the budget might not have previously allowed it. For instance, in the article of Bertram et al. (2015) an adaptive desktop VR application called ViPol was used for team training of cooperation between firefighters and police officers in dangerous situations. Normally, the costs of using a helicopter rescue services in training situations would be too high.

Training scenarios such as practicing the removal of undetonated explosives (Grabowski and Jankowski, 2015), or evaluating socio-cultural appropriateness of designed refugee camps (Albadra et al., 2020) or harshness of living in a 18^th century ship (Barreau et al., 2015) illustrate how XR technology has the potential of overcoming multiple previously unsurpassable obstacles set by time and space, safety, and resources. As a distinct feature from other digital technologies, the possibilities of XR bring forth novel spatial experiences. In an especially detailed manner, the article of Barreau et al. (2015) brings up how analysis of historical living conditions is enabled in a manner otherwise impossible, as the virtual reality modelled the movement of ship, cohabitants, and weather conditions. The literal ability to set yourself in this environment not allowing to forget the conditions in their entirety may pose one of the main strengths of XR: a wholesome shareable experience.

4.1.3 Limitations of XR in actual working life collaboration

Limitations of XR were primarily related to technical usability issues, undeveloped practices of technology applications, and lack of resources. Many of these limitations were the result of the novelty of XR. We had difficulties finding cases of long-term use, the novelty may have an accentuated position as an explanation for limitations. However, the virtual model of a historical ship (Barreau et al., 2015) was an exception, as it was utilized in multiple occasions over the years. In addition, some novel models at the time of the article writing might have been applied later on in long-term use, such as virtual safety training developed for multiple construction businesses in Finland (Nykänen et al., 2020).

Limited accuracy (Hallbeck et al., 2010; Sanchez-Sepulveda et al., 2019), eyesight of XR headsets and relatively heavy and bothersome equipment to carry (Lukosch et al., 2015) counted as technical usability issues, that technical development may ease and remove. There is no patent solution for movement in virtual worlds but case by case technical limitations can be bypassed. Alongside a keyboard and a mouse, handheld devices were the most common way of moving in a fully virtual environment, although it was not specified in all of the articles. Shen et al. (2013) underlined that integration of human movement in VR could equally increase the understanding of the design in question. Passos et al. (2017, p. 327) noted that security training in fully virtual training environment would have been more efficient, if the environment would have “the possibility to have a variable number of agents” in it imitating unpredictable human actions, i.e. virtual world would include more interactive elements. To Sanchez-Sepulveda et al. (2019) crucial point regarding interaction was “to improve the interaction with space and objects, the quality of graphics and the reality of the objects” (pp. 379-380) although participants generally had good experiences with movement in the virtual world.

Technical challenges and undeveloped practices of XR were somewhat liminal. Haavik (2016, p.175) brought up the “Janus face of AR”, referring to AR's possibility to enhance but equally to mislead surgical operations from a desired outcome: they noted that doctors and other staff in operation room seemed more concerned about the representations of AR display rather than patients themselves. With their analysis based on the concept of “sensework”, Haavik calls for adequate design and practices for distribution of work. Similarly, Barreau et al. (2015) noted that when it comes to scientific accuracy, scholars such as themselves must be careful with the conclusions they draw from the cases of utilization of XR models.

Lack of resources, such as time and money, hinder both long-term applications and longitudinal research of XR. Although XR may enable multiple scenarios otherwise impossible to achieve, the development of those specific scenarios with XR was an ongoing process in many cases. Occasionally, the emphasis on experimentalism may have led to a hasty application of XR solutions without adequate understanding of situational affordances and hindrances of the applied technology. However, in the article of Barreau et al. (2015), resources for realizing a VR model seemed exceptionally large and resulted in a mouldable, flexible virtual world experience accessible through multiple devices and utilized in various occasions. This meant that they created a full “VR experience” (Sherman and Craig, 2019, pp.736), which in return could and did provide valuable and immersive experience with historical information for historians and other users of the model.

4.2.1 Types of employed extended reality mediums

From the 26 articles, 13 handled VR, 7 AR, 4 MR and one both VR and AR. If MR is considered part of AR or vice versa, 13 of the articles handled VR and 11 AR, and one both. AR was more common in training, as it allows interactions with the physical world and is less restrictive regarding traditional communication means. MR was applied in urban planning (Wang and Dunston, 2011), a specific urban planning set up called MRTent (Bratteteig and Wagner, 2012; Wagner, 2012) and regarding Ergomix, a simulation tool used to design an efficient and ergonomic workstation (Hallbeck et al., 2010). However, VR was the common form of XR in the articles qualified for this review.

4.2.2 Immersion and extended reality displays

In the analysis, we make a following division between immersive and non-immersive XR in the real-life settings. Whether accessible via a headset or a desktop computer, if a whole virtual world is available (Sherman and Craig, 2019), XR is considered as immersive. If content covers only narrow, separate objects without offering full spatial experience, even if mixed with the physical world and accessed through a headset, the form of XR is non-immersive. With these prerequisites, the overwhelming majority of included articles utilized immersive virtual worlds rather than mere virtual objects. 16 articles represented immersive XR technology and 10 non-immersive. However, the line between a virtual object and a virtual world should be clarified. Some articles represented either vast environments (e.g. (Dembski et al., 2020; Oprean et al., 2011; Passos et al., 2017)) or a singular restricted area such as a building (Maftei and Harty, 2015), a shelter (Albadra et al., 2020) or a boat (Barreau et al., 2015). These were counted as immersive, since they offer a full spatial experience, opposed to a narrower object, such as a human body in the work of surgeons (Haavik, 2016) or a virtual model of a foot for shoemakers (Mandolini et al., 2017).

Headsets or head-mounted displays (HMDs) were in use in fifteen cases, whereas in six cases virtual objects and worlds were accessible only through computer screens. Wang and Chen (2009) also used data gloves besides of a computing unit and vision-based tracker. Occasionally, more traditional desktop-based means were the most useful and purposeful, such as in the article of Mandolini et al. (2017). The case focused on 3D models of feet for shoes' customization process: seemingly, a large field of view would not bring any additional value. In one case of AR-based training, concrete utilized AR devices were not specified (Anastassova and Burkhardt, 2009). Maftei and Harty (2015) inquired a design case, where design models could be perceived through entire rooms devoted to virtual models, i.e. CAVEs. Another practical XR set-up was MRTent (Bratteteig and Wagner, 2012; Wagner, 2012) located usually nearby a planned site, and the set up consisted of a physical table with superimposed virtual objects as well as multiple screens representing different things on the planned urban site.

Barreau et al. (2015) represented an exceptionally flexible XR case. They described how an immersive sailing experience in a historical ship was constructed, and the result was accessible through multiple devices and displays, such as desktop computers, headsets, mobile phones or as a CAVE environment. Immersion in relation to XR technologies is not only difficult to define as a concept. In addition, ways of realization of immersion in practice are multiple. CAVE is immersive in the sense it gives an idea of whole virtual environment, which can be shared with other users, and without losing the possibility to see one's own body and other possible participants. This is an area of future development of XR, as many other XR solutions do not allow direct and smooth communication between participants.

4.2.3 Interactivity

There are many ways of conceptualizing interactivity. Instead of synthetizing these disparate conceptualizations, we will make the differences visible. Some of the articles referred to interactivity as a feature of technology, i.e. real-time interaction with virtual objects, others saw it as action between human participants whether with or without the aid of extended reality medium. In some cases, interactivity was not regarded at all, as XR was treated as a mere medium to access material resources, such as digitally projected information without additional need to modify or ‘interact’ with them (Anastassova and Burkhardt, 2009).

When regarded as a feature of technology, that is, interaction between the virtual world and the user, interaction or interactivity was used to refer to a capability to customize the content of XR in real time (Barreau et al., 2015; Mandolini et al., 2017). The required level of customization varied. Oprean et al. (2018, p. 424) stated that interactivity of a VR system directly affects the degree of control which each remote collaborator feels while navigating in a virtual meeting space. They ended up operationalizing interactivity as navigability. Regarding AR and following Azuma (1997), Gül (2018) included to AR “a combination of the real and the virtual by providing 3D real time interactivity” (pp. 109-110).

Focusing on technology, two articles addressed XR applications as interactive systems (Sanchez-Sepulveda et al., 2019; Wang and Dunston, 2011). The concept is similar, although not entirely parallel, to a virtual world (Sherman and Craig, 2019), as it emphasizes interactions between the user and the virtual objects. Sanchez-Sepulveda et al. (2019, p.375) noted that interactive systems, as opposed to static virtual environments, “generate less stress”. This was measured quantitatively among other variables without more particular explanations.

If interaction was emphasized between the users, usually the virtual environment itself was somewhat interactive as well or mixture of real and virtual objects (Bratteteig and Wagner, 2012; Lukosch et al., 2015; Passos et al., 2017). Shen et al. (2013) utilized interaction as a concept of human-human communication and human-machine-communication. Wagner (2012) applied entirely different perspective on interaction, as they present “embodied interaction” in the context of MR application MRTent in urban planning. The concept of embodied interaction has been previously applied especially in the field of computer-supported cooperative work (CSCW), but Wagner specifically applies the term in the extended reality context analysing it with the aid of spatiality, representation, and haptic engagement. In the context of MRTent, Wagner (2012) noted that “interactions are difficult to study in a complex setting such as the MR-Tent, as they are enmeshed with all the other design issues we are investigating” (p.38).

4.3.1 Research areas of extended reality

We were interested in the concepts and methods used in XR research regarding actual collaborative working life settings. In the analysed articles, we identified four research areas regarding XR: collaboration, work practices, and evaluation of knowledge transfer.

The articles focusing on collaboration (Anastassova and Burkhardt, 2009; Bratteteig and Wagner, 2012; Da Silva and Garcia, 2013; Gül, 2018; Hayashida, 2018; Lukosch et al., 2015; Nilsson et al., 2011; Oprean et al., 2018) had research emphasis specifically focused on the dynamics of collaboration. For instance, Lukosch et al. (2015) studied situational awareness and team cognition in security training and Sanchez-Sepulveda et al. (2019) conducted research on collaboration in urban planning to increase public's motivation.

Four articles focused on the transformation of practices around XR (Bratteteig and Wagner, 2012; Haavik, 2016; Maftei and Harty, 2015; Wagner, 2012). They surveyed the social and material role of XR in different working life settings (design of a new building, surgical work, and urban planning).

Three articles inquired the efficiency of knowledge transfer via XR (Barreau et al., 2015; Bertram et al., 2015; Grabowski and Jankowski, 2015). Barreau et al. (2015) also described the construction of utilized VR model of historical ship, and the latter two were studies in training context.

These research areas somewhat follow the application areas of XR (design, remote collaboration, and training), which might point to the direction of XR specific research from the viewpoint of social sciences. There were varieties in the emphasis of the studies. 19 articles inquired situations where collaboration was local in one physical space and 7 specified on situations were an extended reality application enabled remote collaboration across different physical locations.

4.3.2 Theoretical and conceptual frameworks to study XR

Authors of the analysed articles applied a variety of theoretical approaches and concepts. Generally, theories and concepts on collaboration and cognition were common, such as “community of practice” applied by Anastassova and Burkhardt (2009) or “situational awareness” by Oprean et al. (2018). When the research focus was on the system development, theories on collaborative systems were often applied (e.g. (Du et al., 2018; Mandolini et al., 2017)).

Schön (1983)'s conceptualizations of reflective practice were applied in multiple studies (Bratteteig and Wagner, 2012; Gül, 2018; Lukosch et al., 2015, Maftei and Harty, 2015). Wagner (2012) did not apply Schön in individually written article although it was utilized in the shared work of Bratteteig and Wagner (2012). Schön was also applied in the articles inquiring collaboration (Lukosch et al., 2015) and work practices (Maftei and Harty, 2015). In their investigation of design practices in construction of a new hospital building, Maftei and Harty (2015) concluded in line with Schön's theory that cave automatic virtual environment (CAVE) worked not only as a mediating knowledge but “bounded up in performing design work configured as social, collaborative and context dependent process” (pp. 72-73).

In addition to Schön, Gül (2018) employed theories on design process and gestures, and especially Gibson (1977)'s theory on affordances. Oprean et al. (2018) utilized it too, but only from secondary sources, whereas Wagner (2012) applied Gibson to form a conceptual framework on embodied interaction and multimodality of XR.

Haavik (2016) utilized their own concept of “sensework” as separated from craftsmanship, describing it as “a label for a type of sociotechnical work in safety-critical operations where groups of professionals try to put together pieces of digital sensor data and different sorts of representations to create a coherent picture that gives meaning to familiar and unfamiliar situations” (p.177). Building, among others, on Latour (2005)’s work, they raised two themes over others: “(1) the role of representations in surgical work and (2) the structure of work and the division of labour” (Haavik, 2016, p.185).

Sensework is not to be mixed with Hayashida’s 2018 “sensecomputing”, which in turn investigated how information and communication technology can respond to people’s need for relatedness. Hayashida (2018) from Fujitsu Laboratories relied on behavioural economics and equilibrium theory of intimacy while trying to find solutions to fulfil people’s needs for relatedness in remote work. Extended reality might be a solution for increasing people’s need for relatedness, but as the entire concept and its key features are still maturing, we might ask, following Haavik (2016), what do we actually want or need the technological solutions for.

4.3.3 Methodological frameworks and methods

Case studies represented majority of the research. All research was conducted at least partially as a case study (Yin, 2018), even though not all articles explicitly stated case study as their method. Mainly this was due to our article selection criteria. Presumably, it is difficult to conduct practically oriented research with only interview forms or surveys, although surveys were frequently used to collect additional data (Du et al., 2018; Lukosch et al., 2015; Nykänen et al., 2020; Oprean et al., 2016; Shen et al., 2013). Occasionally, the article included multiple clearly defined cases: Hallbeck et al. (2010) represented five cases, where the same XR tool Ergomix was applied, whereas Bratteteig and Wagner (2012) and Wagner 2012) utilized and inquired MRTent in three different occasions of urban planning. Aldabra et al. (2020) conducted a participatory design research with two forms of workshops, of which one form (“adapt-a-design”) utilised VR.

In some articles, an explicitly stated theory guided methodological choices. For instance, this was the case in Maftei and Harty’s (2015, p.60) article, where Schön’s theory of reflective practice directed the choice of analysing design as practice. In other cases, the connection between methodology and theory was not so specific, such as in Gül (2018)’s article on affordances of AR, where Gül used protocol analysis method (PA) with a theory informed coding scheme.

Other articles addressed the developing methodology for XR research. In addition to their main research aim, four articles from all 23 proposed a new design framework (Shen et al., 2013; Wang and Dunston, 2011; Wang and Chen, 2009) for evaluating the adequacy of XR case by case in different design scenarios or in remote collaboration of security services (Nilsson et al., 2011).

There were few comparative studies with a small sample size, such as a study with three groups of eight people (Bertram et al., 2015) or 21 people (Grabowski and Jankowski, 2015) utilizing quantitative methods. These studies noted the small size of the sample but stated an overall positive result on the use of XR based on quantitative analysis. Bertram et al. (2015), Da Silva and Garcia (2013), and Narasimha et al. (2019) used experimental research design and had control groups for testing different variables, e.g. fastness in a card-sorting task (Narasimha et al., 2019).

Audio and video recordings were the most common way of collecting research material. Whereas interviews were recorded with audio only, video recordings were used for example in Anastassova and Burkhardt's (2009) two-phased study. They first formed a description of automotive technicians training as network of actors based on the concept of community of practice (CoP). In the second study, they observed seven training sessions and gained further insights in CoP “in real training settings” (Anastassova and Burkhardt 2009, p. 717). Video recordings were flexible way to record changing situations in certain space and time: teamwork (Bertram et al., 2015), collaborative urban planning (Bratteteig and Wagner, 2012; Wagner, 2012) or design process of a hospital (Maftei and Harty, 2015). In one case (Haavik, 2016, p.181), the notes were hand-written in an observation room of surgical operations since it was regarded as the “most appropriate”.

As a XR specific data, Dembski et al. (2020) recorded the participants movement in the virtual environment and used it as a research data comparing it to the movement potential of the actual real-life street. This, in its part, helps to estimate effects of new investments in public spaces. Movement recordings were still only a part of the research of Dembski et al. (2020), where a digital twin was used in citizen participation. The research case represented a collaborative design effort of multitude of participants, and equally, virtual reality can be regarded as a part of larger organization of street, building and environmental data collected together.

Although the articles focused on application of technologies, they rarely utilized research methods in natural settings such as ethnography. Bratteteig and Wagner (2012) and Wagner (2012) were the only ones referring explicitly to ethnography regarding their own research. They described their study as an intervention (conducted by the researchers) on three urban planning projects and collected research material through an ethnographic fieldwork. Nevertheless, some ethnographic features existed in the articles in a form of field observations and notes (e.g. (Haavik, 2016)). In addition, Maftei and Harty (2015) briefly referred to ethnomethodology but emphasized simple observation over active participation.