Secure and Trustworthy Artificial Intelligence-extended Reality (AI-XR) for Metaverses

NLP consists of different techniques that are used for automatically analyzing and understanding human languages (i.e., text and speech). There are many NLP applications that will be part of AI-XR metaverse applications, e.g., speech-to-text, text-to-speech, chatbots, and text/speech-based emotion recognition are the most prominent features of the metaverse. In particular, NLP techniques will be used for the recognition and understanding of complicated human conversations and commands. A key driving force behind the success of NLP methods is the advancement in ML/DL, with the development of new techniques such as recurrent neural networks (RNN), long-short term memory (LSTM), and transformer networks [103].

Machine vision or computer vision will be a fundamental component of AI-XR metaverse applications. Different computer vision applications will enable various functionalities in metaverse applications, for example, processing visual data from different sensors to infer high-level visual semantics. The major tasks of the visual processing pipeline include understanding users’ activities, emotion recognition, object detection, scene understanding, semantic segmentation, avatar generation, and so on. In addition to these applications, the metaverse is expected to have AI-empowered quality assessment capabilities, e.g., for satisfying the users’ demands about viewing high-resolution videos [68]. In this regard, advanced AI methods can be used to develop quantitative and qualitative benchmarks for visual quality assessment.

Metaverse is expected to simultaneously entertain a massive number of users with the metaverse services provisioned mainly through wireless networks. Over the past few years, substantial research attention has been devoted to improving the overall throughput and performance of wireless network communication, and the use of different AI techniques is the main driving force behind this innovation [32]. Metaverse will mainly include real-time multimedia services that require a reliable connection, high throughput, and low latency to ensure a seamless user experience. Therefore, it is expected that the metaverse will benefit from 5G and beyond empowered communication. The potential of different AI techniques has already been demonstrated for 5G and 6G, e.g., intelligent resource allocation [129], solving resource slicing problem [12], channel state estimation [91], network modulation [140], and so on.

Service providers in the metaverse will provide users with different incentives in terms of digital assists (e.g., coins) for different events, games, and creative activities. The dispersion of such assets requires a transparent way to record and track such transactions. In this regard, smart contracts–empowered blockchain technology can be leveraged that allows critical information to be stored on an immutable and impenetrable ledger. The decentralized nature of blockchain imbues it with great potential to address security and privacy issues in the metaverse [27]. This potential increases with AI-empowered blockchain applications [167], for example, different AI-based clustering and classification techniques can be used for data analysis stored on blockchain [137]. In addition, different AI techniques can be used for knowledge discovery and learning, efficient data management, perception, reasoning, and planning.

The term digital twin refers to the digital replica (i.e., representation) of real objects. A digital twin is capable of synchronizing regular actions, operations processes, and assets with the real world, e.g., analyzing, monitoring, predicting, and visualizing [138]. The digital twin also acts as a bridge where the actual world and digital world interact with each other through different IoT devices [31]. The digital twin will be one of the most important building sectors of the metaverse that allows users to access and use services in the virtual world while exactly depicting the real world in a virtual environment. For example, surgeons and medical experts can create a digital replica of a patient to study and understand the involved complexities before performing his surgery.

Ensuring the privacy of the end-users will be a major challenge in AI-XR metaverse applications. As these applications are designed to monitor and collect users’ data at an unprecedented fine-grained level, in a bid to create a replica of the digital world [45], there is a greater danger and risk of privacy breaches [150]. For example, to create an immersive virtual scene in the metaverse, data from different sensors will be collected and analyzed using AI models, e.g., facial expressions, brain wave patterns, hand movements, eye movements, biometric, and speech data [150]. This raises obvious concerns regarding the privacy of users and opens a new horizon for digital crimes [45]. Users’ sensitive information including daily routine activities, personal logs, and schedules will be stored on a server, which ultimately becomes a critical privacy challenge in a publicly distributed network. Such data include body movements, voice, reflexes, and even more critical data that include subconscious and unconscious responses such as eye movements and physiological signals. Features such as eye tracking are readily accessible using commercially available products such as the HTC Vive Pro Eye and Pico Neo VR headsets even though the XR expert Louis Rosenberg recommends banning such features and data collection in non-health-related applications for ethical reasons. In general, the data is collected through on-device sensors at the user site, processed at their devices or nearby local server, and logged as storage in the cloud. Considering the above-mentioned procedure, some malicious attacks can be encountered, which are classified as (i) data collection, (ii) data storage, (iii) data usage, and (iv) user profiling. Furthermore, as AR technology depends on the precise localization of users in the physical world, modern smartphones use Lidar sensors and Simultaneous Localization and Mapping (SLAM) algorithms to precisely locate the user in the real world. This opens up the possibility of privacy violations, as sensitive information may be exploited for anti-social purposes. Some important concerns related to the privacy of data are described next.

Data Collection. Generally, the data in AI-XR metaverse applications is collected through users’ input either by voice command or textual input and multi-modal sensors including camera, microphone, textual commands, gesture sensors, and wearable devices. These devices are expected to frequently collect personal information such as daily routine activities, voice and biometric data, and personal preferences including shopping items, TV shows, and preferred food choices. In addition, the private data will be used for the creation of avatars for a digital representation of a real human in the metaverse, which also raises privacy issues. For example, the built-in location sensor in the Oculus headset can be used for tracking users’ presence in the real environment with a precise accuracy [150].

Data Storage. The ultimate aim of developing personalized AI-XR metaverse applications is to aid human beings and ease their daily routine activities. These applications will contain multiple sensing devices that generate a substantial amount of data. Whereas these devices will be resource-constrained with small storage units, which leads to the tradeoff between data generation and storage at the edge level. To address these shortcomings, these devices upload their data along with the corresponding logs to online local or global data centers. Though, the data storage tradeoff is resolved by connecting with the servers. However, it also raises privacy concerns regarding the access permissions and data protection of the consumers. Also, if the communication channel is hacked by an attacker, then the data can be manipulated to get the intended outcomes. The literature suggests that adversaries can extract information regarding the actual data even if the communication is encrypted and can track the location of the users by realizing different attacks such as advanced inference attacks [152] and differential attacks [153].

Data Usage and Consent. Continuous data collection through multi-modal sensors including cameras, microphones, and other sensors will be closely involved in the daily routine activities of metaverse users. These sensors collect continuous data regardless of privacy awareness, which is logged over the local or global server(s). Consequently, this procedure raises legal questions regarding the users’ consent and the kind of data that is collected and shared. Moreover, metaverse service providers can also utilize this data to optimize their inference models and to make them robust and more personalized. However, it also raises privacy concerns such as the collection, disclosure, and sharing of the data without the explicit or implicit consent of users.

User Profiling. Similar to the current social media (in which users are considered as the product), everything will be a product alike in the metaverse. Metaverse will act as a meta-platform for different entities (such as users, developers, content creators, businesses, etc.), and this raises questions about data collection and its utilization for user profiling [40]. Also, the provisioning of the metaverse services requires that the users should be uniquely identifiable in the metaverse. For this purpose, VR headsets/glasses or any other such device can likely be used for illegally tracking users in real life [127]. Moreover, such devices can be attacked by malicious actors and can be exploited to track users’ real locations for possible digital and real-world crimes. Guzman et al. [38] presented a data-centric perspective to avoid unprecedented privacy challenges related to data collection and its usage in MR.

According to the definition of Trust, expressed by Lee and See [80], in the perspective of automated systems, “Trust is an attitude that an agent will help achieve an individual’s goals in a situation characterized by uncertainty and vulnerability.” Metaverse is a data-driven technology in which service providers will use different automated tools to assist human beings as a recommendation engine in various domains, e.g., shopping recommendations, movie recommendations, and even recommendations regarding their health. The efficacy of such recommendation systems is highly dependent on the collection of personalized data for intelligent decision-making, e.g., an AI model is trained using the collected fine-grained health data to suggest more accurate recommendations regarding the well-being of users. Despite the huge potential of such features of AI-XR metaverse applications, it also raises many questions about which parameters have influenced the underlying AI model in producing a particular decision and the trustworthiness of such predictions and decisions. Next, we discuss such challenges across three dimensions: (i) Truthful AI, (ii) Transparency, and (iii) Explainable and Interpretable AI.

Need for Truthful AI. The evolution in technology also brings threats among them. Over the past few years, AI-based personal assistants including Siri, Alexa, Astro, and the like, have managed to gain wide social acceptance, and these devices are being used by consumers daily for automating household routine tasks. Currently, misinformation, falsehood, or malfunction in AI-based text or speech analyses and command execution are not considered a matter of concern. However, it is expected that AI-enabled intelligent systems with linguistic capabilities will be a major feature of AI-XR metaverse applications. In this regard, it will be quite challenging to enforce the criterion of truthfulness in AI-based systems to ensure the safe selection of statements and behavior according to the social norms of users while interacting with human society.

Transparency Issues. In general practices, developing AI-based systems is about training sophisticated algorithms on large-scale data to learn an efficient and generalizable model that can be deployed in the real-world environment. However, it is a well-known fact that the performance of these models is directly proportional to the transparency of training data, i.e., AI models will perform as well as their training data. Numerous factors such as input data riddled with poor cleansing, the selection of inherently biased data, underfitting, and overfitting influence the performance of these models and can result in fairness and accountability issues (discussed later in this section). Unlike typical application development, there are no quality assurance tools available to spot bugs and evaluate the bias factor in the training data. For instance, if it was known at which stage the model is going to infer at a perfect scale, then there would not be a need to perform training on such large-scale data. This process is all about the hit-and-trial procedure, which is a quite challenging task to identify the right approximations with better data, hyperparameters, and configuration settings.

Explainable and Interpretable AI. The rapid adoption of AI-based applications in human society has also grown the complexity of the systems, which ultimately requires system understandability to make them legitimate and trustworthy. In a critical human-facing technology such as the AI-XR metaverse, interpretable and explainable AI models are required to answer questions about accountability and transparency of their decisions and outcomes: for example, how the employed model reached the decision and which factors influenced the models to make that decision [8]. Such questions are particularly important for human-centric applications where the potential impact of AI will be limited if it is not able to provide accurate and transparent AI predictions. Therefore, the key objective of these models is to develop a relationship of trust between human users and AI. Also, in Reference [88], explainability is described as an inherently human-centric property. However, one of the main challenges in developing explainable methods is the tradeoff between achieving the modesty of an algorithm and ensuring the discretion of sensitive user data. In addition, it is a challenging task to identify the right information while generating a simple yet useful explanation for users. It is worth noting that the terms interpretability and explainability are closely related and are often used interchangeably in machine learning literature, however, these terms are different in practice. Interpretability of the models is defined as the extent to which its outcomes are predictable, i.e., for a given change in the input or model parameter(s), the interpretability enables us to predict the respective change in its output. On the counter side, explainability deals with the explanation of internal processes of the ML/DL models in a human-understandable way.

In the literature, various studies have been presented that aim to understand and identify the requirements for XAI from a human perspective. For instance, Liao et al. [87] developed an algorithm-informed questionnaire to be asked by various AI developers during semi-structured interviews to identify the gap between the current XAI work and practices to develop user-centered XAI systems. The study concluded that the current research in XAI lacks an understanding of real-world users’ needs and requirements for enhancing transparency in AI. Furthermore, the authors suggested a way forward to reduce this gap that includes addressing users’ needs and reducing the gap between algorithmic output and developing human-understandable explanations. Similarly, the authors in Reference [88], presented an overview of existing XAI literature from a human-centered perspective. Specifically, they considered the following questions: What is the role of human-computer interaction (HCI)techniques in shaping XAI research? Furthermore, they highlighted three roles that the HCI technique plays in navigating, assessing, and expanding XAI research that includes: (1) Understanding all AI stakeholders’ needs for diverse explainability; (2) Identifying limitations by considering the gap between the explainability and actionable understanding; and (3) Performing theoretical analysis of human explanations, behavioral, and cognitive processes to develop novel computational frameworks to foster enhanced compatibility between humans and XAI. On a similar note, T. Miller emphasized the need to leverage and incorporate insights from various social sciences including psychology, philosophy, and cognitive science to define explanations for XAI [95]. Considering these fields, the author also provided a detailed overview of existing literature related to human explanation along with providing insights about how these existing methods can be incorporated into XAI systems.

Despite the state-of-the-art performance of modern AI techniques including DL-based systems, it has been shown that these models are highly vulnerable to carefully crafted adversarial perturbations known as adversarial ML attacks [136]. The threat of these attacks has been already demonstrated for many critical applications such as healthcare, autonomous vehicles, and so on. On a similar note, AI-XR metaverse applications are essentially critical, as they involve humans, and ensuring their safety from any harm is profoundly important. On the counter side, the existence of these challenges raises many concerns about the safety, security, and robustness of AI-based metaverse applications, thus hindering their practical deployment. As it is equally important that any AI-based system should be equally trusted by all stakeholders involved including service providers, developers, and end users. These challenges are detailed later (Section 4).

Modern AI methods, like advanced DL models, lack fairness and accountability in their decisions [111]. However, such questions are particularly important for critical applications like AI-XR, in which the model’s decisions can have life-threatening consequences for the end-users. Moreover, AI models are developed using training data, which will be mainly collected from human users in AI-XR metaverse applications for providing immersive experiences. Humans possess certain biases that will be readily reflected in the data they generate, and when this data is used for training AI models, the data bias will be directly translated into the developed AI-based system. As a result, the model will be biased towards certain samples that contain certain features (bias), and its decision will not be fair. On a similar note, the critical nature of AI-XR metaverse applications demands accountable decisions. Consequently, data bias if remained unaddressed can ultimately lead to unintended consequences [79].

Users’/avatars’ identities in the metaverse can be stolen or impersonated illegally, leading to authentication and access control issues in the interconnected virtual worlds. Identity theft in the metaverse will be more dangerous than traditional attacks. The identity of a user once stolen will reveal everything about that person’s digital assets, avatars, and social relationships. The attackers can exploit different vulnerable VR gadgets and other service authentication loopholes to realize identity theft attacks and can steal the victim’s secret keys of digital assets and bank details. It has been reported that about 17 users in the OpenSea NFT marketplace were hacked through a phishing attack and flaws in the smart contract that resulted in a loss of $1.7 million.¹

Metaverse will leverage different biometrics and password-driven technologies to authenticate users and their avatars in the virtual worlds. The attacker can evade such authentication systems to impersonate real users’ identities to get control of the whole virtual world. Evading AI-based biometric systems has become easier with the advancements in adversarial ML research. Therefore, AI-empowered speech-and-face-recognition-based biometric systems can be easily attacked to realize impersonation attacks. Once the attacker has access to the metaverse, it can exploit the data generated by the victim’s devices to deceive him, committing a crime in the virtual space. However, the exposure of biological data when used for authentication purposes can also lead to severe consequences [77]. Moreover, the authentication of social friends of a user using their avatars is much more challenging in the metaverse as compared to real-world identity authentication. In this regard, facial data, voice, and videos can be used to develop an AI-based avatar authentication system, however, the unsolved inherent issues of AI can still hinder its practicality.

Bias refers to a model making certain unethical assumptions about the data. Human bias along with its many aspects has been studied by researchers in many disciplines, including law, psychology, and so forth. In Reference [99], bias is defined as “the prejudice or inclination of a decision made by an AI system which is in a way considered to be unfair for or against one person or group.” Bias in recommendation systems, advertising algorithms, facial recognition systems, and risk assessment tools has been widely studied in recent years. In metaverse applications, data will be collected from a heterogeneous group of people and sources having their own characteristics, stereotypes, and behaviors, which introduces different biases in the collected data. In References [93, 102, 135], the authors discuss different kinds of bias based on the sources and the types of bias. On the base of sources, these biases have been divided into further categories: biases caused by data, biases caused by algorithms, and biases caused by user interaction.

There are various concerns regarding the mental and physical safety of metaverse users. There are several reported incidents of digital harassment, theft, and bullying in XR applications [92]. The report on “Immersive and Addictive Technologies” highlights rampant incidents of sexual harassment, cyberbullying, and grooming online [100]. Ensuring the safety of users is a major challenge for AI-XR metaverse applications because of the fact that such incidents have real damage and harm to users despite being experienced in the virtual world. The avatars generated using recent advancements in AI techniques, in particular, generative models, can appear more realistic in AI-XR metaverse applications and can engage users in promotional conversation, thus providing a false sense of a real human behind the avatar. The avatars in such a promotional content are fueled with more personalized data (such as your vitals, emotions, expressions, etc.) to look more realistic. Also, these sales avatars can pitch products to you more persuasively than any real salesman or even a recommendation system due to their access to rich cyber-physical data about you. The research in deep fake technology and photorealistic avatars is already at the stage where computer-generated content is indistinguishable from the real. Such advancements can be leveraged to realize an adversarial attack on AI-XR metaverse applications to get the intended behavior and outcomes.

There are various opinions regarding the antisocial aspects of AI-XR metaverse applications. Many people think that introducing AI-XR metaverse may detract the users (humans) from their purposes and may have a somatic effect. In the literature, it has been shown that extended times online can result in users demonstrating post-VR sadness and detachment from reality. For instance, Aldous Huxley in 1932 wrote in his social science dystopian fiction novel that using technology can lead to self-inflicted harm that can influence people to be diverted from their higher priorities and become more prone to being influenced by other interests. As a result of such a quest for technological utopia, the human psyche and society as a whole are greatly afflicted. Social critics have long argued that various digital media, such as television, the Internet, and the Web, make people docile and less connected to the real world. For example, Jerry Mander in 1978 wrote in his book Four Arguments for the Elimination of Television that TV removes the sense of reality from people, promotes capitalism, can be used as a scapegoat, and all these three factors work together negatively. The modern technological disruptions including the web and social networking services have created a filter bubble detaching people from the real world and the truth. Due to these reasons, the current era is also referred to as a post-truth era [51]. We may reasonably expect that alienation from the real world will exacerbate with the increasing adoption of VR, AR, and AI-XR metaverse applications, which aim at changing the human perspective of the world. This argument can be supported by the fact that, in 2018, the World Health Organization formally included “gaming disorder” in its International Classification of Diseases following research that shows that technology can promote addictive behavior in people. Moreover, the literature focused on analyzing the social implications of metaverse argues to understand and identify potential psychological problems that can arise in metaverse [23].

Any technological intervention involving humans suffers from some serious ethical issues, especially the one that contains intelligence. The Institute of Electrical and Electronics Engineers (IEEE) has recently published a report on Ethically Aligned Design that mainly focuses on the Ethics of Autonomous and Intelligent Systems [126]. This report emphasizes the need of developing ethical autonomous and intelligent systems (A/IS) that promote human well-being and protect human rights through transparent and accountable A/IS and the prevention of the misuse of AI. This report is the collective effort of hundreds of researchers having diverse backgrounds and expertise in important areas such as governance, technology, civil society, and policy-making. This report has a dedicated section on XR and, interestingly, IEEE also has a Global Initiative on Ethics of XR.³ In this report, various ethical issues related to XR have been highlighted, including users’ preference for virtual life over the real world and complete disengagement with society. In addition, the reports conclude with the following remark regarding XR: “The nature of XR environments fosters unique legal and ethical challenges that can directly affect users’ privacy, identity, and rights. Society will need to rethink notions of privacy, accessibility, and governance across public and private spaces. New laws or regulations regarding data ownership, free use, universal access, and adaptive accessibility within XR environments may need to be developed.”

The big tech companies have resisted regulation, decrying the fact that regulation will slow down innovation. However, there are various ethics researchers and social scientists who are arguing for much greater regulation to ensure that consumer rights are protected. In this regard, the EU General Data Protection Regulation (GDPR) has paved the way for many countries and regions attempting to develop similar regulatory laws to protect Internet users. These regulations mainly emphasize the importance of the non-profiling of users (i.e., limiting the storage of tracking data) and for better transparency (i.e., online services and applications should specify why and what information is being stored). On a similar note, AI-XR metaverse applications are subject to the requirement of being transparent in terms of data collection and utilization and should also be subject to the informed consent of users. Also, the development of such applications requires thoughtful deliberation from a regulatory perspective. For instance, it is worth considering banning non-medical applications to collect vital biomedical statistics due to the high risks of being exploited maliciously. To mitigate the risk of users being manipulated deceptively, metaverse operators may be bounded to transparently declare the staging of virtual products and experiences in the metaverse. Rosenberg, one of the pioneers of VR and AR, has already started to argue about the need for regulation for metaverse applications [118]. For instance, he suggested leveraging the arguments regarding the regulation of social media for developing a legal and philosophical basis for metaverse regulation. As the metaverse can be deemed as an evolutionary expansion of similar services. Rosenberg argued that the only solution to eliminate ethical and privacy-related concerns associated with metaverse is to shift from an advertising-based to a subscription-based business model in which users pay a subscription fee for accessing the metaverse platform. This eliminates the service providers’ need to monitor their user base to a greater extent, however, this is not a feasible solution, as it is difficult to say whether or not users will pay for a safer metaverse.

Computer vision is one of the central building blocks in the foundation of the metaverse. In recent years, DL algorithms have enabled major advances in computer vision ranging from image classification [25, 39] to scene understanding [26, 36] and generating realistic images [158]. However, the discovery of the adversarial vulnerabilities of DL-based image processing models by Szegedy et al. [136] sparked a growing concern regarding the reliability and security of these deep models [7, 76, 143, 145]. Numerous works have analyzed these adversarial vulnerabilities in greater depth under different threat models [75]. In general, adversarial attacks work by optimizing the perturbation, \(\Delta x\) , to an input image, \(x\) , such that the output of the model, \(\mathcal {F}\) , is significantly changed, \(\text{maximize}|| \mathcal {F}(x) - \mathcal {F}(x+\Delta x) ||\) . \(\Delta x\) is typically optimized based on the gradients, which are either computed directly (white-box scenarios) or estimated by introducing random noise (black-box scenarios). A summary of various adversarial ML attacks on different computer vision applications can be seen in Table 2.

Similar to the vulnerabilities of ML models for vision applications, the literature demonstrates that the ML methods for modeling NLP tasks are also vulnerable to malicious attacks, at both the training and the inference stages of a typical ML pipeline [9, 11]. Below, we discuss such attacks.

Poisoning and Trojaning Attacks: Poisoning attacks and trojaning (also known as backdoor) attacks are the most widely known training stage attacks in NLP. Poisoning attacks aim to tamper with the training of the model so it is unable to perform satisfactorily on the test inputs [17, 96, 134]. Trojaning attacks aim to insert a trojan—typically characterized by a specific pattern of words known only to the attacker in the input sequence—into a model such that the model behaves normally on natural test inputs but malfunctions as desired by the attacker (through the use of the trojan pattern of words [57, 125]). Similar to the case with the computer vision applications, the trojaning attacks, being more difficult to be detected as compared to the poisoning attacks, pose a greater threat to NLP applications in the metaverse.

Adversarial Attacks. Although adversarial examples have been extensively studied in computer vision, they have received significantly limited attention in NLP tasks mainly due to the discrete input search space—minimal adversarial perturbations in the input are no longer feasible in NLP [8, 9]. Recently, however, there have been numerous works highlighting the adversarial vulnerabilities of the NLP-based ML models. Notable adversarial attacks include Text-bugger, Text-fooler, PWWS, and BERT Adversarial Example (BAE).

Adversarial attacks against NLP models generally follow three major steps—evaluation, perturbation, and selection—to achieve some adversarial goal—for example, targeted or untargeted misclassification—under a predefined threat model. Consider, for example, an input sequence \(X = \lbrace x_1, x_2,\ldots , x_i,\ldots ,x_n\rbrace\) correctly classified by an NLP model, \(\mathcal {F}\) , in class, \(\mathcal {F}(X)=y \in \mathbb {R}^M\) . At the evaluation stage, the attacker uses some impact scoring function to compute a set, \(I_x\) , representing the impact of each word over the output. At the perturbation stage, the attacker repeatedly perturbs the most impactful words in \(I_x\) using some pre-defined perturbation mechanism such that the semantic and contextual value of \(X\) remains preserved. At the selection stage, the most optimal perturbation is selected. Table 3 provides a summary of various adversarial ML attacks on different NLP applications.

AI-XR metaverse applications will provide ubiquitous connectivity to a massive number of users over wireless networks. Over the past few years, many AI-based algorithms have been developed to improve the performance of wireless communication and networking systems that will be used in different layers of network architecture [32]. The use of AI in wireless communication empowers wireless devices to perform many important intelligent functions such as network composition, analyzing traffic patterns, managing content requests, analyzing wireless channel dynamics, and so on. Moreover, AI-based algorithms have been used for optimizing different network constraints such as high throughput and low latency for different multimedia applications. A prominent use case is to leverage intelligent proactive load management in 5G and 6G communication networks and predictive data analytics to improve network operations. Despite the significant potential of using various AI algorithms for different optimizing applications in wireless networks, recent studies have highlighted that AI models are highly susceptible to adversarial ML attacks. For instance, Usama et al. [141] used a generative adversarial network (GAN) for realizing adversarial attacks on network intrusion detection. The threat of adversarial ML attacks on network traffic classification is demonstrated in Reference [144] and for cognitive self-driving networks is presented in References [143, 145]. Similarly, the threat of adversarial ML for 5G networks is analyzed in Reference [142]. A summary of various adversarial ML attacks on network applications is presented in Table 4. We refer interested readers to a detailed survey highlighting the threat of adversarial ML in network security [69].

In addition to the above-motioned adversarial vulnerabilities associated with the use of AI techniques in many network applications, some other critical network-related issues can hinder the smooth operation of the metaverse at a global level. For instance, centralized network architecture provides flexibility in terms of cost-saving, simplicity, and ease in performing different operations. However, such architectures are more prone to a single point of failure (SPoF) and distributed denial of service (DDoS) attacks [149]. For example, if a powerful attacker gets control of the network, then it may lead to severe challenges such as SPoF and DDoS. To address such issues, the literature suggests leveraging decentralized network architecture [97]. In addition, decentralization will potentially amplify the transparency and trust of users in exchanging their virtual belongings (such as digital assets and virtual currencies) among each other and across different virtual worlds in the metaverse. However, many issues arise with the use of decentralized approaches, e.g., reaching a consensus on an ambiguous operation among the huge number of entities in a dynamic metaverse. DDoS: Metaverse will include a massive number of IoT devices, which can be compromised by an attacker to form a botnet to realize DDoS attacks [15]. Sybil Attacks: In a Sybil attack, the adversary pretends to have fake (or manipulated) identities of legitimate users or devices. Using such stolen identities, he can take over the network.

Outsourcing the training of ML/DL models to third-party services that offer powerful computational resources on the cloud is prevalent nowadays. These services allow ML developers to upload their data and models for training over their cloud platforms. It is expected that such services will be featured in AI-XR metaverse applications, as they provide the flexibility of developing AI models using sufficiently large training datasets while reducing the cost and time. However, the literature demonstrates that such services are vulnerable to a variety of attacks, such as backdoor attacks [34], exploration attacks [124], model inversion [168] and model extraction attacks [74], and so on. More details about various attacks and defenses for cloud-hosted ML models can be found in Reference [109]. Visual illustration of adversarial ML attacks on different potential applications in the AI-XR metaverse is presented in Figure 8.

Cryptography refers to a practice of methodologies, aiming to construct and analyze communication protocols to ensure secure communication while achieving data integrity, authentication, non-repudiation, and data confidentiality. Generally, there are two common types of encryption methods: (i) symmetric encryption and (ii) asymmetric encryption method. The symmetric encryption method is a secret-key algorithm, in which the sender and receiver must share the same key to perform encryption and decryption of the data. Whereas, asymmetric encryption method (also known as public-key cryptography) uses two keys, i.e., public and private key, associated with an entity that requires to authenticate its identity electronically or encrypt data. The public key of each entity is published, whereas, the corresponding private key is always kept secret to perform encryption or decryption of data. In literature, Ron Rivest, Adi Shamir, and Leonard Adleman (RSA) [117], Data Encryption Standard (DES) [133], Advanced Encryption Standard (AES) [106], and Secure Hash Algorithm (SHA) [24] are a few commonly used algorithms used for data encryption. Different cryptographic techniques can be employed to convert readable information to an encrypted state, which can be later used at the receiver end after performing decryption. Below, we discuss some of the most commonly used encryption methods that can be used for the development of privacy-aware AI models.

Homomorphic Encryption. Homomorphic encryption (HE) is a computational approach that performs encryption while allowing computational tasks to be executed over encrypted data at the same time to ensure the privacy of the data. HE is defined as a public key cryptographic technique in which a pair of public and private keys is created to perform encryption and decryption operations on the data. The public key is used to encrypt the data before sharing it with the third party for further computational tasks including training and/or inference. Due to the homomorphic characteristics of this approach, the results can be decoded using the private key to visualize the results without showing them to third-party servers or unauthenticated users. In the ML literature, HE has been used for protecting the privacy of the users’ data for different applications such as genome imputation [123], misinformation detection in text messages [6], and so on. Specifically, the AI models are trained and inferred using encrypted training and testing data, thus preserving the privacy of the sensitive data.

Secure Multi-party Computation. Secure multi-party computation (also known as secure computation) is a type of cryptographic technique that is focused on the development of collaborative methods to perform joint computation and calculate a function over joint inputs while possessing those inputs in an isolated fashion. Contrary to the traditional cryptographic methods, where cryptography ensures confidentiality and integrity of communication or storage, while the adversary is outside the system of users, this approach protects the users’ privacy from each other while performing ML-based tasks, including training and inference activities.

Garbled Circuits. The idea of garbled circuits was first proposed by Yao in 1986 to perform two-party computation [169]. Garbled circuits can be used in a scenario where multiple parties are interested in performing some computation without sharing their data. Let us assume two parties (e.g., Alice and Bob, for the sake of simplicity) want to perform some computation using garbled circuits. Alice will send his input and function in the form of a garbled circuit, and Bob will utilize his garbled input with the garbled circuit to get the result of the required function, once he obtains it from Alice in an oblivious fashion. In Reference [21], garbled circuits along with HE have been used to develop privacy-aware ML models, where the authors trained three classification models, namely, decision tree, Naïve Bayes, and hyperplane decision classifier, using the encrypted data.

Secret Sharing. In secret sharing, multiple parties collaborate in the computation by sharing their secrets among them while holding a “share” of the individual secrets. The secret can only be reconstructed by combining all the individual shares kept by participating parties, otherwise, it will be useless. In the literature, the secret-sharing technique has been successfully used for training AI models in a privacy-preserving way. For instance, Bonawitz et al. [20] used the secret-sharing technique to train an ML model by aggregating model updates from multiple parties in a privacy-aware way. In a similar study [67], authors used this technique for the development of a privacy-aware ML-based emotion recognition system leveraging client-server architecture. In their proposed framework, the secret-sharing technique was used for the communication of audio-visual data from the client side to the server, where an ensemble model based on a sparse autoencoder and a CNN model was used for the feature extraction from the collected data. The SVM classifier was then trained using the extracted features for the emotion recognition task. A secret-sharing-based parallelized variant of principal component analysis (PCA) for preserving data privacy is presented in Reference [19].

Secure Processors. Secure processors were pioneered by rogue software to protect sensitive code from being accessed by malicious actors at higher privilege levels. Secure processors are being used in different processors now to perform privacy-preserving operations, e.g., the Intel SGX processor. In Reference [101], SGX processors were used to develop a data-oblivious system for different ML techniques that include SVM, decision tree, matrix factorization, and k-mean clustering. The primary goal was to facilitate collaboration between multiple data proprietors performing the ML task on an SGX-empowered data center.

The idea of differential privacy is based on introducing noise in the data to protect sensitive information while ensuring the usefulness of the data [43]. Differential privacy is defined in terms of the task-specific concept of neighbor datasets, and it provides strong guarantees in ensuring the privacy of the data during algorithmic analysis [1]. Numerous differential privacy-based methods have been presented in the literature, such as differentially private stochastic gradient descent (DP-SGD) [42], private-aggregation of teacher ensembles (PATE) [174], exponential noise-based differential privacy-preserving methods to ensure privacy on large-scale data. These methods demonstrated better applicability in ML-based applications in various domains, including intelligent transportation services, smart/virtual personal assistants, and smart healthcare services.

Federated learning (FL) refers to a distributed-ML paradigm that is capable of learning global ML models without directly accessing and/or exchanging data from edge devices. Intuitively, basic FL-based methods consist of a collaborative learning framework where each participant such as an edge device, network node, and local server can independently train a model using its local data. These edge devices then share their model parameters with a server, which then performs aggregation of the parameters after receiving parameter updates from each edge device. Finally, the server updates the parameters of the global model and shares the updated parameters with all participants. The iterative process of FL is continuous until the desired criteria as been fulfilled, e.g., validation accuracy/loss or the maximum number of communication rounds. In this way, a global model is trained without requiring the actual data from the FL participants. Subsequently, this sharing mechanism allows ML-based systems to learn from large-scale diverse data and develop a global model. Such methods can demonstrate better applicability in terms of dealing with sensitive data in various human-centered applications such as AI-XR metaverse applications. Despite the success of FL in training an ML model with reliable performance while maintaining the privacy of the actual data, different attacks can be realized on the model being trained using the FL paradigm, e.g., backdoor attacks [160], label flipping attacks [110], free-riding attacks [89], poisoning attacks [5], and so on. Also, it has been demonstrated that sensitive information can be extracted from the shared parameters in FL settings [18].

Data modification methods modify the input data during the training or inference phase to mitigate the effects of adversarial noise. A few such famous methods are briefly described below.

Model modification methods aim at modifying the parameters of trained ML models to defuse the effect of adversarial attacks. Commonly used model modification methods are described below.

Methods aiming to robustify ML models in this category use an additional model either to detect adversarial examples or to clean adversarial perturbations. A few methods are described below.

An AI model is referred to as explainable if it can explain the ability of parameters to justify the results. Explainability aims at making an AI model understandable to humans. In recent years, substantial research efforts have been conducted to enhance explainability, trustworthiness, and interpretability in AI models. Fairness, Accountability, and Transparency in Machine Learning (FAT-ML) [47] and Defense Advanced Research Projects Agency (DARPA), explainable AI program [59] are the two famous research groups working in this context. The literature argues that explainable models can be the first step toward converting black-box AI models into white-box models [2]. We refer interested readers to a comprehensive survey focused on explaining black-box models for a more detailed discussion of related terminologies and for a detailed classification of various explainable AI methods for a given type of black box system [58]. On the contrary, in Reference [120], a counter argument is provided where the author emphasized the need to develop inherently interpretable models instead of developing methods to explain black-box models (involving critical decisions).

Interpretable models refer to the models that explain themselves. In simple words, an AI model is said to be interpretable if its decision against some input is logically understandable such as which factors influenced the AI model to reach that decision. In the literature, various methods have been presented to leverage interpretability in ML models. These methods ensure that the predictions of interpretable models are unbiased, which ultimately makes it easier to trust these systems in human society. It is worth noting that the terms interpretable and explainable are interchangeably used in the literature, however, they are different in terms of domain-specific definitions; moreover, there is no exact definition of these terms [114]. In summary, explainable AI is more focused on explaining and communicating AI’s output to humans, while interpretable AI is concerned with the understanding of the internal processing of AI models. A detailed taxonomy of different explainable and interpretable AI methods can be found in References [2, 114].

The relevant literature emphasizes two famous sets of principles that can be used to attain trustworthy AI. One of them is developed by European Commission’s AI High-Level Expert Group (HLEG) [66] and the other one is defined by Organisation for Economic Co-operation and Development (OECD) [170]. The following are the seven essential principles outlined in OECD: (1) Human agency and oversight; (2) Technical robustness and safety; (3) Privacy and data governance; (4) Transparency; (5) Diversity, non-discrimination, and fairness; (6) Environmental and societal well-being; and (7) Accountability.

Similarly, the following principles are outlined in HLEG: (1) Inclusive growth, sustainable development, and well-being; (2) Human-centered values and fairness; (3) Transparency and explainability; (4) Robustness, security, and safety; and (5) Accountability. One of the key noticeable insights from the above two principles set is that they mainly emphasized the explainability, security, fairness, safety, and robustness aspects of AI. Therefore, these are the essential requirements that need to be fulfilled to develop trustworthy AI-based applications. In addition, we can see that these principles are essentially human-centric and respect ethical norms. As potential AI-XR metaverse applications will be more human-focused, therefore, the above-mentioned principles can be leveraged to develop trustworthy AI-based applications for the metaverse.

The metaverse’s inherent complexity raises different security issues. For instance, it can be envisioned that the metaverse administrators will have to push automation, that is, to handle more tasks with algorithms, rather than with human operators, due to the requirement of managing a large number of users, applications, and services. The generated data will be much larger than those managed by the current Web platforms. Delegating tasks to algorithms, especially those implemented with state-of-the-art AI approaches, is necessary to meet high-level efficiency and scalability. However, in the current social media and the Internet, we have even started to realize the implications of using algorithms for managing societally relevant tasks. Despite the significant performance, these algorithms suffer from various issues. Some authors writing on the governance of metaverse have proposed the use of a modular approach for the development of metaverse applications, as it allows adapting regulations to specific scenarios and then controlling the system accordingly [49].

To ensure socially desirable AI decisions, novel ways are required to be figured out to simultaneously minimize potential harms associated with the use of AI and its potential benefits. In this regard, the importance of taking an ethics-first approach towards the development of AI-based technologies becomes more plausible [52]. However, there are many challenges associated with the development of ethical AI pipelines due to distinct social norms and demographics of the human population, i.e., one ethical solution may be beneficial for a group of people but it is highly possible that it will not be suitable for another group at the same time. Therefore, customized solutions are required to address such issues that can consider the social norms of target users while making AI-based decisions. In this regard, different ethical guidelines can be leveraged that can be potentially used for the development of pro-social AI solutions. The literature shows a groundswell of interest in ensuring ethical and responsible AI [107].