QoS-Aware Graph Contrastive Learning for Web Service Recommendation
^††thanks: Duksan Ryu is the corresponding author. This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF- 2022R1I1A3069233) and the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2023-2020-0-01795) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation) and the Nuclear Safety Research Program through the Korea Foundation Of Nuclear Safety (KoFONS) using the financial resource granted by the Nuclear Safety and Security Commission (NSSC) of the Republic of Korea. (No. 2105030)

QoS properties such as response time and throughput have different values depending on the user [3]. When considering a specific service-based application, for example a web service providing video, user-dependent QoS data is mainly determined by the user’s calling environment [12]. Therefore, it is generally accepted that if two users have similar past QoS data, they are likely to experience similar QoS in the future due to similar calling environments. From this perspective, a collaborative filtering approach that essentially works by modeling the similarity between users and services [13, 14] becomes suitable for QoS prediction[12, 3]. So far, QoS prediction models based on collaborative filtering have made great progress in providing efficient solutions for many service-based applications, such as cloud computing-based applications [15] and multimedia service-based applications [16].

In the realm of QoS-based web service recommendation, various studies have explored collaborative filtering techniques to improve recommendation performance [8, 17, 12, 18, 19]. Additionally, frameworks utilizing random walks on user-item bipartite graphs have been proposed to predict Web QoS values [4, 5, 6].

However, despite these efforts, data sparsity and the cold-start problem remain significant challenges in web service recommendation [6, 20]. Data sparsity refers to the scarcity of user-service interaction data, where only a fraction of possible user-service pairs are observed. This sparsity limits the effectiveness of collaborative filtering approaches that heavily rely on historical interactions to make recommendations.

The cold-start problem arises when there is insufficient information about new users or services, making it challenging to provide accurate recommendations. In the context of web service recommendation, this problem can occur when new services are introduced to the system, or when new users join and have limited interaction history.

While existing research has made strides in addressing these challenges, they still pose limitations in terms of handling data sparsity and the cold-start problem effectively. More advanced techniques are required to overcome these obstacles and improve the accuracy and coverage of web service recommendations. Therefore, this study proposes a graph contrastive learning framework for QoS-based web service recommendation. Until recently, QoS values were predicted using graph structures, but there is no example of QoS-based web service recommendation models that apply both graph convolution networks and contrastive learning [21, 22].

Let $\mathbf{R}\in\{0,1\}^{|\mathcal{U}|\times|\mathcal{V}|}$, where $\mathcal{U}$ is a set of users and $\mathcal{V}$ is a set of services, be an interaction matrix. $\mathbf{R}_{u,v}$ is 1 iff an interaction $(u,v)$ is observed in data, or otherwise 0. Let $\mathbf{A}\in\mathbb{R}^{N\times N}$ be the adjacency matrix, where $N=|\mathcal{U}|+|\mathcal{V}|$ is the number of nodes. The normalized adjacency matrix is defined as $\tilde{\mathbf{A}}:=\mathbf{D}^{-\frac{1}{2}}\mathbf{A}\mathbf{D}^{-\frac{1}{2}}$, where $\mathbf{D}\in\mathbb{R}^{N\times N}$ is the diagonal degree matrix.

The user-service interactions in QoS datasets are closely related to the field of graph-based CF in recommendation systems. We introduce several graph-based CFs, which are mainly used in the general domain of recommender systems suitable for QoS-aware service recommendations. The goal of graph-based CF is to predict user-item (e.g., user-service) ratings by leveraging the relationships and interactions captured in an interaction graph. This approach aims to learn embeddings for users and items in the graph and utilize their inner product to compute the predicted rating.

The user-item relationships can be represented by a bipartite graph and thus, it recently became popular to adopt Graph Convolutional Networks (GCNs) for CF [23, 24, 25, 26, 27, 28, 29, 30, 31, 32]. GCN is a type of neural network that can operate on graphs. GCN’s node embedding is updated by neighbor nodes, and GCN can access $l$-hop neighbor nodes.

NGCF [24] grafted the GCN into CF as it is. LightGCN [25] has emerged as a standard model by introducing a more lightweight model than NGCF. Its linear GCN layer definition is as follows:

where $\mathbf{E}^{(0)}\in\mathbb{R}^{N\times D}$ is the learnable initial embedding matrix with $D$ dimensions of embedding, and $\mathbf{E}^{(l)}$ denotes the embedding matrix at $l$-th layer. In the message passing perspective, the GCN layer can be rewritten as follows:

where $\mathbf{e}^{(l)}$ is the feature vector of node $i$ at layer $l$, $\mathcal{N}_{i}$ is the set of neighbors of node $i$, $d_{i}$ and $d_{j}$ are the degrees of nodes $i$ and $j$, respectively. We continue to use matrix-expressed formula of Eq. (1) later.

The predicted rating, denoted as $\hat{r}$, is calculated by taking the inner product between the user embedding, $\mathbf{e}_{u}$, and the item embedding, $\mathbf{e}_{i}$:

In the context of QoS-based web service recommendation, graph contrastive learning is employed to leverage the geolocation context and augment the graph. We introduce several graph contrastive learning methods for recommendation systems.

Graph-based CFs have demonstrated impressive performance but face challenges such as data sparsity and cold-start problems, as they heavily rely on positive user-item interactions as labels [33, 34]. To overcome these challenges, contrastive learning (CL) for CF methods have been proposed to extract valuable information from unlabeled interactions [35, 36]. These methods utilize different views and contrast them to align node representations, have shown promising results.

SGL [36] utilizes LightGCN as its backbone encoder and employs three operators (node dropouts, edge dropouts, and random walks) to generate augmented views. SimGCL [37] simplifies the graph augmentation process by introducing random noises to perturb node representations. LightGCL [38] proposes a graph augmentation strategy based on singular value decomposition to capture global collaborative signals effectively.

These methods consider the views of the same node as positive pairs and views of different nodes as negative pairs. The positive pairs are defined as $\{(\mathbf{e}_{u}^{\prime},\mathbf{e}_{u}^{\prime\prime})|u\in\mathcal{U}\}$, and the negative pairs are defined as $\{(\mathbf{e}_{u}^{\prime},\mathbf{e}_{v}^{\prime\prime})|u,v\in\mathcal{U},u\neq v\}$. The supervision of positive pairs promotes the similarity between different views of the same user, while the negative pairs encourage the distinction between different nodes. It adopts InfoNCE [39], which allows it to learn better user/item representations to preserve node-specific properties and improve generalization ability. The contrastive loss InfoNCE is as follows:

where $s(\cdot)$ is the cosine similarity function between two vectors, and $\tau$ is the temperature of softmax, which is a hyper-parameter. It is also applied to items in the same way, which is $\mathcal{L}^{item}_{CL}$. These two loss functions are combined as $\mathcal{L}_{CL}=\mathcal{L}^{user}_{CL}+\mathcal{L}^{item}_{CL}$ and $\mathcal{L}_{CL}$ is the objective function for contrastive learning.

To incorporate geolocation context into the recommendation process, the graph is augmented for contrastive learning techniques. This approach allows for aligning node representations based on different views, leveraging geolocation information alongside other contextual features. By contrasting the augmented views, the model can capture fine-grained similarities and differences between services, enhancing the accuracy and robustness of the recommendation system. Our proposed method utilzes contrastive learning and geolocation-based graph augmentation offers a valuable framework for improving QoS-based web service recommendations.

Web QoS-based service recommendation heavily relies on understanding the interactions between users and services. GCN excels in capturing the dependencies and relationships within the graph structure, allowing it to model the complex nature of user-service interactions. By leveraging neighborhood information, GCN can effectively propagate and aggregate QoS-related signals across the graph.

In the context of CF, the basic assumption is that similar users would have similar preferences on items (services). GCNs can capture high-order connectivity patterns in the user-item interaction graph and propagate information from neighbors to learn embeddings of users and items. This is especially advantageous in CF since GCNs can capture the similarity between users by propagating information from similar users and learning their embeddings. Thus, incorporating GCNs into CFs offers a natural way to encode collaborative signals in the graph structure of the interaction network.

The integration of GCNs with CF enhances the ability to capture collaborative signals, as it considers not only the user-item interactions but also the inherent structure and connectivity patterns within the graph. This approach effectively leverages high-order connectivity on the graph to better understand user preferences and item characteristics. By incorporating this information, the recommendation system can improve the quality of web service recommendations and deliver more relevant suggestions to users.

Contextual information is crucial for QoS-based service recommendation due to the following reasons:

• •

  Geolocation captures location-specific preferences: Users’ service preferences and requirements may vary based on their locations. Incorporating geolocation information enables the recommendation system to consider regional preferences and recommend services that are relevant to users’ current or intended locations.

• •

  Environmental factors impact service performance: Geolocation information can reflect environmental factors that may affect service performance. For instance, network conditions, availability of resources, and infrastructure quality can vary across different locations. By considering geolocation, the recommendation system can account for these factors and recommend services that are suitable for specific environments.

To leverage geolocation information effectively, an augmented view can be constructed by incorporating it into the user-service interaction graph. This augmented view integrates geolocation attributes into the graph representation, creating a richer context-aware representation of the user-service interactions.

Contrastive learning techniques can be applied to the augmented view for QoS-based service recommendation. Contrastive learning aims to learn discriminative representations by contrasting positive (similar) and negative (dissimilar) instances. By using the augmented view, the recommendation system can learn contextual embeddings that capture the geolocation-related patterns and preferences in the user-service interaction graph. Contrastive learning enables the system to effectively model the relationships between services and users in the context of geolocation, enhancing the accuracy of recommendations.

Fig. 2 shows our web service recommendation framework named QoS-aware graph contrastive learning (QAGCL). Our overall framework is as follows:

• •

  First, we preprocess the user and service invoke data to make a graph structure. We create interactions according to the Web QoS values of users and services. This graph is the original graph used for the recommendation task.

• •

  Next, we construct other graph views based on the original graph. We create new graphs based on distance using geolocation information such as latitude and longitude of users and services. We also randomly drop the edges of the graph to create another view of the graph.

• •

  Finally, the initial user and service embeddings enter into different GCNs for the three augmented graphs. Each embedding passed through the final layer has a representation of a different view. One of the embeddings from the other view performs the recommendation task, and the other two are used for contrastive learning.

We describe the two graph augmentation operators in the following subsubsections.

Mining inherent patterns in graphs is helpful for representation learning. To do this, we use GCNs to capture better user-service interaction, devising GCNs for Haversine distance-based edge drop and random edge drop to create different views for nodes. These GCNs can be represented as follows:

where $\mathbf{E}^{(l)}=[\mathbf{E}_{u}^{(l)},\mathbf{E}_{s}^{(l)}]$ is the input feature matrix at layer $l$ and $\tilde{\mathbf{A}}$ is calculated from original bipartite graph $\mathbf{A}$. $\tilde{\mathbf{A}}_{\text{HD}}$ and $\tilde{\mathbf{A}}_{\text{ED}}$ are calculated by augmented operators $g_{\text{HD}}(\cdot)$ and $g_{\text{ED}}(\cdot)$, respectively. Three different views are computed from each of the three different graphs.

The prediction layer is the layer to construct the final embedding after propagating through all the $L$ layers. The final embedding uses the weighted sum of the embeddings of each layer:

where $w_{i}$ means the weight in each layer. If the $w$ value is the same in all $i$, the average of the embedding values in all layers is used. The weighted sum can achieve good performance by using not only the last layer’s embedding, but also the previous layer’s embedding. After calculating $\mathbf{E}_{u}^{(final)}$ and $\mathbf{E}_{s}^{(final)}$, the rating of user $u$ for service $s$ is predicted. The dot product of $\mathbf{E}_{u}^{(final)}$ and $\mathbf{E}_{s}^{(final)}$ is performed to predict ratings:

where $\hat{r}_{u,s}$ is the predicted rating value.

We use two generated views from Eqs. (10) and (11) as follows:

After generating the different two views, we employ a contrastive objective that enforces the filtered representations of each node in the two views to agree with each other. We perform the CL training by directly contrasting the distance-based augmented view $\mathbf{e}^{\prime(final)}$ with the random edge drop-based augmented view $\mathbf{e}^{\prime\prime(final)}$ using the InfoNCE [39] loss:

where $i$, $j$ are a user and an item in a sampled batch $\mathcal{B}$, and $\mathbf{e}^{\prime(final)}_{i}$, $\mathbf{e}^{\prime\prime(final)}_{i}$, and $\mathbf{e}^{\prime\prime(final)}_{j}$ are node representations from Eqs. (14) and (15).

We use the Bayesian Personalized Ranking (BPR) loss function [40] together with Eq. (16). As shown in Eq. (17), therefore, our joint learning objective is as follows:

which consists of the Bayesian personalized ranking (BPR) loss $\mathcal{L}_{main}$ and the CL loss $\mathcal{L}_{CL}$. The hyperparameters $\lambda_{1}$ and $\lambda_{2}$ control the trade-off among the two loss functions and the regularization term. $\mathbf{\Theta}$ denotes the embeddings to learn, i.e, $\mathbf{\Theta}=\mathbf{E}(0)$ in our framework. $\mathcal{L}_{main}$ is defined as:

where $\sigma$ is the sigmoid function, $\hat{r}_{ui}$ and $\hat{r}_{uj}$ denote the predicted rating scores for a pair of positive and negative services of user $u$.

After minimizing the joint loss in Eq. (17), we use the output embeddings of the GCN, i.e., $\mathbf{E}_{u}^{(final)}$ and $\mathbf{E}_{i}^{(final)}$, as the final representation. The exact training method is described in Alg. 1.

In the WSDream (Warm-start) dataset of Table II, we compare the recommendation performances between our proposed QAGCL method and baseline models. QAGCL shows the best performance in all evaluation metrics. Based on Recall@40, QAGCL improved by 7.17% over SGL, and based on NDCG@40, it showed a performance improvement of 3.85% over the recommended performance of BPR-MF. Among the baseline models, recommendation models based on graph contrastive learning generally show better results. This shows that the effect of contrastive learning works in performing web service recommendation on the user-service interaction dataset.

In contrast, graph convolution-based CFs show superior recommendation performance compared to BPR-MF and NeuMF. This difference can be attributed to the limitations of the MF-based methods in effectively utilizing neighbor information and high-order connectivity of users and services. NGCF and LightGCN, on the other hand, exhibit slightly improved performance due to their explicit exploration of higher-order connections within the neighborhood. UMEAN and IMEAN notably show relatively lower recommendation performance than the other baselines.

We test the effectiveness of the QAGCL model and baseline models in mitigating the cold-start problem by configuring the cold-start environment by tuning the sparsity of WSDream dataset. In Tables III and IV, we compare the results of the WSDream (Cold-start) dataset, the cold-start setting of WSDream, and the extreme cold-start setting, WSDream (Cold-start-ex).

As shown in Table V, using a combination of HD and ED augmetnation operators for QoS-based service recommendations is effective. The HD takes into account the physical proximity of services. The ED reflects the unpredictable nature of service usage patterns and incorporates diverse perspectives. The randomness of ED allows the QAGCL to adapt to different user behaviors and environmental factors, enhancing the robustness and generalization ability of the recommendation system. By combining these approaches, the augmented graph benefits from localized proximity information and adaptability to diverse user preferences and network conditions. This hybrid approach enhances recommendation performance from a Web QoS perspective.

To understand how the different setting influenced the effectiveness of the QAGCL, we varied the number of graph layers of our methods. When the number of graph convolution layers $L$ is 2, the best performance is shown for Recall@40 and NDCG@40. However, in the case of Recall@20, it is best when $L=2$. And when $L=4$, recommendation performance tends to drop. Through this, the best performance is shown with an optimal $L$.