A feature-enhanced knowledge graph neural network for machine learning method recommendation

Graph neural networks

Graph neural networks (GNN) are widely applied in recommendation systems by utilizing auxiliary graph-structured information. GNN offers a unified framework to manage the substantial data in recommender systems and explicitly encode critical collaborative signals to enhance the representations of users and items. Numerous GNN variants have been proposed in recent years. For instance, dynamic representation learning via recurrent graph neural networks (Zhang et al., 2023) designed a one-stage model that integrates a recurrent neural network into GNN to generate compact representations. Gated recursion-based graph neural network (Ge, Zhao & Zhao, 2022) proposed a gated recursive algorithm in order to address the node aggregation issue and extract deep dependency features among nodes. Multi-relational graph attention networks (Li et al., 2022b) captured more complex semantic relationships between entities through an attention mechanism and embedding edge relationship types. Although GNN-based recommendation systems have enhanced the effectiveness of recommendation, it is necessary to exploit more comprehensive graph structures and semantic relationships between entities. Thus, researchers attempt to apply the graph neural network in knowledge graphs to further improve the recommendation performance.

Knowledge graph-based recommender systems

A knowledge graph is a graphical database that is specifically crafted to organize, represent, and store structured knowledge (Li, Qu & Wang, 2023). Its fundamental concept was visually representing real-world information and concepts with a triple <entity, relationship, entity>. A knowledge graph is utilized in various domains, such as question answering, text classification and recommendation systems (Qiu et al., 2020; Zhu et al., 2023; Cui et al., 2022b; Zhang et al., 2022b). Recommendation methods based on knowledge graphs can be divided into three categories: embedding-based, path-based and propagation-based methods.

Embedding-based methods. The embedding-based methods leveraged knowledge graph information to enhance the representation of each entity. The entities and relationships in the knowledge graph were encoded as low-dimensional vectors in the embedding-based methods to preserve the inherent graph structure. For instance, personalized recommendation system based on knowledge embedding and historical behavior (Hui et al., 2022) used self-attention to determine user preferences and integrated knowledge graph embeddings with user behavior. The deep interest network based on knowledge graph embedding (Zhang et al., 2022a) addressed the limitations of static interaction matrices by combining gated recurrent units with an attention mechanism. A knowledge graph-based approach for visualization recommendation (Li et al., 2022a) enhanced the performance of TransE by replacing evidently incorrect triples with self-adversarial negative sampling. Despite the simplicity and flexibility, these approaches overlooked the complex semantic relationships between entities in the knowledge graph, which leads to the fact that the entity expression in the knowledge graph was monotonous.

Path-based methods. The path-based approach enhanced recommendations by identifying connectivity similarities among paths from users to items. This was achieved by constructing a user-item graph and leveraging entity connectivity to enhance recommendation effectiveness while preserving interpretability. For instance, reinforced sequential learning with gated recurrent unit (Cui et al., 2022a) combined reinforcement path reasoning network components and gated recurrent units to enhance path reasoning capabilities. Reinforcement learning framework for multi-level recommendation reasoning (Wang et al., 2022) addressed local optimization issues by incorporating abstract markov decision processes and developed a multi-layer path extraction algorithm to improve model performance. Path language modeling recommendation (Geng et al., 2022) predicted new paths based on higher joint path probability allocation scores, effectively extending the reachability of items that traditional methods cannot achieve. However, valuable information may be lost by separating complex user-item connectivity into discrete linear paths in the path-based recommendation methods.

Propagation-based methods. The propagation-based recommendation methods optimized the utilization of knowledge graph information by integrating connectivity information and semantic representations of entities and relationships. These methods utilized embedding propagation, which aggregated the embeddings of multi-hop neighbor nodes in the knowledge graph, to enhance entity representation. These methods facilitated the anticipation of user preferences by comprehensively representing users and items. For instance, knowledge graph convolutional networks (Wang et al., 2019b) and knowledge graph convolutional networks with label smoothness (Wang et al., 2019c) leveraged graph convolutional networks to compute neighborhood-propagated embeddings of items in knowledge graphs. Description enhanced knowledge graph recommendation (Cao et al., 2021) overcame the limitations of lacking textual descriptive information by combining knowledge graph-based and text-based approaches. Cross-modal knowledge graph contrastive learning (Cao et al., 2022) learned node representations by considering descriptive attributes and structural connections as two modalities and maximizing the consistency between them. Knowledge-adaptive contrastive learning (Wang et al., 2023) learned the generated user-item interaction view and knowledge graph view by introducing contrastive learning. These techniques took the entire knowledge graph as input, obtaining the embedding for each entity by aggregating information from all its neighbors. However, the recommendation performance may be hurt by the lack of constraints between the target entity and its neighbors since the embedding learning process inevitably introduces noise (less informative neighbors) and involves substantial computation (a large number of neighbors).

The influence of direct neighbors during higher-order propagation is ignored in these propagation-based methods. In this article, the features of nodes involved in the high-order propagation process of the target entity are enhanced through their first-order neighbor information. An anti-smoothing aggregation network is designed to reduce the impact of over-smoothing.

Overall framework

The proposed FEGNN approach consists of two main components. The first component is a graph neural network that integrates the feature-enhanced graph neural network and an anti-smoothing aggregation network by exploring the high-order connectivity of entities in the machine learning knowledge graph. The other component is a deep text-based collaborative filtering network that captures linear and nonlinear textual content interactions. Finally, the outcomes of the two components are integrated to predict the interaction probability of a particular target entity. The proposed FEGNN approach is illustrated in Fig. 1.

All of the parts mentioned below contribute to the novel component, namely “feature-enhanced knowledge graph neural network”. The first part of FEGNN is “Linking and Propagating”. It can be easily observed that the proposed method takes the target dataset and method as inputs. In the “Linking and Propagating” part, the target dataset and candidate machine learning method are initially linked to the corresponding entity nodes in the machine learning knowledge graph $G$. The linked entities $e_d$ and $e_m$ serve as the seed nodes. Different from the propagation process in DEKR, the features of general nodes involved in the propagation process for the target entity are enhanced by their first-order neighborhood information, as shown in yellow. The article defines these feature-enhanced nodes as $\left\{g_1,g_2,\dots ,g_i\right\}$. Then, the target node extends progressively to higher-order neighbor nodes through these feature-enhanced ones. This article designs an anti-smoothing aggregation network to acquire entity representations to predict the probability ${\hat{y}}_1$.

Similar to DEKR, the article uses the text-based deep collaborative network component to capture interaction information in the textual descriptive information and predict the probability ${\hat{y}}_2$. Both components determine the ultimate interaction probability of the target dataset and method.

Feature-enhanced knowledge graph neural network

Feature enhancement. On the basis of the knowledge graph, graph neural networks provide a method of further exploiting the rich connections between entities. Considering the close relationship between a node and its direct neighbors, these neighbors may convey informative messages. For instance, the propagated node represents a machine learning method, and its embedding information in the traditional knowledge graph method may only include the name, which is insufficient. More messages in the knowledge graph method such as its associated task, datasets and authors, are needed to explain a machine learning method better. These additional messages can be obtained from its direct neighbors in the knowledge graph. Therefore, the article actively enhances the features of these general nodes $\left\{e_1,e_2,\dots ,e_i\right\}$ involved in the propagation process of the target entity as feature-enhanced nodes $\left\{g_1,g_2,\dots ,g_i\right\}$ by aggregating information from itself and its direct neighbors. Then, the target entity is propagated over these feature-enhanced nodes to obtain a more comprehensive higher-order representation.

Nevertheless, the significance of the neighbors for a target entity should not be uniformly assessed. For example, given a specific dataset, a machine learning method experimented on the dataset theoretically provides more impact than the author with which the dataset has interacted. This distinction arises because the methods using the same dataset typically address similar tasks, while authors may be involved in diverse fields. Therefore, it is crucial to distinguish the importance of the neighbors. In this article, the following formula is designed to calculate the attention scores for different relations ( $r$) of node ( $e$):

(2) $${\tilde{\pi }}_{r,e_t}={\displaystyle \frac{exp\left({\pi }_{r,e_t}\right)}{{\sum }_{e_t\in N\left(e\right)}exp\left({\pi }_{r,e_t}\right)}.}$$where $e_t$ is the neighbor of node $e$, ${\tilde{\pi }}_{r,e_t}$ represents the relevant weight of neighbors, and the higher weights indicate the necessary to transmit more information for the target node. As each node has a varying number of neighbors, a fixed sampling strategy is employed to enhance the training efficiency.

A weighted summation approach is used to calculate the representation of neighborhood information. This approach utilizes correlation weights derived from Eq. (2) denoted as Nei.

(3) $$e_{Nei}={\sum }_{e_t\in N\left(e\right)}{\tilde{\pi }}_{r,e_t}{e}_t.$$

Thus, more informative messages are exploited by means of integrating the neighbor information and its own information. The calculation formula is as follows:

(4) $$g=LeakyReLU\left(W_0\left(e^{\left(0\right)}+e_{Nei}^{\left(0\right)}\right)+b_0\right).$$

Here, LeakyRelu is adopted as the activation function, which can handle both positive and negative signals. $W_o$ and $b_0$ are the learnable parameters.

Consequently, high-order representations of entities are derived based on these feature-enhanced nodes $\left\{g_1,g_2,\dots ,g_i\right\}$.

Anti-smoothing aggregation networks

One-hop Anti-smoothing Propagation. A single propagation layer is taken as an example to better explain the mechanism of the anti-smoothing aggregation network. A specific dataset-method pair is treated as the seed node in this illustrated example. Once these nodes are mapped to the corresponding nodes $\left(e_d-e_m\right)$ in the knowledge graph, connections can be established with other entities (first-order neighbors) through various relationships. Distinguishing the contributions of different relations ( $r$) for the feature-enhanced node ( $g$) is essential. Therefore, the following formula is used to calculate the contribution weights:

(5) $${\tilde{\pi }}_{r,g}^d={\displaystyle \frac{exp\left({\pi }_{r,g}^d\right)}{{\sum }_{g\in N\left(m\right)}exp\left({\pi }_{r,g}^d\right)}.}$$

(6) $${\tilde{\pi }}_{r,g}^m={\displaystyle \frac{exp\left({\pi }_{r,g}^m\right)}{{\sum }_{g\in N\left(d\right)}exp\left({\pi }_{r,g}^m\right)}}$$

Here, ${\tilde{\pi }}_{r,g}^d$ and ${\tilde{\pi }}_{r,g}^m$ denote the correlation weights calculated by $N\left(m\right)$ and $N\left(d\right)$ with reference to $g_d$ and $g_m$, respectively.

Then, a weighted summation is adopted to indicate the aggregated neighborhood information using the formula below:

(7) $$g_{N\left(d\right)}={\sum }_{g\in N\left(d\right)}{\tilde{\pi }}_{r,g}^{d}g$$

(8) $$g_{N\left(m\right)}={\sum }_{g\in N\left(m\right)}{\tilde{\pi }}_{r,g}^{m}g.$$

In the following step, the node’s self-information is updated by merging the information of its neighbors during the propagation process to accumulate more enriched messages with each convolutional layer. At the same time, node features are augmented to provide richer information for the FEGNN.

However, these may exacerbate the issue of over-smoothing. Considering these factors, an anti-smooth aggregation network is proposed for the aggregation process. This structure can systematically reduce the effect of self-information during each aggregation. The goal is to prevent an undue reliance on progressively enriched self-information to alleviate the problem of over-smoothing. The calculation formula for the anti-smoothing structure is shown below.

(9) $$a_l=\alpha \ast e^{-\left(l+1\right)}$$where $l$ is denoted as the current number of propagation layers, and $\alpha$ is a decay parameter that adjusts smoothing. After aggregating neighborhood information, the final representation in one-hop propagation for a given entity is shown as follows.

(10) $$g_d^{\left(1\right)}=LeakyReLU\left(W_0\left(a_0\ast g_d^{\left(0\right)}+g_{N\left(d\right)}^{\left(0\right)}\right)+b_0\right)$$

(11) $$g_m^{\left(1\right)}=LeakyReLU\left(W_0\left(a_0\ast g_m^{\left(0\right)}+g_{N\left(m\right)}^{\left(0\right)}\right)+b_0\right)$$where $W_0$ and $b_0$ are the learnable parameters, LeakyReLU is adopted as the activation function. $g_d^{\left(0\right)}$ and $g_{N\left(d\right)}^{\left(0\right)}$ denote the original representations of the node and its neighbors, respectively.

Higher-order propagation. The ultimate representation of the entity is acquired by multiple layers of propagation to capture more complex connection details and information from distant neighbors. The computational formula is expressed as follows.

(12) $$g_d^{\left(l\right)}=LeakyReLU\left(W_{l-1}\left(a_{l-1}\ast g_d^{\left(l-1\right)}+g_{N\left(d\right)}^{\left(l-1\right)}\right)+b_{l-1}\right)$$

(13) $$g_m^{\left(l\right)}=LeakyReLU\left(W_{l-1}\left(a_{l-1}\ast g_m^{\left(l-1\right)}+g_{N\left(m\right)}^{\left(l-1\right)}\right)+b_{l-1}\right).$$

In this equation, $W_{l-1}$ represents the trainable weight matrix, and LeakyReLU is configured as the activation function. $g^{\left(l-1\right)}$ denotes the entity representation derived from the preceding aggregation layer, and the higher-order representation for a target entity is obtained by multiple high-order propagations.

Prediction layer. After multiple hops of aggregation, the final representations for the target dataset and machine learning method, namely $g_d^{\left(l\right)}$, $g_m^{\left(l\right)}$, are derived by incorporating information from its $l$-hop neighbors. The interaction probability between the target dataset and the machine learning method is predicted by leveraging the graph neural network, as shown below.

(14) $${\hat{y}}_1=\sigma \left({\left(g_d^{\left(l\right)}\right)}^T{g}_m^{\left(l\right)}\right)$$where $\sigma$ is the sigmoid activation function.

Text-based collaborative filtering

The graph neural network captures the structure information by utilizing auxiliary knowledge graph data. However, relying solely on the graph structure poses a challenge due to the brevity of the illustration of machine learning methods and datasets, which leads to factual inconsistencies. For instance, the path in (MovieLens-20M, dataset. method, DeepFM), (DeepFM, method. dataset, Bing News), (Bing News, dataset. method, DKN) might incorrectly suggest MovieLens-20M is suitable for DKN. Actually, DKN is a text-based recommendation method, while MovieLens-20M is a film dataset without effective textual information.

This article uses a content-based deep collaborative filtering network to predict the interaction probability for the target dataset and method. Note that the description nodes of core entities are not engaged in the propagation to higher-order neighbors. For the textual description related to the target entity, the n words of the original description are represented as t = $w_{1:n}$ = [ $w_1$, $w_2$, …, $w_n$]. The matrix $s_{1:n}\in R^{p\times n}$ signifies the embedding matrix of the sentence. Pretrained words to vectors are produced on GloVe (Pennington, Socher & Manning, 2014), the initial embedding representation for each word in which global statistical information and local contextual features are contained. The sentence representation $s_t\in R^p$ is derived by aggregating the average of the word representations.

For the descriptive information $e_{\left(d\right)}^t$ and $e_{\left(m\right)}^t$ associated with a given dataset-method pair $g_{\left(d\right)}$- $g_{\left(m\right)}$, the low-dimensional vectors $v_d$ and $v_m$ are obtained through matrix transformation. This process can be expressed as follows.

(15) $$v_d=W_d{s}_d$$

(16) $$v_m=W_m{s}_m.$$

To further explore the interaction between the dataset and the machine learning method, a neural collaborative filtering framework (He et al., 2017) is utilized in this article. Generalized matrix factorization captures linear interactions in the descriptive information for the dataset and method. Simultaneously, a multi-layer perceptron layer captures nonlinear interactions in the descriptive information. A concept similar to matrix factorization is employed to extract linear interaction between the dataset and method description features.

(17) $${\varphi }_l=v_d\odot v_m$$

(18) $${\hat{y}}_l=\sigma \left(G^T{\varphi }_l\right).$$

Here, $⨀$ denotes the element-wise product. $G^T$ denotes the learnable weight matrix. A sigmoid activation function is adopted as $\sigma$.

Meanwhile, the two feature vectors $v_d$ and $v_m$ are concatenated to capture nonlinear interaction features through a multi-layer perceptron. The formulation is described as follows.

(19) $$h_o=v_d\parallel v_m$$

(20) $${\varphi }_{nl}=h_n=\sigma \left(W_n^T{h}_{n-1}+b_n\right)$$

(21) $${\hat{y}}_{nl}=\sigma \left(M^T{\varphi }_{nl}\right).$$

Here $\parallel$ donates the concatenation operation of two vectors, and $\sigma$ represents the sigmoid activation function. $M^T$ denotes the learnable weight matrix.

Ultimately, the two hidden vectors of the last layer of both networks are concatenated as inputs for the neural matrix factorization layer. Both linear and non-linear features are integrated in the text-based collaborative filtering component. The interaction probability between the target dataset and the machine learning method is predicted as follows.

(22) $${\hat{y}}_2=\sigma \left(W_t^T\left({\varphi }_l\parallel {\varphi }_{nl}\right)\right).$$

Prediction

After incorporating the two critical components in FEGNN, the embedding representations for both the dataset and the method are obtained. This involved utilizing both the graph structure and text description information. As the two embedding representations are obtained through different learning methods, our approach enables the model to learn and optimize these representations independently. Subsequently, the final predicted probability is expressed as follows.

(23) $${\hat{y}}_{dm}=W^T(\sigma \left({\left(g_d^{\left(h\right)}\right)}^T{g}_m^{\left(h\right)}\right)+\sigma \left(W_t^T\left({\varphi }_l\parallel {\varphi }_{nl}\right)\right)).$$

The loss function is defined as follows:

(24) $$\mathcal{L}={\sum }_{d\in D,m\in M}\left(\mathcal{J}\left({\hat{y}}_1,y_{dm}\right)+\mathcal{J}\left({\hat{y}}_2,y_{dm}\right)\right)+\lambda \parallel \mathrm{\Theta }\parallel$$where $\mathcal{J}$ is the cross-entropy function, when the actual output ${\hat{y}}_1$ or ${\hat{y}}_2$ is close to the desired output $y_{dm}$, the value of the cost function is close to zero. The cross-entropy function $\mathcal{J}$ helps to accelerate the speed of updating the weights. $\lambda$ controls the regularisation strength and $\mathrm{\Theta }$ is the set of parameters.

Dataset and preprocessing

The Machine Learning Dataset (Cao et al., 2021) utilized in this article includes machine learning-related datasets, methods, attributes, and relevant entities sourced from open academic platforms such as Paperswithcode and GitHub. Actually, 19 areas are involved in the dataset, such as computer vision, natural language processing, reinforcement learning, and so on. The detailed messages about the dataset are listed in Table 1. Descriptive information for the datasets and methods is extracted from the context of the tasks using the datasets, the titles or abstracts of articles related to the methods. Table 2 presents the fundamental statistics of the dataset.

Baselines

This article evaluates the proposed FEGNN approach with various existing strong baselines.

● BPR (Rendle et al., 2009). This approach utilizes Bayesian analysis on the traditional factorization machine model for optimization.

● KGCN (Wang et al., 2019b). This approach utilizes the similarity of the relationship of different users to assign varying weights to neighbors to obtain a higher-order representation of the entity.

● KGNN-LS (Wang et al., 2019c). This approach generates personalized embeddings for each item by introducing label smoothing regularization based on KGCN.

● KGAT (Wang et al., 2019a). This approach employs the graph attention mechanism to evaluate the importance of different neighbors and derives entity representations for recommendation through graph neural networks.

● CKE (Zhang et al., 2016). This approach integrates collaborative filtering with structural, text, and visual knowledge into a unified recommendation framework.

● MKGAT (Sun et al., 2020). This approach performs the multimodal graph attention mechanism in multimodal graphs to obtain a better embedding representation.

● DEKR (Cao et al., 2021). This approach constructs a description-enhanced knowledge graph and combines knowledge graph-based and text-based approaches.

● CKGC (Cao et al., 2022). This approach achieves cross-modal knowledge graph contrastive learning by maximizing the agreement between the descriptive view representations and structural view representations.

Experiments setup

Evaluation Metrics: In this experiment, Precision@K, Recall@K, and NDCG@K are used as Top-K recommendation metrics. Additionally, the evaluation metrics of click-through-rate (CTR) prediction are assessed through area under curve (AUC), Accuracy (ACC) and F1-score.

Parameter settings: The proposed FEGNN approach is implemented by using PyTorch. The dimensions for graph structure embedding and text embedding are set to 64, with each node having eight neighbors. Moreover, the number of graph convolution iterations is set to 2. The Adam optimizer (Kingma & Ba, 2017) is employed for all the methods to optimize the training process, and a batch size of 128 is chosen. A grid search is conducted for the learning rate and regularization factor, with values {10⁻⁴, 5 × 10⁻⁴, …, 10⁻¹, 5 × 10⁻¹} and {10⁻⁶, 10⁻⁵, …, 10⁻², 10⁻¹}, respectively. The embedding size for all baseline models is set to 64. For KGCN, KGNN-LS and KGAT methods, the number of propagation hops is set to 2 and CKGC is set to 3. Specifically, for the KGCN method, the number of neighborhood samples is set to 8, and the sum aggregator is chosen as the aggregation method. For the KGAT method, the bi-interaction aggregator is employed for aggregation, and node drop is applied, just as KGAT suggested.

Comparison of results (Q1)

Table 3 presents the Top-K recommendation results and Table 4 shows the results of CTR prediction, respectively. From these experimental results, the article makes the following observations:

• In most cases, FEGNN behaves the best both in the Top-K recommendation and CTR prediction. To validate the significance of the experimental results, the Unpaired Two Sample t-test (Mishra et al., 2019) with p < 0.05 is conducted on the CTR prediction results of DEKR and FEGNN. The significant results indicate that a notable improvement of over 40% in the Top-10 recommendation is observed, and over 55% in the Top-20 recommendation is captured by applying the proposed FEGNN method on the Machine Learning Dataset.

• Among the baseline methods, the hybrid recommendation models based on knowledge graphs, including MKGAT, DEKR, CKGC and FEGNN, achieve superior performances. In these hybrid recommendation models, the multimodal knowledge is believed to bring benefits for enhancing the recommendation performance. The performance of CKE also benefits from its consideration of multimodality, but the simple embedding of knowledge may hurt the effectiveness of the recommendation.

• Both CKGC and FEGNN perform significantly better than DEKR. DEKR obtains a comprehensive representation of entities by combining knowledge graph-based and text-based approaches. CKGC achieves contrastive knowledge learning by maximizing the agreement between the descriptive view representations and structural view representations. FEGNN captures a more comprehensive graph structure representations through the feature-enhanced graph neural network.

• Compared with CKGC, FEGNN behaves better when Top-K recommendation is taken as the evaluation measure. The click-through-rate prediction by applying both CKGC and FEGNN, are comparable. It is suggested that the graph structure representation obtained through the traditional graph neural network limits the recommendation performance of CKGC, while more comprehensive graph structure representations are captured in FEGNN through a novel feature-enhanced graph neural network.

Ablation experiment (Q2)

To verify whether node feature enhancement and the anti-smoothing aggregation network can improve the performance of recommendation, ablation experiments are conducted. Two variants of FEGNN are devised: FEGNN-anti (only the anti-smoothing network contained) and FEGNN-fea (only feature enhancement contained).

Based on the results in Fig. 2 and Table 5, it is evident that FEGNN-fea brings notable advantages in improving the recommendation performance while FEGNN-anti has minimal impact on the recommendation performance. It is attributed to the limited number of model iterations and neighborhood samples, making them less susceptible to embedding smoothing. Nevertheless, as the propagation aggregation process relies on the feature-enhanced nodes, the entity embedding contains more information and is more susceptible to embedding smoothing issues. Therefore, the effectiveness of the proposed FEGNN method is further improved by combining feature enhancement and the anti-smoothing network.

Study of parameters (Q3)

This section examines the effects of different numbers of sampled neighbors, propagation layers, and embedding dimensions.

Impact of neighbor sampling size: We assess the model’s performance by varying the number of neighbor samples. Table 6 indicates that the optimal results can be fetched when the number of neighbor samples is set to eight or 16. It is believed that each entity and relationship is relatively condensed in the machine learning knowledge graph compared to other knowledge graphs. Specifically, a too-small value for K lacks the capacity to collect sufficient neighborhood information, while a huge K introduces noise, which may lead to a decline in recommendation performance. K is set to eight in the experiments to strike a balance between effectiveness and efficiency.

Impact of receptive field depth: The number of iterations H is varied from 1 to 4. The results in Table 7 indicate that the optimal performance occurs with two and three iterations. However, when the iteration is set to 4, the recommendation performance declines significantly, accompanied by a substantial increase in training time. This decline is attributed to insufficient graph structure information. In the process of inter-item similarity inference, excessively long connectivity relationships between entities are almost meaningless (Wang et al., 2019b). Based on these experimental results, the number of iterations is set to 2 in the experiments.

Impact of embedding dimension: The dimensions of entity embedding are varied to analyze experimental results in Table 8. Initially, the recommendation performance is enhanced by increasing d. However, an excessively high value for d becomes sensitive, which negatively impacts the recommendation performance.