More Related Content
Similar to 240325_JW_labseminar[node2vec: Scalable Feature Learning for Networks].pptx
Similar to 240325_JW_labseminar[node2vec: Scalable Feature Learning for Networks].pptx (20)
More from thanhdowork
More from thanhdowork (20)
Recently uploaded
Recently uploaded (20)
240325_JW_labseminar[node2vec: Scalable Feature Learning for Networks].pptx
- 1. Jin-Woo Jeong Network Science Lab Dept. of Mathematics The Catholic University of Korea E-mail: zeus0208b@gmail.com Aditya Grover, Jure Leskovec
- 2. 1 Introduction • Motivation • Introduction Feature Learning Framework • Classic search strategies • Node2vec • Random Walks • Search bias 𝛼 • The node2vec algorithm • Learning edge features Experiments • Case Study: Les Misérables network • Multi-label classification • Link prediction Conclusion Q/A
- 3. 2 Introduction Motivation Supervised machine learning algorithms in network prediction tasks require informative, discriminating, and independent features. Typically, these features are constructed through hand-engineering based on domain-specific expertise. However, this process is tedious and the resulting features may not generalize well across different prediction tasks. Previous techniques fail to satisfactorily define and optimize a reasonable objective required for scalable unsupervised feature learning in networks. Classic approaches based on linear and non-linear dimensionality reduction techniques such as Principal Component Analysis, Multi-Dimensional Scaling and their extensions optimize an objective that transforms a representative data matrix of the network such that it maximizes the variance of the data representation. Consequently, these approaches invariably involve eigendecomposition of the appropriate data matrix which is expensive for large real-world networks. Moreover, the resulting latent representations give poor performance on various prediction tasks over networks.
- 4. 3 Introduction Introduction A node could be organized based on communities they belong to (i.e., homophily); in other cases, the organization could be based on the structural roles of nodes in the network (i.e., structural equivalence). Real-world networks commonly exhibit a mixture of such equivalences.
- 5. 4 Introduction Introduction they propose node2vec, a semi-supervised algorithm for scalable feature learning in networks. they optimize a custom graph-based objective function using SGD motivated by prior work on natural language processing. Intuitively, their approach returns feature representations that maximize the likelihood of preserving network neighborhoods of nodes in a d-dimensional feature space. They use a 2nd order random walk approach to generate (sample) network neighborhoods for nodes. Contributions 1. They propose node2vec, an efficient scalable algorithm for feature learning in networks that efficiently optimizes a novel network-aware, neighborhood preserving objective using SGD. 2. They show how node2vec is in accordance with established principles in network science, providing flexibility in discovering representations conforming to different equivalences. 3. They extend node2vec and other feature learning methods based on neighborhood preserving objectives, from nodes to pairs of nodes for edge-based prediction tasks. 4. They empirically evaluate node2vec for multi-label classification and link prediction on several real- world datasets.
- 6. 5 Feature Learning Framework Feature Learning Framework They formulate learning in networks as a maximum likelihood optimization problem. • 𝐺 = 𝑉, 𝐸 be a given graph • 𝑓 ∶ 𝑉 → ℝ𝑑 be the mapping function from nodes to feature representations. • 𝑓 is a matrix of size 𝑉 × 𝑑 parameters. • 𝑑 be a number of dimension of feature representation. • ∀ 𝑢 ∈ 𝑉, 𝑁𝑆 𝑢 ⊂ 𝑉 𝑁𝑆 𝑢 : a network neighborhood of node u generated through a neighborhood sampling strategy S. where
- 7. 6 Feature Learning Framework Classic search strategies • Breadth-first Sampling (BFS) • The neighborhood 𝑁𝑆 is restricted to nodes which are immediate neighbors of the source. For example, in Figure 1 for a neighborhood of size k = 3, BFS samples nodes s1, s2, s3. • Depth-first Sampling (DFS) • The neighborhood consists of nodes sequentially sampled at increasing distances from the source node. In Figure 1, DFS samples s4, s5, s6.
- 8. 7 Feature Learning Framework node2vec • Random Walks • 𝑢 : A source node • 𝑙 : Walk of fixed length • 𝑐𝑖 : 𝑖th node in the walk • 𝑐0 = 𝑢 • 𝜋𝑣𝑥 : unnormalized transition probability • 𝑍 : normalizing constant
- 9. 8 Feature Learning Framework node2vec • Search bias 𝜶 𝜋𝑣𝑥 = 𝛼𝑝𝑞(𝑡, 𝑥) ∙ 𝜔𝑣𝑥 • Return parameter, p • Parameter p controls the likelihood of immediately revisiting a node in the walk. • In-out parameter, q • Parameter q allows the search to differentiate between “inward” and “outward” nodes.
- 10. 9 Feature Learning Framework node2vec The node2vec algorithm • SkipGram algorithm in DEEPWALK Φ → 𝑓 𝑤 → 𝑘 𝑊𝑣𝑖 → 𝑤𝑎𝑙𝑘
- 11. 10 Feature Learning Framework Learning edge features Given two nodes 𝑢 and 𝑣, they define a binary operator ◦ over the corresponding feature vectors f(𝑢) and f(𝑣) in order to generate a representation g(𝑢, 𝑣) such that 𝑔 ∶ 𝑉 × 𝑉 → ℝ𝑑′ where 𝑑′ is the representation size for the pair g 𝑢, 𝑣 . They consider several choices for the operator ◦ such that 𝑑′ = 𝑑 which are summarized in Table 1
- 12. 11 Experiments Case Study: Les Misérables network They use a network where nodes correspond to characters in the novel Les Misérables and edges connect coappearing characters. The network has 77 nodes and 254 edges. They set d = 16 and run node2vec to learn feature representation for every node in the network. The feature representations are clustered using k- means. p = 1, q = 0.5 p = 1, q = 2 homophily structural equivalence
- 16. 15 Conclusion Conclusion In this paper, they researched feature learning in networks as a search-based optimization problem. This perspective provides us with several advantages. It can elucidate traditional search strategies based on the balance between exploration and exploitation. Additionally, it imparts interpretability to the learned representations when applied in prediction tasks. The search strategy in node2vec allows flexible exploration and control of network neighborhoods through parameters p and q. While these search parameters have intuitive interpretations, we achieve the best results on complex networks when we can learn them directly from data. From a practical standpoint, node2vec is scalable and robust. We demonstrated the superiority of extending node embeddings over widely used heuristic scores for link prediction. Their methodology allows additional binary operators beyond those listed in Table 1. They aim to explore the reasons behind the success of the Hadamard operator over others in future work, as well as establish interpretable equivalence notions for edges based on the search parameters.
- 17. 16 Q & A Q / A
Editor's Notes
- thank you, the presentation is concluded