This Talk

§ 1) Node embeddings
§ Map nodes to low-dimensional
embeddings.

§ 2) Graph neural networks

§ Deep learning architectures for graph-
structured data

§ 3) Applications
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 1
 Part 1:
Node Embeddings

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 2

 Embedding Nodes
A B

Input Output
Intuition: Find embedding of nodes to d-
dimensions so that “similar” nodes in the graph
have embeddings that are close together.
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 3
 Setup
§ Assume we have a graph G:
§ V is the vertex set.
§ A is the adjacency matrix (assume binary).
§ No node features or extra information
is used!

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 4

 Embedding Nodes
• Goal is to encode nodes so that similarity in
the embedding space (e.g., dot product)
approximates similarity in the original network.

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 5

 Embedding Nodes
Goal: similarity(u, v) ⇡ z>
v zu

Need to define!

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 6

 Learning Node Embeddings
1. Define an encoder (i.e., a mapping
from nodes to embeddings)
2. Define a node similarity function
(i.e., a measure of similarity in the
original network).
3. Optimize the parameters of the
encoder so that:
similarity(u, v) ⇡ z>
v zu
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 7
 Two Key Components
§ Encoder maps each node to a low-
dimensional vector. d-dimensional
enc(v) = zv embedding
node in the input graph
§ Similarity function specifies how relationships
in vector space map to relationships in the
original network.
similarity(u, v) ⇡ z>
v zu
Similarity of u and v in dot product between node
the original network embeddings
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 8
 “Shallow” Encoding
§ Simplest encoding approach: encoder
is just an embedding-lookup

enc(v) = Zv
d⇥|V| matrix, each column is node
Z2R embedding [what we learn!]
|V| indicator vector, all zeroes
v2I except a one in column
indicating node v
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 9
 “Shallow” Encoding
§ Simplest encoding approach: encoder
is just an embedding-lookup
embedding vector for a
embedding specific node
matrix

Dimension/size
Z= of embeddings

one column per node

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 10
 “Shallow” Encoding
§ Simplest encoding approach: encoder
is just an embedding-lookup.

i.e., each node is assigned a unique

embedding vector.
§ E.g., node2vec, DeepWalk, LINE

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 11

 “Shallow” Encoding
§ Simplest encoding approach: encoder
is just an embedding-lookup.
§ We will focus on shallow encoding
in this section…
§ After the break we will discuss more
encoders based on deep neural
networks.
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 12
How to Define Node Similarity?
§ Key distinction between “shallow”
methods is how they define node
similarity.
§ E.g., should two nodes have similar
embeddings if they….
§ are connected?
§ share neighbors?
§ have similar “structural roles”?
§ …?
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 13
 Outline of This Section

1. Adjacency-based similarity
2. Multi-hop similarity
3. Random walk approaches
High-level structure and material from:
• Hamilton et al. 2017. Representation Learning on Graphs:
Methods and Applications. IEEE Data Engineering Bulletin
on Graph Systems.

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 14

 Adjacency-based Similarity

Material based on:

• Ahmed et al. 2013. Distributed Natural Large Scale Graph Factorization.
WWW.

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 15

 Adjacency-based Similarity
• Similarity function is just the edge weight
between u and v in the original network.
• Intuition: Dot products between node
embeddings approximate edge existence.
X
L= kz>
u zv Au,v k2
(u,v)2V ⇥V
loss (what we embedding
similarity (weighted)
want to minimize)
sum over all adjacency matrix
node pairs for the graph
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 16
 Adjacency-based Similarity
X
L= kz>
u zv Au,v k2
(u,v)2V ⇥V
• Find embedding matrix 𝐙 ∈ ℝ𝒅 𝐱 |𝑽|that
minimizes the loss ℒ
• Option 1: Use stochastic gradient descent (SGD)
as a general optimization method.
• Highly scalable, general approach
• Option 2: Solve matrix decomposition solvers (e.g.,
SVD or QR decomposition routines).
• Only works in limited cases.
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 17
 Adjacency-based Similarity
X
L= kz>
u zv Au,v k2
(u,v)2V ⇥V
§ Drawbacks:
§ O(|V|2) runtime. (Must consider all node pairs.)
§ Can make O([E|) by only summing over non-zero edges
and using regularization (e.g., Ahmed et al., 2013)
§ O(|V|) parameters! (One learned vector per node).
§ Only considers direct, local connections.
e.g., the blue node is obviously more
similar to green compared to red node,
despite none having direct connections.
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 18
 Multi-hop Similarity

Material based on:

• Cao et al. 2015. GraRep: Learning Graph Representations with Global
Structural Information. CIKM.
• Ou et al. 2016. Asymmetric Transitivity Preserving Graph Embedding. KDD.

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 19

 Multi-hop Similarity
§ Idea: Consider k-hop node neighbors.
§ E.g., two or three-hop neighbors.

• Red: Target node

• Green: 1-hop neighbors
• A (i.e., adjacency matrix)
• Blue: 2-hop neighbors
• A2
• Purple: 3-hop neighbors
• A3

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 20

 Multi-hop Similarity
X
§ Basic idea: L = kz>
u zv Aku,v k2
(u,v)2V ⇥V

§ Train embeddings to predict k-hop neighbors.

§ In practice (GraRep from Cao et al, 2015):
§ Use log-transformed, probabilistic adjacency matrix:
✓ ◆k !
(Ai,j /di )
Ãki,j = max log P k
↵, 0
l2V (Al,j /dl )
node degree constant shift
§ Train multiple different hop lengths and concatenate output.
21

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018

 Multi-hop Similarity
§ Another option: Measure overlap
between node neighborhoods.

§ Example overlap functions:

§ Jaccard similarity
§ Adamic-Adar score
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 22
 Multi-hop Similarity
X
L= kz>
u zv Su,v k2
(u,v)2V ⇥V
multi-hop network similarity
embedding (i.e., any neighborhood
similarity overlap measure)
§ Su,v is the neighborhood overlap between
u and v (e.g., Jaccard overlap or Adamic-
Adar score).
§ This technique is known as HOPE (Yan et
al., 2016).
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 23
 Summary so far
§ Basic idea so far:
§ 1) Define pairwise node similarities.
§ 2) Optimize low-dimensional embeddings to
approximate these pairwise similarities.
§ Issues:
§ Expensive: Generally O(|V|2), since we need to
iterate over all pairs of nodes.
§ Brittle: Must hand-design deterministic node
similarity measures.
§ Massive parameter space: O(|V|) parameters
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 24
 Random Walk Approaches

Material based on:

• Perozzi et al. 2014. DeepWalk: Online Learning of Social Representations.
KDD.
• Grover et al. 2016. node2vec: Scalable Feature Learning for Networks.
KDD.
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 25
Random-walk Embeddings

probability that u
> and v co-occur on
zu zv ⇡ a random walk over
the network

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 26

 Random-walk Embeddings

1. Estimate probability of visiting

node v on a random walk
starting from node u using
some random walk strategy R.

2. Optimize embeddings to
encode these random walk
statistics.

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 27

 Why Random Walks?
1. Expressivity: Flexible stochastic
definition of node similarity that
incorporates both local and higher-
order neighborhood information.
2. Efficiency: Do not need to consider
all node pairs when training; only
need to consider pairs that co-occur
on random walks.
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 28
 Random Walk Optimization
1. Run short random walks starting from each
node on the graph using some strategy R.
2. For each node u collect NR(u), the multiset*
of nodes visited on random walks starting
from u.
3. Optimize embeddings to according to:
X X
L= log(P (v|zu ))
u2V v2NR (u)
* NR(u) can have repeat elements since nodes can be visited
multiple times on random walks.
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 29
 Random Walk Optimization
X X
L= log(P (v|zu ))
u2V v2NR (u)
• Intuition: Optimize embeddings to maximize
likelihood of random walk co-occurrences.
• Parameterize P(v | zu) using softmax:
exp(z>
u zv )
P (v|zu ) = P >z )
n2V exp(z u n
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 30
 Random Walk Optimization
Putting things together:
X X ✓ >
◆
exp(zu zv )
L= log P >z )
n2V exp(z u n
u2V v2NR (u)

sum over all sum over nodes v predicted probability of u

nodes u seen on random and v co-occuring on
walks starting from u random walk

Optimizing random walk embeddings =

Finding embeddings zu that minimize L
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 31
 Random Walk Optimization
But doing this naively is too expensive!!

X X ✓ ◆
exp(z>
u zv )
L= log P >
u2V v2NR (u) n2V exp(zu zn )

Nested sum over nodes

gives O(|V|2) complexity!!

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 32

 Random Walk Optimization
But doing this naively is too expensive!!

X X ✓ ◆
exp(z>
u zv )
L= log P >
u2V v2NR (u) n2V exp(zu zn )

The normalization term from the softmax is

the culprit… can we approximate it?

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 33

 Negative Sampling
Solution: Negative sampling
✓ ◆
exp(z>
u zv )
log P >
n2V exp(zu zn )
k
X
⇡ log( (z>
u zv )) log( (z>
u zni )), ni ⇠ PV
i=1
random distribution
sigmoid function
over all nodes
i.e., instead of normalizing w.r.t. all nodes, just
normalize against k random “negative samples”
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 34
 Negative Sampling
✓ ◆
exp(z>
u zv )
log P > random distribution
n2V exp(zu zn ) over all nodes
k
X
⇡ log( (z>
u zv )) log( (z>
u zni )), ni ⇠ PV
i=1

§ Sample negative nodes proportional to degree.

§ Two considerations for k (# negative samples):
1. Higher k gives more robust estimates.
2. Higher k corresponds to higher prior on negative events.

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 35

Random Walks: Stepping Back
1. Run short random walks starting from each
node on the graph using some strategy R.
2. For each node u collect NR(u), the multiset of
nodes visited on random walks starting from u.
3. Optimize embeddings to according to:
X X
L= log(P (v|zu ))
u2V v2NR (u)
We can efficiently approximate this
using negative sampling!
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 36
How should we randomly walk?
§ So far we have described how to optimize
embeddings given random walk statistics.
§ What strategies should we use to run
these random walks?
§ Simplest idea: Just run fixed-length, unbiased
random walks starting from each node (i.e.,
DeepWalk from Perozzi et al., 2013).
§ But can we do better?

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 37

Feature Learning for Networks
node2vec: Biased Walks
Jure Leskovec
Idea: use flexible,
Stanfordbiased
University random walks
jure@cs.stanford.edu
that can trade off between local and
global views of the network (Grover and
Leskovec, 2016).
s1 s2 s8
careful
s7
cent re-
BFS
d to sig- u s6
features DFS
sitive to s4 s9
s3 s5
r learn- Figure 1: BFS and DFS search strategies from node u (k = 3).
vec, we Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 38
 node2vec: Biased Walks
Stanford University
Jure Leskovec
jure@cs.stanford.edu

Two classic strategies to define a

neighborhood 𝑁@ 𝑢 of a given node 𝑢:
s1 s2 s8
careful
s7
cent re-
BFS
to sig- u s6
eatures DFS
itive to s4 s9
s3 s5
r learn- Figure 1: BFS and DFS search strategies from node u (k = 3).
𝑁
vec, we 𝑢 = { 𝑠 , 𝑠 , 𝑠 } Local microscopic view
eaturesBCD H I J
and edges. A typical solution involves hand-engineering domain-
een net-
𝑁
node’sLCD 𝑢 = { 𝑠 , 𝑠 , 𝑠 } Global macroscopic view
specific features based on expert knowledge. Even if one discounts
M N O
the tedious work of feature engineering, such features are usually
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 39
Interpolating BFS and DFS
Biased random walk 𝑅 that given a node
𝑢 generates neighborhood 𝑁@ 𝑢
§ Two parameters:
§ Return parameter 𝑝:
§ Return back to the previous node
§ In-out parameter 𝑞:
§ Moving outwards (DFS) vs. inwards (BFS)

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 40

 Biased Random Walks
Biased 2nd-order random walks explore
network neighborhoods:
§ Rnd. walk started at 𝑢 and is now at 𝑤
§ Insight: Neighbors of 𝑤 can only be:
Same distance to 𝒖
s2 s3
w Farther from 𝒖

u s1
Closer to 𝒖

Idea: Remember where that walk came from

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 41
 Biased Random Walks
§ Walker is at w. Where to go next?
s2 1
s3
1/𝑝, 1/𝑞, 1 are
w
1/𝑞 unnormalized
u s1 1/𝑝 probabilities

§ 𝑝, 𝑞 model transition probabilities

§ 𝑝 … return parameter
§ 𝑞 … ”walk away” parameter
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 42
 Biased Random Walks
§ Walker is at w. Where to go next?
s2 1
s3
s1 w 1/𝑝
1/𝑞 w → s2 1
u s1 1/𝑝 s3 1/𝑞
§ BFS-like walk: Low value of 𝑝
§ DFS-like walk: Low value of 𝑞 Unnormalized
Unnormalized
transition
transitionprob.
prob.

𝑁D (𝑢) are the nodes visited by the

walker
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 43
 BFS vs. DFS

BFS: DFS:
Micro-view of Macro-view of
neighbourhood neighbourhood

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 44

 Experiments: Micro vs. Macro
Algorithm Dataset
BlogCatalog PPI Wikipe
Spectral Clustering 0.0405 0.0681 0.039
DeepWalk 0.2110 0.1768 0.127
LINE 0.0784 0.1447 0.116
node2vec 0.2581 0.1791 0.155
node2vec settings (p,q) 0.25, 0.25 4, 1 4, 0.

Interactions of characters in a novel: Gain of node2vec [%] 22.3 1.3

Table 2: Macro-F1 scores for multilabel classification on

21.8

Catalog, PPI (Homo sapiens) and Wikipedia word coo

rence networks with a balanced 50% train-test split.

labeled data with a grid search over p, q 2 {0.25, 0.50, 1,

Under the above experimental settings, we present our resul
two tasks under consideration.

4.3 Multi-label classification

In the multi-label classification setting, every node is ass
one or more labels from a finite set L. During the training phas
observe a certain fraction of nodes and all their labels. The t
to predict the labels for the remaining nodes. This is a challe
Figure 3: Complementary visualizations of Les Misérables co- task especially if L is large. We perform multi-label classific

p=1, q=2
appearance network generated by node2vec with label colors
reflecting homophily (top) and structural equivalence (bottom). p=1, q=0.5
on the following datasets:
• BlogCatalog [44]: This is a network of social relation
of the bloggers listed on the BlogCatalog website. Th
Microscopic view of the
also exclude a recent approach, GraRep [6], that generalizes LINE
Macroscopic view of the
bels represent blogger interests inferred through the
data provided by the bloggers. The network has 10,312
network neighbourhood
to incorporate information from network neighborhoods beyond 2-
hops, but does not scale and hence, provides an unfair comparison network neighbourhood
333,983 edges and 39 different labels.
• Protein-Protein Interactions (PPI) [5]: We use a sub
with other neural embedding based feature learning methods. Apart
of the PPI network for Homo Sapiens. The subgraph
from spectral clustering which has a slightly higher time complex-
responds to theWWW
graph induced by nodes for which we
ity since it involves matrix factorization, our
Representation experiments
Learning stand
on Networks, out
snap.stanford.edu/proj/embeddings-www, 2018 45
 Other random walk ideas
(not covered in detail here)
§ Different kinds of biased random walks:
§ Based on node attributes (Dong et al., 2017).
§ Based on a learned weights (Abu-El-Haija et al., 2017)
§ Alternative optimization schemes:
§ Directly optimize based on 1-hop and 2-hop random
walk probabilities (as in LINE from Tang et al. 2015).
§ Network preprocessing techniques:
§ Run random walks on modified versions of the original
network (e.g., Ribeiro et al. 2017’s struct2vec, Chen et
al. 2016’s HARP).

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 46

 Summary so far
§ Basic idea: Embed nodes so that
distances in embedding space reflect
node similarities in the original network.
§ Different notions of node similarity:
§ Adjacency-based (i.e., similar if connected)
§ Multi-hop similarity definitions.
§ Random walk approaches.

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 47

 Summary so far
§ So what method should I use..?
§ No one method wins in all cases….
§ e.g., node2vec performs better on node classification
while multi-hop methods performs better on link
prediction (Goyal and Ferrara, 2017 survey).
§ Random walk approaches are generally
more efficient (i.e., O(|E|) vs. O(|V|2))
§ In general: Must choose def’n of node
similarity that matches application!
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 48
 Multilayer Networks

Material based on:

• Zitnik and Leskovec. 2017. Predicting Multicellular Function through
Multilayer Tissue Networks. ISMB.

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 49

 Multilayer Networks

Let’s generalize to multilayer networks!

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 50
 Multilayer Networks
§ Each network is a layer 𝐺Z = (𝑉Z , 𝐸Z )
§ Similarities between layers are given in
hierarchy ℳ, map 𝜋 encodes parent-
child relationships

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 51

Multilayer Network Embeddings
§ Given: Layers 𝐺Z , hierarchy ℳ
§ Layers 𝐺Z Z_H..` are in leaves of ℳ
§ Goal: Learn functions: 𝑓Z : 𝑉Z → ℝc

Nodes have different

embeddings in different
layers, but we want
these embeddings to be
related!

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 52

Multilayer Network Embeddings
§ Approach has two components:
§ Per-layer objectives: Standard node
embedding objective (e.g., node2vec).
§ Hierarchical dependency objectives:
Nodes in nearby layers in hierarchy are
encouraged to share similar features

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 53

 Interdependent Layers
§ How do we incorporate the hierarchy ℳ
§ Per-layer node2vec objectives are learned
independently of each other
How to model dependencies between
layers when learning node features?

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 54

 ndencies
ode among
u that the layers
resides to define ai:joint objective for regul
in hierarchy
e learned features of proteins.
Interdependent Layers
We propose 1 to use the hierarchy 2in the learning pro
ci (u) = kfi (u) f⇡(i) (u)k2 . (
rporating a2recursive structure into the regularization term f
ct in§ theGiven nodeSpecifically,
hierarchy. 𝑢, learn 𝑢’swerepresentation in
propose the following
mlarization
of regularization
layer 𝑖 to be
for node enforces
close
u that the
to 𝑢’s
resides featuresi:of node
representation
in hierarchy in u
parent
e similar 𝜋(𝑖):
to the features of node u in i’s parent ⇡(
1
n norm. Whenciregularizing
(u) = kfi (u) features
f⇡(i)of all22nodes
(u)k . in th
2
erarchy,
§ we obtain:Repeat at every level of ℳ
Multi-scale:
recursive form of regularization enforces the features of n
X to the features of node u in i’s par
hierarchy i to be similar
Ci = ci (u), (
er the Euclidean norm. When regularizing features of all nod
u2Li
ments𝐿 ihas
ofallthe hierarchy, we obtain:
layers appearing in sub-hierarchy rooted at 𝑖
Z
X
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 55
he mapping functions f1 , f2 , . . . , fT that maxim
OhmNet: Final Model
he data, OhmNet
Learning aims in
node features tomulti-layer
solve thenetworks
following
ptimization problem:
Solve maximum likelihood problem:
X X
max ⌦i Cj ,
f1 ,f2 ,...,f|M |
i2T j2M
Per-layer Hierarchical
network dependency
des the single-layer network
net
objectives
objectives for all
objectives
rarchical dependency objectives for all hierarc
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 56
Experiments: Biological Nets
§ Proteins are worker molecules
§ Understanding protein function
has great biomedical and
pharmaceutical implications
107 genome-wide tissue-specific
protein interaction networks
§ 584 tissue-specific cellular functions
§ Examples (tissue, cellular function):
§ (renal cortex, cortex development)
§ (artery, pulmonary artery morphogenesis)
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 57
 Brain Tissues

Brain

Brainstem
Cerebellum Frontal Parietal Occipital Temporal
lobe lobe lobe lobe

Midbrain Substantia Pons Medulla

nigra oblongata

9 brain tissue PPI networks

in two-level hierarchy
Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 58
Meaningful Node Embeddings

Representation Learning on Networks, snap.stanford.edu/proj/embeddings-www, WWW 2018 59