Graph Summarization with Latent Variable Probabilistic Models

Abstract

This study addresses the issue of summarizing a static graph, known as graph summarization, effectively and efficiently. The resulting compact graph is referred to as a summary graph. Based on the minimum description length principle (MDL), we propose a novel graph summarization algorithm called the graph summarization with latent variable probabilistic models (GSL) for a static graph. MDL asserts that the best statistical decision strategy is the one that best compresses the data. The key idea of GSL is encoding the original and summary graphs simultaneously using latent variable probabilistic models with two-part coding, that is, first encoding a summary graph, then encoding the original graph given the summary graph. When encoding these graphs, we can use various latent variable probabilistic models. Therefore, we can encode a more complex graph structure than the conventional graph summarization algorithms. We demonstrate the effectiveness of GSL on both synthetic and real datasets.

A MDL Principle

Two-Part Coding. Let Y be a random variable and y be a realization. Assume that we observe a set of data $y^{n} = y_{1}, \dots , y_{n} \in \mathcal {Y}^{n}$, where $y_{i} \in \mathbb {R}^{d} \, (i=1, \dots , n)$, and $\mathcal {Y}$ denotes the data domain. The fundamental idea of the MDL principle is to select the best model that minimizes the total code-lengths of the model and data. Let us consider the following parametric model families:

$$\begin{aligned} \mathcal {F} = \{ f(Y^{n} ; \theta , M \in \mathcal {M}) : \theta \in \varTheta \}, \end{aligned}$$

(13)

where $\mathcal {M}$ is a model space and M is a model. Here, in the case of SBM, a model refers to a family of models with the fixed number of groups (clusters), and $\mathcal {M}$ is a set of the family of models with various numbers of groups (clusters). Let us consider how to encode data $y^{n}$ and model M, simultaneously. When we use a prefix code-length L, the total code-length required to encode data $y^{n}$ and M is decomposed into a sum of the code-length of the data given the model and that of the model itself as

$$\begin{aligned} L(y^{n} : M) = L(y^{n} | M) + L(M), \end{aligned}$$

(14)

where $L(y^{n} : M)$, $L(y^{n} |M)$, and L(M) denote the total code-length required to encode $y^{n}$ and M, the code-length to encode $y^{n}$ for the given M, and M, respectively. The prefix code-length refers to the code-length encoded with the prefix coding that satisfies the following Kraft’s inequalities [24]: $\sum _{ y \in \mathcal {Y} } 2^{-L(y | M) } \le 1$ and $\sum _{M \in \mathcal {M}} 2^{-L(M)} \le 1$. Then, the MDL criterion asserts that the total code-length $L(y^{n}: M)$ should be minimized with respect to M:

$$\begin{aligned} L(y^{n} : M) = L(y^{n} | M) + L(M) \quad \Longrightarrow \min . \, \, \mathrm {w.r.t.} \, M. \end{aligned}$$

Normalized Maximum Likelihood (NML) Code-Length. We consider how to achieve the shortest code-length for a family of distributions in Eq. (13). Here, we introduce the normalized maximum likelihood (NML) code-length. The NML code-length is the optimal code-length that achieves Shtarkov’s minimax regret [25]. We consider a model class in Eq. (13). The NML code-length achieves the following Shtarkov’s minimax regret:

$$\begin{aligned} \min _{g} \max _{y^{n} \in \mathcal {Y}^{n}} \left\{ -\log { g(y^{n}) } - \min _{\theta } (-\log { f(y^{n} ; \theta , M) } ) \right\} , \end{aligned}$$

where g is called the normalized maximum likelihood (NML) distribution, a distribution that achieves the minimum regret. The NML distribution $f_{\mathrm {NML}}$ is described as follows: $f_{\mathrm {NML}}(y^{n}) = \frac{ f(y^{n}; \hat{\theta }(y^{n}), M) }{ \int f(Y^{n}; \hat{\theta }(Y^{n}), M) \, d{Y^{n}} }$, where $\hat{\theta }(y^{n})$ is the following maximum likelihood estimator: . It is possible to encode $y^{n}$ with the following code-length using the NML distribution:

(15)

The code-length $L_{\mathrm {NML}}(y^{n}; M)$ in Eq. (15) is called the NML code-length and the last term in the right-hand side is called the parametric complexity to $\mathcal {F}$ in Eq. (13) with data length n.