Clustering-based gradual pattern mining

Abstract

Generally, the classical problem of gradual pattern mining involves generating pattern candidates and determining the number of concordant object pairs associated with them. Given a numeric data set with n objects and m features, each feature yields two gradual items. Gradual pattern candidates can be formed by combining different sets of gradual items. In fact, a gradual pattern is composed of gradual items with similar concordant object pairs. However, computing the object pairs for each item has a complexity that is approximately quadratic in terms of the number of objects. As the main contribution of this paper, we propose finding gradual patterns by clustering gradual items based on their similarity in object pairs. First, we project the object pairs of each gradual item onto an n-dimensional subspace, thus reducing the complexity of computing object pairs from a quadratic function to a linear function. Second, we group gradual items into r clusters based on the similarity of object pairs in the n-dimensional subspace. As part of our experiments, we evaluated our approach using a variety of clustering algorithms. We found that the best clustering algorithms (across all the data sets we used) achieved precision scores above 55%, recall scores close to 100%, and F1 scores above 71%.

Appendices

A Theorems and Proofs

Theorem 1

Under the conditions stated in Sect. 3.4; assume that $b \le \frac{4}{3}$ for any arbitrary small constant or $b > \frac{4}{3}$ for some large constant, with high probability, there exists a permutation $\pi$ such that

$$\begin{aligned} |{\mathcal {C}}_{k} ~\triangle ~ \widehat{{\mathcal {C}}}_{\pi (k)} | \le \frac{C'\cdot r~\text {max}\{n, 2m\}\log {n}\log {2m}}{(1 - \epsilon )n^{2}}, ~~ \forall k. \end{aligned}$$

Particularly, when $(1 - \epsilon )Kn^{2} \ge C'\cdot r~\text {max}\{n, 2\,m\}$ $\log {n}\log ^{2}{2\,m}$ then ${\frac{|{\mathcal {C}}_{k} ~\triangle ~ \widehat{{\mathcal {C}}}_{\pi (k)} | }{K} \le \frac{1}{\log {2\,m}}}$.

Theorem 1 implies that if n and 2m are on the same order, then each gradual item only needs to provide $r^{2}\text {poly}(\log {n})$ object pairs to allow accurate clustering except for $K/\log {2m}$ gradual items.

Theorem 2

Assume the conditions stated in Sect. 3.4. Define

$$\begin{aligned} \eta _{1} = \frac{r~\text {max}\{n,2m\}\log {n}\log {2m}}{(1-\epsilon )Kn^{2}}, ~~ \eta _{2} = \sqrt{\frac{\log {n}}{(1-\epsilon )Kn}} \end{aligned}$$

Assume $b \le \frac{4}{3}$ for any arbitrarily small constant, then there exists a constant $C > 0$ such that with high probability

$$\begin{aligned} \frac{||{\hat{\theta }} _{g} - \theta _{g} ||_{2}}{||\theta _{g} ||_{2}} \le \frac{C (e^{b} + 1)^2}{be^{b}}\text {max}\{\eta _{1}, \eta _{2} \} \end{aligned}$$

Particularly, when

$(1 - \epsilon )Kn^{2} \ge r\text {max}\{n, 2m\}\log {n}\log ^{2}{2m}$ then ${\frac{||{\hat{\theta }} _{g} - \theta _{g} ||_{2}}{||\theta _{g} ||_{2}} = O(\frac{1}{\log {2\,m}})}$ except for $K/\log {2m}$ gradual items.

Theorem 2 demonstrates that the estimation error depends on the maximum of $\eta _{1}$ and $\eta _{2}$. It also shows that Algorithm 1 requires approximately $(1 - \epsilon )Kn^{2} = O(r\text {max}\{n, 2\,m\}\log {n}\log ^{2}{2\,m})$ object pairs per cluster.

Proof of Theorem 1

Let ${\rho = \frac{(1-\epsilon )n}{\sqrt{\log {n}}}}$. We say a gradual item is a good gradual item if $||{\widetilde{S}}_{g} - {\bar{S}}_{g} ||_{2} \le \frac{\rho }{2}$. Let ${\bar{S}}_{k}$ denote the common expected net-win vectors for gradual items in cluster k, where $k = 1, 2,..., r$. Under the model assumptions in Sect. 3.4, for all good gradual items g, for any $k \not = k'$,

$$\begin{aligned} S_{g} - {\bar{S}}_{g} ||_{2} \le \frac{\rho }{2} < {\bar{S}}_{k} - {\bar{S}}_{k'} ||_{2} \end{aligned}$$

$\square$

Following the proofs of Lemma 3, Lemma 4, and Lemma 5 in [31]; if we let ${\mathcal {G}}$ denote the set of good gradual items, then the number of bad gradual items is given by

$$\begin{aligned} |{\mathcal {G}}^{c}| \le \frac{C'\cdot r~\text {max}\{n, 2m\}\log {n}\log {2m}}{(1 - \epsilon )n^{2}}. \end{aligned}$$

Following the proof of Proposition 1 in [32], we make the conclusion that there exists a permutation $\pi$ such that

$$\begin{aligned} |{\mathcal {C}}_{k} ~\triangle ~ \widehat{{\mathcal {C}}}_{\pi (k)} | \le |{\mathcal {G}}^{c}| \end{aligned}$$

Proof of Theorem 2

From Theorem 1, we get that there exists a permutation $\pi$ such that

$$\begin{aligned} |{\mathcal {C}}_{k} ~\triangle ~ \widehat{{\mathcal {C}}}_{\pi (k)} | \le \frac{C'\cdot r~\text {max}\{n, 2m\}\log {n}\log {2m}}{(1 - \epsilon )n^{2}}, ~~ \forall k. \end{aligned}$$

$\square$

In order to get the result of Theorem 2, we apply Theorem 3 in [31]. If we want to achieve $\frac{||{\hat{\theta }} _{g} - \theta _{g} ||_{2}}{||\theta _{g} ||_{2}} = O(\frac{1}{\log {2\,m}})$ for the good gradual items, we need

$$\begin{aligned} \frac{r~\text {max}\{n,2m\}\log {n}\log {2m}}{(1-\epsilon )Kn^{2}} \le \frac{1}{\log {2m}}\\ \sqrt{\frac{\log {n}}{(1-\epsilon )Kn}} \le \frac{1}{\log {2m}} \end{aligned}$$

which requires that $(1 - \epsilon )Kn^{2} > r\text {max}\{n, 2\,m\}\log {n}\log ^{2}{2\,m}$ and $(1 - \epsilon )Kn > n\log {n}\log ^{2}{2\,m}$ respectively. Note that the former requirement is more strict than the latter one; this implies that the clustering step of Algorithm 1 needs more object pairs that the score vector estimation step to achieve the same error rate.

B some properties of bradley-terry model

The proofs in this section are adopted from [28] and [31]. First, we introduce some additional notation used in this section. Let I denote the identity matrix. Let ${\textbf{1}}$ denote the vector with all-one entries and $\textbf{11}^{\top }$ denote the matrix with all-one entries. This section aims to prove that if parameters $\theta$ from Bradley-Terry model are drawn from a uniform distribution on a hypercube, then

$$\begin{aligned} \begin{aligned} ||{\bar{S}}_{u} - {\bar{S}}_{v} ||_{2} \ge C(1-\epsilon )n,\\ \text {where } u, v \text { belong to different clusters.} \end{aligned} \end{aligned}$$

We fix $b \in \mathbb {R}$. We assume that for each cluster $k \in \{1, 2,..., r\}$ and each item $i \in \{1, 2,..., n\}$, $\theta _{k, i}$ is drawn from a uniform distribution on [0, b]. The proof is divided into two parts: (1) the case of $b \le \frac{4}{3}$ and, (2) the case of $b > \frac{4}{3}$. Due to space constraints, these proofs appear in the Appendix of [28].

It should be noted that $AA^{\top } = nI - J \triangleq L_{n}$, which is the Laplacian of the complete graph. It is easy to verify that the eigenvalues of $L_{n}$ are 0, n, and the eigenvector corresponding to the zero eigenvalue is given by $\frac{1}{\sqrt{n}}{\textbf{1}}$. Therefore, A is of rank $m - 1$ and all its nonzero singular values are $\sqrt{n}$. The SVD of A is $A = \sqrt{n}UV^{\top }$ and $S_{g} = \frac{1}{\sqrt{n}}R_{g}A^{\top }$, we have that

$$\begin{aligned} {\bar{S}}_{g} = {\bar{R}}_{g}UV^{\top }. \end{aligned}$$

Using the probability density function in Eq. 2 Let us compute ${\bar{R}}_{g}$:

$$\begin{aligned} \begin{aligned}&\mathbb {E}[R_{g, i, j}] = (1 - \epsilon )\frac{e^{\theta _{g,i}} - e^{\theta _{g, j}}}{e^{\theta _{g,i}} + e^{\theta _{g, j}}} \triangleq (1 - \epsilon )f(\theta _{g, i} - \theta _{g, j}), \text { where }\\&f(x) = \frac{e^{x} - 1}{e^{x} + 1}. \text { Then,}\\&\mathbb {E}[R_{g}] = (1 - \epsilon )f(\theta _{g}A),\\&\text { where } f:x \in \mathbb {R}^{m} \mapsto (f(x_{1}),..., f(x_{n})). \end{aligned} \end{aligned}$$