Learning interpretable shared space via rank constraint for multi-view clustering

Abstract

Multi-view clustering aims to assign appropriate labels for multiple views data in an unsupervised manner, which explores the underlying clustering structures shared by multi-view data. Currently, multi-view data is commonly collected from various feature spaces with different properties or distributions. Existing methods mainly utilize the original features to reconstruct the low-dimensional representation of all views, which fail to take the latent relationship and complementarity from multiple views in a unified space into consideration. Therefore, it is urgent to explore a unified space from multi-view ensemble to address the distribution differences between views. In light of this, we learn an interpretable shared space via rank constraint for multi-view clustering (SSRC), which directly reconstructs multi-view data into shared space to explore the underlying complementarity and low-dimensional representation from multiple views. Specifically, SSRC embeds the low-dimensional representation into a reproducing kernel Hilbert space to learn the similarity matrix, which ensures the high correlation between the shared similarity matrix and low-dimensional representation. Furthermore, the rank constraint is imposed on the Laplacian matrix so that the connected component of the similarity matrix is equal to the number of clusters. It can directly obtain the final clustering results in a unified framework through regularization constraints. Then, an ADMM based optimization scheme is devised to seek the optimal solution efficiently. Experiments on 6 benchmark multi-view datasets corroborate that our approach outperforms the state-of-the-art methods.

Appendix:

Appendix contains the solution details of parameter λ. Obviously, there are various values of the regularization parameters. In the appendix, we provide the optimization steps in this paper.

For problem (11), Lagrange multiplier method is utilized, and its Lagrange equation is:

$$ \lambda \left( {\boldsymbol {S_{i}},\theta ,\delta } \right) = \frac{1}{2}\left\| {\boldsymbol {S_{i}} + \frac{\alpha {{d_{i}^{Q}}}}{{2{\lambda_{i}}}}} \right\|_{2}^{2} - \theta \left( {\boldsymbol {S_{i}^{T}}\boldsymbol 1 - 1} \right) - {\delta_{i}^{T}}\boldsymbol {S_{i}}. $$

(25)

where 𝜃 and δ are Lagrange multipliers. By calculating the partial derivative of S and setting it equation to zero, (25) is reformulated as:

$$ \boldsymbol {S_{i}}{\text{ + }}\frac{\alpha {{d_{i}^{Q}}}}{{2{\lambda_{i}}}} - \theta - {\delta_{i}} = 0. $$

(26)

According to KKT condition [1], we can obtain the optimal solution for S as follows:

$$ \boldsymbol {S_{ij}} = {\left( - \frac{\alpha{d_{ij}^{Q}}}{{2{\lambda_{i}}}} + \theta \right)_ + }. $$

(27)

In order to obtain the optimal nearest neighbor, we suppose the variable ${d_{ij}^{Q}}$ are ordered from small to large: ${d_{i1}^{Q}},{d_{i2}^{Q}},...,{d_{in}^{Q}}$.

If there are only k nonzero elements, that S can get the optimal solution. That is, S_i,k > 0 and S_i,k+ 1. So, we have:

$$ \left\{ \begin{array}{l} - \frac{\alpha{d_{ik}^{Q}}}{{2{\lambda_{i}}}} + \theta > 0,\\ - \frac{\alpha {d_{i,k + 1}^{Q}}}{{2{\lambda_{i}}}} + \theta \le 0. \end{array} \right. $$

(28)

According to the constraint $\boldsymbol {S_{i}^{T}}\boldsymbol 1 = 1$ and (27), we can get the solution of parameter 𝜃:

$$ \theta {\text{ = }}\frac{1}{k}\left( 1 + \frac{\alpha}{{2{\lambda_{i}}}}\sum\limits_{j = 1}^{k} {d_{ij}^{Q}} \right). $$

(29)

According to (25) and (26), λ can be formulated as:

$$ \left\{ \begin{array}{l} \lambda \le \frac{1}{2}\left( kd_{ik + 1}^{Q} - \alpha \sum\limits_{j = 1}^{k} {d_{ij}^{Q}} \right),\\ \lambda > \frac{1}{2}\left( kd_{ik}^{Q} - \alpha \sum\limits_{j = 1}^{k} {d_{ij}^{Q}} \right). \end{array} \right. $$

(30)

The optimal solution of S can be obtained by limiting the number of nonzero entries in k. So the parameter λ is set to:

$$ \lambda {\text{ = }}\frac{\alpha}{2}\left( kd_{ik + 1}^{Q} - \sum\limits_{j = 1}^{k} {d_{ij}^{Q}} \right). $$

(31)