research-article

Low-rank Representation with Adaptive Dimensionality Reduction via Manifold Optimization for Clustering

Authors:

Xiao WangAuthors Info & Claims

ACM Transactions on Knowledge Discovery from Data, Volume 17, Issue 9

Article No.: 128, Pages 1 - 18

https://doi.org/10.1145/3589767

Published: 15 June 2023 Publication History

Abstract

The dimensionality reduction techniques are often used to reduce data dimensionality for computational efficiency or other purposes in existing low-rank representation (LRR)-based methods. However, the two steps of dimensionality reduction and learning low-rank representation coefficients are implemented in an independent way; thus, the adaptability of representation coefficients to the original data space may not be guaranteed. This article proposes a novel model, i.e., low-rank representation with adaptive dimensionality reduction (LRRARD) via manifold optimization for clustering, where dimensionality reduction and learning low-rank representation coefficients are integrated into a unified framework. This model introduces a low-dimensional projection matrix to find the projection that best fits the original data space. And the low-dimensional projection matrix and the low-rank representation coefficients interact with each other to simultaneously obtain the best projection matrix and representation coefficients. In addition, a manifold optimization method is employed to obtain the optimal projection matrix, which is an unconstrained optimization method in a constrained search space. The experimental results on several real datasets demonstrate the superiority of our proposed method.

References

[1]

M. V. Afonso, J. M. Bioucas-Dias, and M. Figueiredo. 2011. An augmented lagrangian approach to the constrained optimization formulation of imaging inverse problems. IEEE Transactions on Image Process 20, 3 (2011), 681–695.

Digital Library

[2]

R. Bro and A. K. Smilde. 2014. Principal component analysis. Analytical Methods 6, 9 (2014), 2812–2831. DOI:

[3]

J. F. Cai, E. Cand, and Z. W. Shen. 2010. A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization 20, 4 (2010), 1956–1982.

[4]

E. J. Candès, X. Li, Y. Ma, and J. Wright. 2011. Robust principal component analysis? Journal of the ACM 58, 3 (2011), 1–37.

Digital Library

[5]

H. Chen, Y. Sun, J. Gao, Y. Hu, and B. Yin. 2019. Solving partial least squares regression via manifold optimization approaches. IEEE Transactions on Neural Networks and Learning Systems 30, 2 (2019), 588–600. DOI:

[6]

J. Chen, H. Mao, Z. Wang, and X. Zhang. 2021. Low-rank representation with adaptive dictionary learning for subspace clustering. Knowledge-Based Systems 223 (2021), 107053.

[7]

J. Chen, H. Zhang, H. Mao, Y. Sang, and Z. Yi. 2016. Symmetric low-rank representation for subspace clustering. Neurocomputing 173 (2016), 1192–1202. DOI:

Digital Library

[8]

X. Chen, W. Hong, F. Nie, J. Z. Huang, and L. Shen. 2020. Enhanced balanced min cut. International Journal of Computer Vision 128, 7 (2020), 1982–1995. DOI:

[9]

Y. Chen, L. Zhou, N. Bouguila, C. Wang, Y. Chen, and J. Du. 2021. BLOCK-DBSCAN: Fast clustering for large scale data. Pattern Recognition 109 (2021), 107624.

Digital Library

[10]

E. Elhamifar and R. Vidal. 2009. Sparse subspace clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2790–2797.

[11]

E. Elhamifar and R. Vidal. 2013. Sparse subspace clustering: Algorithm, theory, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 11 (2013), 2765–2781.

Digital Library

[12]

Z. Fu, Y. Zhao, D. Chang, and Y. Wang. 2021. A hierarchical weighted low-rank representation for image clustering and classification. Pattern Recognition 112 (2021), 107736.

[13]

Z. Fu, Y. Zhao, D. Chang, X. Zhang, and Y. Wang. 2021. Double low-rank representation with projection distance penalty for clustering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5320–5329.

[14]

A. Georghiades, P. Belhumeur, and D. Kriegman. 2001. From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 6 (2001), 643–660.

Digital Library

[15]

Z. Hu, G. Gao, H. Gao, S. Wu, D. Zhu, and D. Yue. 2018. Robust face recognition via dual nuclear norm low-rank representation and self-representation induced classifier. In Proceedings of the IEEE International Conference on Cloud Computing and Intelligence Systems. IEEE, 920–924.

[16]

P. Ji, M. Salzmann, and H. Li. 2014. Efficient dense subspace clustering. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision. 461–468. DOI:

[17]

K. Lee, J. Ho, and D. Kriegman. 2005. Acquiring linear subspaces for face recognition under variable lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 5 (2005), 684–698.

Digital Library

[18]

Y. Li, P. Hu, Z. Liu, D. Peng, J. T. Zhou, and X. Peng. 2021. Contrastive clustering. Proceedings of the AAAI Conference on Artificial Intelligence 35, 10 (2021), 8547–8555.

[19]

Y. Li, M. Yang, D. Peng, T. Li, J. Huang, and X. Peng. 2022. Twin contrastive learning for online clustering. International Journal of Computer Vision 130 (2022), 2205–2221.

Digital Library

[20]

Z. Li, F. Nie, X. Chang, L. Nie, H. Zhang, and Y. Yang. 2018. Rank-constrained spectral clustering with flexible embedding. IEEE Transactions on Neural Networks and Learning Systems 29, 12 (2018), 6073–6082. DOI:

[21]

Z. Li, F. Nie, X. Chang, Y. Yang, C. Zhang, and N. Sebe. 2018. Dynamic affinity graph construction for spectral clustering using multiple features. IEEE Transactions on Neural Networks and Learning Systems 29, 12 (2018), 6323–6332. DOI:

[22]

Z. Lin, M. Chen, and Y. Ma. 2010. The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. arXiv:1009.50559. Retrieved from https://arxiv.org/abs/1009.50559.

[23]

Z. Lin, R. Liu, and Z. Su. 2011. Linearized alternating direction method with adaptive penalty for low-rank representation. Advances in Neural Information Processing Systems 24 (2011), 612–620.

[24]

G. Liu, Z. Lin, S. Yan, J. Sun, Y. Yu, and Y. Ma. 2013. Robust recovery of subspace structures by low-rank representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 1 (2013), 171–184.

Digital Library

[25]

G. Liu and S. Yan. 2011. Latent low-rank representation for subspace segmentation and feature extraction. In Proceedings of the 2011 International Conference on Computer Vision. 1615–1622.

Digital Library

[26]

R. Liu, Z. Lin, F. De la Torre, and Z. Su. 2012. Fixed-rank representation for unsupervised visual learning. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. 198–605.

[27]

C. D. Manning, P. Raghavan, and H. Schütze. 2010. Introduction to information retrieval. Natural Language Engineering 16, 1 (2010), 100–103.

Digital Library

[28]

S. A. Nene, S. K. Nayar, and H. Murase. 1996. Columbia object image library (COIL-100). Technical Report CUCS-006-96 (1996).

[29]

S. A. Nene, S. K. Nayar, and H. Murase. 1996. Columbia object image library (COIL-20). Technical Report CUCS-005-96 (1996).

[30]

H. Oviedo and H. L. Urdaneta. 2018. A riemannian conjugate gradient algorithm with implicit vector transport for optimization in the Stiefel manifold. Preprint in Optimization-online. Retrieved from http://www.optimization-online.org/DB_FILE/2018/02/6484.pdf. (2018), 1–16.

[31]

X. Peng, J. Feng, S. Xiao, W. Y. Yau, J. T. Zhou, and S. Yang. 2018. Structured AutoEncoders for subspace clustering. IEEE Transactions on Image Processing 27, 10 (2018), 5076–5086. DOI:

[32]

X. Peng, Y. Li, I. W. Tsang, H. Zhu, J. Lv, and J. T. Zhou. 2022. XAI beyond classification: Interpretable neural clustering. Journal of Machine Learning Research 23, 6 (2022), 28.

[33]

F. S. Samaria and A. C. Harter. 1994. Parameterisation of a stochastic model for human face identification. In Proceedings of 1994 IEEE Workshop on Applications of Computer Vision. 138–142.

[34]

H. Sato. 2016. A Dai Yuan type riemannian conjugate gradient method with the weak wolfe conditions. Computational Optimization and Applications 64 (2016), 101–118.

Digital Library

[35]

J. Shi and J. Malik. 2000. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis & Machine Intelligence 22, 8 (2000), 888–905.

Digital Library

[36]

X. Shu, J. Tang, Z. Li, H. Lai, L. Zhang, and S. Yan. 2018. Personalized age progression with Bi-Level aging dictionary learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 40, 4 (2018), 905–917. DOI:

[37]

X. Shu, J. Tang, G. Qi, Z. Li, Y. Jiang, and S. Yan. 2018. Image classification with tailored fine-Grained dictionaries. IEEE Transactions on Circuits and Systems for Video Technology 28, 2 (2018), 454–467. DOI:

[38]

K. Song, X. Yao, F. Nie, X. Li, and M. Xu. 2021. Weighted bilateral k-means algorithm for fast co-clustering and fast spectral clustering. Pattern Recognition 109 (2021), 107560.

Digital Library

[39]

Z. Tao, H. Liu, S. Li, Z. Ding, and Y. Fu. 2019. Robust spectral ensemble clustering via rank minimization. ACM Trans. Knowl. Discov. Data 13, 1 (2019), 25 pages. DOI:

Digital Library

[40]

B. Wang, Y. Hu, J. Gao, Y. Sun, and B. Yin. 2018. Partial sum minimization of singular values representation on Grassmann Manifolds. ACM Transactions on Knowledge Discovery from Data 12, 1 (2018), 22 pages. DOI:

Digital Library

[41]

Y. Wang, C. Chen, J. Lai, L. Fu, Y. Zhou, and Z. Zheng. 2022. A self-representation method with local similarity preserving for fast multi-view outlier detection. ACM Transactions on Knowledge Discovery from Data (2022). DOI:

Digital Library

[42]

J. Wen, X. Fang, Y. Xu, C. Tian, and L. Fei. 2018. Low-rank representation with adaptive graph regularization. Neural Networks 108 (2018), 83–96.

[43]

J. Wen, N. Han, X. Fang, L. Fei, K. Yan, and S. Zhan. 2018. Low-rank preserving projection via graph regularized reconstruction. IEEE Transactions on Cybernetics 49, 4 (2018), 1279–1291.

[44]

J. Wen, B. Zhang, Y. Xu, J. Yang, and N. Han. 2018. Adaptive weighted nonnegative low-rank representation. Pattern Recognition 81 (2018), 326–340.

Digital Library

[45]

C. Yan, X. Chang, M. Luo, Q. Zheng, X. Zhang, Z. Li, and F. Nie. 2020. Self-weighted robust LDA for multiclass classification with edge classes. ACM Transactions on Intelligent Systems and Technology 12, 1 (2020). DOI:

Digital Library

[46]

M. Yin, J. Gao, and Z. Lin. 2015. Laplacian regularized low-rank representation and its applications. IEEE Transactions on Pattern Analysis and Machine Intelligence 38, 3 (2015), 504–517.

Digital Library

[47]

S. Yin, Y. Sun, J. Gao, Y. Hu, B. Wang, and B. Yin. 2021. Robust image representation via low rank locality preserving projection. ACM Transactions on Knowledge Discovery from Data 15, 4 (2021), 22 pages. DOI:

Digital Library

[48]

X. Zhang, Z. Ren, H. Sun, K. Bai, X. Feng, and Z. Liu. 2021. Multiple kernel low-rank representation-based robust multi-view subspace clustering. Information Sciences 551 (2021), 324–340. DOI:

[49]

X. Zhang, X. Xue, H. Sun, Z. Liu, L. Guo, and X. Guo. 2021. Robust multiple kernel subspace clustering with block diagonal representation and low-rank consensus kernel. Knowledge-Based Systems 227 (2021), 107243. DOI:

Digital Library

[50]

Y. Zhang, X. Zhang, S. Yang, and L. Jiao. 2013. Low-rank representation with local constraint for graph construction. Neurocomputing 122 (2013), 398–405.

Digital Library

[51]

W. Zhu and B. Peng. 2020. Sparse and low-rank regularized deep subspace clustering. Knowledge-Based Systems 204 (2020), 106199.

[52]

L. Zhuang, H. Gao, Z. Lin, Y. Ma, X. Zhang, and N. Yu. 2012. Non-negative low rank and sparse graph for semi-supervised learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2328–2335. DOI:

Cited By

Liang KLiu HShan MZhao JLi XZhou L(2023)Enhancing scenic recommendation and tour route personalization in tourism using UGC text miningApplied Intelligence10.1007/s10489-023-05244-654:1(1063-1098)Online publication date: 29-Dec-2023
https://dl.acm.org/doi/10.1007/s10489-023-05244-6

Index Terms

Low-rank Representation with Adaptive Dimensionality Reduction via Manifold Optimization for Clustering
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision representations
        Image representations
2. Mathematics of computing
  1. Probability and statistics
    1. Statistical paradigms
      1. Cluster analysis
      2. Dimensionality reduction

Recommendations

Robust Dimensionality Reduction via Low-rank Laplacian Graph Learning
Manifold learning is a widely used technique for dimensionality reduction as it can reveal the intrinsic geometric structure of data. However, its performance decreases drastically when data samples are contaminated by heavy noise or occlusions, which ...
Nonparametric discriminant multi-manifold learning for dimensionality reduction

Based on that data sampled from the same class locate on one manifold and those labeled different classes reside on the corresponding manifolds, traditional data classification problem can be reasoned to multiply manifolds identification. Thus in this ...
Dimensionality reduction-based spoken emotion recognition

To improve effectively the performance on spoken emotion recognition, it is needed to perform nonlinear dimensionality reduction for speech data lying on a nonlinear manifold embedded in a high-dimensional acoustic space. In this paper, a new supervised ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Knowledge Discovery from Data

ACM Transactions on Knowledge Discovery from Data Volume 17, Issue 9

November 2023

373 pages

ISSN:1556-4681

EISSN:1556-472X

DOI:10.1145/3604532

Editor:
Charu Aggarwal
IBM T. J. Watson Research, USA

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 June 2023

Online AM: 17 April 2023

Accepted: 20 March 2023

Revised: 09 October 2022

Received: 20 April 2022

Published in TKDD Volume 17, Issue 9

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China
Key Technologies R&D Program of Henan Province
Academic Degrees & Graduate Education Reform Project of Henan Province
Key Research Project of Colleges and Universities of Henan Province
Key Science and Technology Development Program of Henan Province
Science and Technology Project of Henan Province
Training Program of Young Backbone Teachers in Colleges and Universities of Henan Province
Research on Key Technologies of Blockchain System Security
Startup Project of Doctor Scientific Research of Zhengzhou University of Light Industry

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
309
Total Downloads

Downloads (Last 12 months)201
Downloads (Last 6 weeks)5

Reflects downloads up to 12 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liang KLiu HShan MZhao JLi XZhou L(2023)Enhancing scenic recommendation and tour route personalization in tourism using UGC text miningApplied Intelligence10.1007/s10489-023-05244-654:1(1063-1098)Online publication date: 29-Dec-2023
https://dl.acm.org/doi/10.1007/s10489-023-05244-6

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

Media

Figures

Other

Tables

View full text|Download PDF

View Issue’s Table of Contents