Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
SpringerLink
Log in

LSTM-assisted evolutionary self-expressive subspace clustering

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Massive volumes of high-dimensional data that evolve over time are continuously collected by contemporary information processing systems, which bring up the problem of organizing these data into clusters, i.e. achieving the purpose of dimensional reduction, and meanwhile learning their temporal evolution patterns. In this paper, a framework for evolutionary subspace clustering, referred to as LSTM–ESCM, is introduced, which aims at clustering a set of evolving high-dimensional data points that lie in a union of low-dimensional evolving subspaces. In order to obtain the parsimonious data representation at each time step, we propose to exploit the so-called self-expressive trait of the data at each time point. At the same time, LSTM networks are implemented to extract the inherited temporal patterns behind data in the overall time frame. An efficient algorithm has been proposed. Numerous experiments are carried out on real-world datasets to demonstrate the effectiveness of our proposed approach. The results show that the suggested algorithm dramatically outperforms other known similar approaches in terms of both run time and accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. As these dimensions of those two: input and output variable, do not align, it is impossible to define a customized loss function in MATLAB deep learning toolbox. But fortunately we have managed to rewrite the loss as the form that is compatible with the requirement.

  2. http://www.vision.jhu.edu/

  3. https://usgodae.org/argo/argo.html.

  4. http://sio-argo.ucsd.edu/RG_Climatology.html

  5. https://www.who.int/data/collections

  6. https://data.worldbank.org/indicator

References

  1. BoussaïD I, Lepagnot J, Siarry P (2013) A survey on optimization metaheuristics. Inform Sci 237:82–117

    Article  MathSciNet  Google Scholar 

  2. Bradley PS, Mangasarian OL (2000) K-plane clustering. J Global Opt 16(1):23–32

    Article  MathSciNet  Google Scholar 

  3. Cao X, Zhong Y, Zhou Y, Wang J, Zhu C, Zhang W (2018) Interactive temporal recurrent convolution network for traffic prediction in data centers. IEEE Access 6:5276–5289

    Article  Google Scholar 

  4. Chakrabarti D, Kumar R, Tomkins A (2006) Evolutionary clustering. In: Proceedings of the 12th ACM SIGKDD international conference on knowledge discovery and data mining, ACM, New York, NY, USA, pp 554–560, 10.1145/1150402.1150467

  5. Chen SS, Donoho DL, Saunders MA (2001) Atomic decomposition by basis pursuit. SIAM Rev 43(1):129–159

    Article  MathSciNet  Google Scholar 

  6. Chi Y, Song X, Zhou D, Hino K, Tseng BL (2007a) Evolutionary spectral clustering by incorporating temporal smoothness. In: Proceedings of the 13th ACM SIGKDD international conference on knowledge discovery and data mining, ACM, New York, NY, USA, KDD ’07, pp 153–162, 10.1145/1281192.1281212

  7. Chi Y, Song X, Zhou D, Hino K, Tseng BL (2007b) Evolutionary spectral clustering by incorporating temporal smoothness. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, pp 153–162

  8. Chi Y, Song X, Zhou D, Hino K, Tseng BL (2009) On evolutionary spectral clustering. ACM Trans Knowl Discovery from Data 3(4):17

    Article  Google Scholar 

  9. Derksen H, Ma Y, Hong W, Wright J (2007) Segmentation of multivariate mixed data via lossy coding and compression. In: Proceedings of visual communications and image processing, international society for optics and photonics, vol 6508, p 65080H

  10. Dokeroglu T, Sevinc E, Kucukyilmaz T, Cosar A (2019) A survey on new generation metaheuristic algorithms. Comput Industrial Eng 137:106040

  11. Dyer EL, Sankaranarayanan AC, Baraniuk RG (2013) Greedy feature selection for subspace clustering. J Mach Learning Res 14(1):2487–2517

    MathSciNet  MATH  Google Scholar 

  12. Elhamifar E (2016) High-rank matrix completion and clustering under self-expressive models. In: Advances in Neural Information Processing Systems, pp 73–81

  13. Elhamifar E, Vidal R (2009) Sparse subspace clustering. In: Proceedings of IEEE conference on computer vision and pattern recognition, IEEE, pp 2790–2797

  14. Feng J, Lin Z, Xu H, Yan S (2014) Robust subspace segmentation with block-diagonal prior. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 3818–3825

  15. Gao H, Nie F, Li X, Huang H (2015) Multi-view subspace clustering. In: Proceedings of IEEE international conference on computer vision, pp 4238–4246

  16. Goh A, Vidal R (2007) Segmenting motions of different types by unsupervised manifold clustering. In: Proceedings of IEEE conference on computer vision and pattern recognition, IEEE, pp 1–6

  17. Hashemi A, Vikalo H (2016) Sparse linear regression via generalized orthogonal least-squares. In: Proceedings of IEEE global conference on signal and information processing, IEEE, pp 1305–1309

  18. Hashemi A, Vikalo H (2017) Accelerated sparse subspace clustering. preprint arXiv:171100126

  19. Hashemi A, Vikalo H (2018) Evolutionary self-expressive models for subspace clustering. IEEE J Selected Topics Signal Process 12(6):1534–1546

    Article  Google Scholar 

  20. Heckel R, Bölcskei H (2015) Robust subspace clustering via thresholding. IEEE Trans Inform Theory 61(11):6320–6342. https://doi.org/10.1109/TIT.2015.2472520

    Article  MathSciNet  MATH  Google Scholar 

  21. Hussain K, Mohd Salleh MN, Cheng S, Shi Y (2019) Metaheuristic research: a comprehensive survey. Artif Intell Rev 52(4):2191–2233

    Article  Google Scholar 

  22. Jolliffe IT (2003) Principal component analysis. Technometrics 45(3):276

    MathSciNet  Google Scholar 

  23. Laptev N, Yosinski J, Li LE, Smyl S (2017) Time-series extreme event forecasting with neural networks at Uber. Proc Int Conf Mach Learn 34:1–5

    Google Scholar 

  24. Li CG, You C, Vidal R (2017a) Structured sparse subspace clustering: a joint affinity learning and subspace clustering framework. IEEE Trans Image Process 26(6):2988–3001

    Article  MathSciNet  Google Scholar 

  25. Li H, Xu F, Zhou W, Wang D, Wright JS, Liu Z, Lin Y (2017b) Development of a global gridded argo data set with Barnes successive corrections. J Geophys Res 122(2):866–889

    Article  Google Scholar 

  26. Li S, Fu Y (2015) Learning robust and discriminative subspace with low-rank constraints. IEEE Trans Neural Netw Learn Syst 27(11):2160–2173

    Article  MathSciNet  Google Scholar 

  27. Li S, Li K, Fu Y (2015) Temporal subspace clustering for human motion segmentation. In: Proceedings of the IEEE international conference on computer vision, pp 4453–4461

  28. Li Z, Tang J, He X (2017c) Robust structured nonnegative matrix factorization for image representation. IEEE Trans Neural Netw Learn Syst 29(5):1947–1960

    Article  MathSciNet  Google Scholar 

  29. Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y (2013) Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Analysis Machine Intell 35(1):171–184

    Article  Google Scholar 

  30. Lu C, Feng J, Lin Z, Yan S (2013) Correlation adaptive subspace segmentation by trace LASSO. In: Proceedings of IEEE international conference on computer vision, pp 1345–1352

  31. Ng AY, Jordan MI, Weiss Y (2002) On spectral clustering: analysis and an algorithm. In: Advances in neural information processing systems, pp 849–856

  32. Nie F, Huang H (2016) Subspace clustering via new low-rank model with discrete group structure constraint. In: Proceedings of international joint conference on artificial intelligence, pp 1874–1880

  33. Rosswog J, Ghose K (2008) Detecting and tracking spatio-temporal clusters with adaptive history filtering. In: Proceedings of IEEE international conference on data mining workshops, IEEE, pp 448–457

  34. Sainath TN, Vinyals O, Senior A, Sak H (2015) Convolutional, long short-term memory, fully connected deep neural networks. In: Proceedings of 2015 IEEE international conference on acoustics, speech and signal processing, IEEE, pp 4580–4584

  35. Smith SM, Brady JM (1995) ASSET-2: real-time motion segmentation and shape tracking. IEEE Trans Pattern Anal Machine Intell 17(8):814–820

    Article  Google Scholar 

  36. Swaroop S, Nguyen CV, Bui TD, Turner RE (2019) Improving and understanding variational continual learning. preprint arXiv:190502099

  37. Tron R, Vidal R (2007) A benchmark for the comparison of 3-d motion segmentation algorithms. In: Proceedings of IEEE conference on computer vision and pattern recognition, IEEE, pp 1–8

  38. Tsakiris MC, Vidal R (2018) Theoretical analysis of sparse subspace clustering with missing entries. preprint arXiv:180100393

  39. Vahdat A, Heywood MI (2014) On evolutionary subspace clustering with symbiosis. Evol Intell 6(4):229–256

    Article  Google Scholar 

  40. Vahdat A, Heywood M, Zincir-Heywood N (2010) Bottom-up evolutionary subspace clustering. In: IEEE Congress evol Comput IEEE, pp 1–8

  41. Vahdat A, Heywood MI, Zincir-Heywood AN (2012) Symbiotic evolutionary subspace clustering. In: Proceedings of IEEE congress on evolutionary computation, IEEE, pp 1–8

  42. Vidal R (2011a) Subspace clustering. IEEE Signal Process Mag 28(2):52–68

    Article  Google Scholar 

  43. Vidal R (2011b) Subspace clustering. IEEE Signal Process Mag 28:52–68. https://doi.org/10.1109/MSP.2010.939739

    Article  Google Scholar 

  44. Vidal R, Tron R, Hartley R (2008) Multiframe motion segmentation with missing data using power factorization and GPCA. Int J Comput Vis 79(1):85–105

    Article  Google Scholar 

  45. Wöllmer M, Eyben F, Schuller B, Douglas-Cowie E, Cowie R (2009) Data-driven clustering in emotional space for affect recognition using discriminatively trained lstm networks. In: Proceedings of interspeech, Brighton, UK, pp 1595–1598

  46. Wright J, Ganesh A, Rao S, Peng Y, Ma Y (2009) Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization. In: Advances in neural information processing systems, pp 2080–2088

  47. Xu KS, Kliger M, Hero Iii AO (2014) Adaptive evolutionary clustering. Data Min Knowl Discovery 28(2):304–336

    Article  MathSciNet  Google Scholar 

  48. Yan J, Pollefeys M (2006) A general framework for motion segmentation: Independent, articulated, rigid, non-rigid, degenerate and non-degenerate. In: Proceedings of European conference on computer vision, Springer, pp 94–106

  49. Zhang H, Wang S, Xu X, Chow TWS, Wu QMJ (2018) Tree2vector: learning a vectorial representation for tree-structured data. IEEE Trans Neural Netw Learn Syst 29(11):5304–5318

    Article  MathSciNet  Google Scholar 

  50. Zheng Y, Ma J, Wang L (2017) Consensus of hybrid multi-agent systems. IEEE Trans Neural Netw Learning Syst 29(4):1359–1365

    Article  Google Scholar 

Download references

Author information

Author notes

  1. D. Xu and M. Bai have equal contribution.

Authors and Affiliations

  1. AI in Imaging and Neuroscience Lab, Faculty of Computer Science, The University of California, Los Angeles, Los Angeles, CA, 90024, USA

    Di Xu

  2. Discipline of Business Analytics, The University of Sydney Business School, The University of Sydney, Sydney, NSW, 2006, Australia

    Mingyuan Bai & Junbin Gao

  3. Beijing Advanced Innovation Center for Future Internet Technology, Beijing Municipal Key Lab of Multimedia and Intelligent Software Technology, Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China

    Tianhang Long

Authors
  1. Di Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingyuan Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tianhang Long
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Junbin Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Junbin Gao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, D., Bai, M., Long, T. et al. LSTM-assisted evolutionary self-expressive subspace clustering. Int. J. Mach. Learn. & Cyber. 12, 2777–2793 (2021). https://doi.org/10.1007/s13042-021-01363-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-021-01363-z

Keywords

Search

Navigation