Incremental algorithms for truncated higher-order singular value decompositions
Abstract
We develop and study incremental algorithms for truncated higher-order singular value decompositions. By combining the SVD updating and different truncated higher-order singular value decompositions, two incremental algorithms are proposed. Not only the factor matrices but also the core tensor are updated in an incremental style. The costs of these algorithms are compared and the approximation errors are analyzed. Numerical results demonstrate that the proposed incremental algorithms have advantages in online computation.
References
[1]
Bader, B.W., Kolda, T.G., et al.: MATLAB Tensor Toolbox Version 3.0-dev (2017). https://www.tensortoolbox.org
[2]
Brand M Fast low-rank modifications of the thin singular value decomposition Linear Algebra Appl. 2006 415 1 20-30
[3]
Bunch JR and Nielsen CP Updating the singular value decomposition Numer. Math. 1978 31 2 111-129
[4]
Carroll JD and Chang JJ Analysis of individual differences in multidimensional scaling via an N-way generalization of “Eckart-Young” decomposition Psychometrika 1970 35 3 283-319
[5]
Cheng Y, Roemer F, Khatib O, and Haardt M Tensor subspace Tracking via Kronecker structured projections (TeTraKron) for time-varying multidimensional harmonic retrieval EURASIP J. Adv. Signal Process. 2014 2014 1 1-14
[6]
Cocosco, C.A., Kollokian, V., Kwan, R.K.S., Pike, G.B., Evans, A.C.: Brainweb: online interface to a 3D MRI simulated brain database. In: NeuroImage. Citeseer (1997)
[7]
De Lathauwer L, De Moor B, and Vandewalle J A multilinear singular value decomposition SIAM J. Matrix Anal. Appl. 2000 21 4 1253-1278
[8]
De Lathauwer L, De Moor B, and Vandewalle J On the best rank-1 and rank- approximation of higher-order tensors SIAM J. Matrix Anal. Appl. 2000 21 4 1324-1342
[9]
De Silva V and Lim LH Tensor rank and the ill-posedness of the best low-rank approximation problem SIAM J. Matrix Anal. Appl. 2008 30 3 1084-1127
[10]
Grasedyck L Hierarchical singular value decomposition of tensors SIAM J. Matrix Anal. Appl. 2010 31 4 2029-2054
[11]
Gu, M., Eisenstat, S.C.: A stable and fast algorithm for updating the singular value decomposition. Research Report YALEU/DCS/RR-966, Dept. of Computer Science, Yale University (1993)
[12]
Gu M and Eisenstat SC Downdating the singular value decomposition SIAM J. Matrix Anal. Appl. 1995 16 3 793-810
[13]
Hackbusch W and Kühn S A new scheme for the tensor representation J. Fourier Anal. Appl. 2009 15 5 706-722
[14]
Harshman, R.A.: Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multimodal factor analysis. UCLA Working Papers in Phonetics, vol. 16, pp. 1–84. University Microfilms, Ann Arbor, Michigan, No. 10,085 (1970).
[15]
Hu W, Li X, Zhang X, Shi X, Maybank S, and Zhang Z Incremental tensor subspace learning and its applications to foreground segmentation and tracking Int. J. Comput. Vis. 2011 91 3 303-327
[16]
Kilmer ME, Braman K, Hao N, and Hoover RC Third-order tensors as operators on matrices: a theoretical and computational framework with applications in imaging SIAM J. Matrix Anal. Appl. 2013 34 1 148-172
[17]
Kilmer ME and Martin CD Factorization strategies for third-order tensors Linear Algebra Appl. 2011 435 3 641-658
[18]
Kolda TG and Bader BW Tensor decompositions and applications SIAM Rev. 2009 51 3 455-500
[19]
Letourneau, P.D., Baskaran, M., Henretty, T., Ezick, J., Lethin, R.: Computationally efficient CP tensor decomposition update framework for emerging component discovery in streaming data. In: 2018 IEEE High Performance Extreme Computing Conference (HPEC), pp. 1–8. IEEE (2018)
[20]
Ma, X., Schonfeld, D., Khokhar, A.: Dynamic updating and downdating matrix SVD and tensor HOSVD for adaptive indexing and retrieval of motion trajectories. In: 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1129–1132. IEEE (2009)
[21]
Minster R, Saibaba AK, and Kilmer ME Randomized algorithms for low-rank tensor decompositions in the Tucker format SIAM J. Math. Data Sci. 2020 2 1 189-215
[22]
Moonen M, Van Dooren P, and Vandewalle J A singular value decomposition updating algorithm for subspace tracking SIAM J. Matrix Anal. Appl. 1992 13 4 1015-1038
[23]
Nion D and Sidiropoulos ND Adaptive algorithms to track the PARAFAC decomposition of a third-order tensor IEEE Trans. Signal Process. 2009 57 6 2299-2310
[24]
Oseledets IV Tensor-train decomposition SIAM J. Sci. Comput. 2011 33 5 2295-2317
[25]
Pasricha, R., Gujral, E., Papalexakis, E.E.: Identifying and alleviating concept drift in streaming tensor decomposition. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 327–343. Springer (2018)
[26]
Rodriguez P and Wohlberg B Incremental principal component pursuit for video background modeling J. Math. Imaging Vis. 2016 55 1 1-18
[27]
Ross DA, Lim J, Lin RS, and Yang MH Incremental learning for robust visual tracking Int. J. Comput. Vis. 2008 77 1–3 125-141
[28]
Seeling P and Reisslein M Video transport evaluation with H. 264 video traces IEEE Commun. Surv. Tutor. 2011 14 4 1142-1165
[29]
Smith, S., Huang, K., Sidiropoulos, N.D., Karypis, G.: Streaming tensor factorization for infinite data sources. In: Proceedings of the 2018 SIAM International Conference on Data Mining, pp. 81–89. SIAM (2018)
[30]
Sobral, A., Baker, C.G., Bouwmans, T., Zahzah, E.: Incremental and multi-feature tensor subspace learning applied for background modeling and subtraction. In: International Conference on Image Analysis and Recognition, pp. 94–103 (2014)
[31]
Sun, J., Tao, D., Faloutsos, C.: Beyond streams and graphs: dynamic tensor analysis. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 374–383. ACM (2006)
[32]
Sun J, Tao D, Papadimitriou S, Yu PS, and Faloutsos C Incremental tensor analysis: theory and applications ACM Trans. Knowl. Discov. Data (TKDD) 2008 2 3 11
[33]
Sun Y, Guo Y, Luo C, Tropp J, and Udell M Low-rank Tucker approximation of a tensor from streaming data SIAM J. Math. Data Sci. 2020 2 4 1123-1150
[34]
Tucker L Some mathematical notes on three-mode factor analysis Psychometrika 1966 31 3 279-311
[35]
Vandecappelle, M., De Lathauwer, L.: Low multilinear rank updating. In: 2019 53rd Asilomar Conference on Signals, Systems, and Computers, pp. 437–441. IEEE (2019)
[36]
Vandecappelle, M., Vervliet, N., De Lathauwer, L.: Nonlinear least squares updating of the canonical polyadic decomposition. In: 2017 25th European Signal Processing Conference (EUSIPCO), pp. 663–667. IEEE (2017)
[37]
Vannieuwenhoven N, Vandebril R, and Meerbergen K A new truncation strategy for the higher-order singular value decomposition SIAM J. Sci. Comput. 2012 34 2 A1027-A1052
[38]
Zeng C and Ng MK Incremental CP tensor decomposition by alternating minimization method SIAM J. Matrix Anal. Appl. 2021 42 2 832-858
[39]
Zhou, S., Vinh, N.X., Bailey, J., Jia, Y., Davidson, I.: Accelerating online CP decompositions for higher order tensors. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1375–1384. ACM (2016)
Recommendations
Hermitian Tensor Decompositions
Hermitian tensors are generalizations of Hermitian matrices, but they have very different properties. Every complex Hermitian tensor is a sum of complex Hermitian rank-1 tensors. However, this is not true for the real case. We study basic properties for ...
The QLP Approximation to the Singular Value Decomposition
In this paper we introduce a new decomposition called the pivoted QLP decomposition. It is computed by applying pivoted orthogonal triangularization to the columns of the matrix X in question to get an upper triangular factor R and then applying the same ...
A Randomized Singular Value Decomposition for Third-Order Oriented Tensors
AbstractThe oriented singular value decomposition (O-SVD) proposed by Zeng and Ng provides a hybrid approach to the t-product-based third-order tensor singular value decomposition with the transformation matrix being a factor matrix of the higher-order ...
Comments
Information & Contributors
Information
Published In
© The Author(s), under exclusive licence to Springer Nature B.V. 2024. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
Publisher
BIT Computer Science and Numerical Mathematics
United States
Publication History
Published: 08 January 2024
Accepted: 03 December 2023
Received: 11 March 2023
Author Tags
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Natural Science Foundation of China
- HKRGC GRF
- CRF
- Joint NSFC and RGC
- Natural Science Foundation of Sichuan Province
- Fundamental Research Funds for the Central Universities, Nankai University
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 05 Feb 2025