Article

Tensor-CUR decompositions for tensor-based data

Authors:

Michael W. Mahoney,

Mauro Maggioni,

Petros DrineasAuthors Info & Claims

KDD '06: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining

Pages 327 - 336

https://doi.org/10.1145/1150402.1150440

Published: 20 August 2006 Publication History

Abstract

Motivated by numerous applications in which the data may be modeled by a variable subscripted by three or more indices, we develop a tensor-based extension of the matrix CUR decomposition. The tensor-CUR decomposition is most relevant as a data analysis tool when the data consist of one mode that is qualitatively different than the others. In this case, the tensor-CUR decomposition approximately expresses the original data tensor in terms of a basis consisting of underlying subtensors that are actual data elements and thus that have natural interpretation in terms ofthe processes generating the data. In order to demonstrate the general applicability of this tensor decomposition, we apply it to problems in two diverse domains of data analysis: hyperspectral medical image analysis and consumer recommendation system analysis. In the hyperspectral data application, the tensor-CUR decomposition is used to compress the data, and we show that classification quality is not substantially reduced even after substantial data compression. In the recommendation system application, the tensor-CUR decomposition is used to reconstruct missing entries in a user-product-product preference tensor, and we show that high quality recommendations can be made on the basis of a small number of basis users and a small number of product-product comparisons from a new user.

References

[1]

G. Adomavicius and A. Tuzhilin. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17:734--749, 2005.

Digital Library

[2]

Y. Azar, A. Fiat, A. R. Karlin, F. McSherry, and J. Saia. Spectral analysis of data. In Proceedings of the 33rd Annual ACM Symposium on Theory of Computing, pages 619--626, 2001.

Digital Library

[3]

M. W. Berry, S. A. Pulatova, and G. W. Stewart. Computing sparse reduced-rank approximations to sparse matrices. Technical Report UMIACS TR-2004-32 CMSC TR-4589, University of Maryland, College Park, MD, 2004.

[4]

D. Billsus and M. J. Pazzani. Learning collaborative information filters. In Proceedings of the 15th International Conference on Machine Learning, pages 46--54, 1998.

Digital Library

[5]

J. Breese, D. Heckerman, and C. Kadie. Empirical analysis of predictive algorithms for collaborative filtering. In Proceedings of the 14th Annual Conference on Uncertainty in Artificial Intelligence, pages 43--52, 1998.

Digital Library

[6]

J. D. Carroll and J. J. Chang. Analysis of individual differences in multidimensional scaling via an N-way generalization of "Eckart- Young" decomposition. Psychometrika, 35(3):283--319, 1970.

[7]

W. W. Cohen, R. E. Schapire, and Y. Singer. Learning to order things. In Annual Advances in Neural Information Processing Systems 10: Proceedings of the 1997 Conference, pages 451--457, 1998.

Digital Library

[8]

P. Drineas, R. Kannan, and M. W. Mahoney. Fast Monte Carlo algorithms for matrices I: Approximating matrix multiplication. To appear in: SIAM Journal on Computing.

Digital Library

[9]

P. Drineas, R. Kannan, and M. W. Mahoney. Fast Monte Carlo algorithms for matrices II: Computing a low-rank approximation to a matrix. To appear in: SIAM Journal on Computing.

Digital Library

[10]

P. Drineas, R. Kannan, and M. W. Mahoney. Fast Monte Carlo algorithms for matrices III: Computing a compressed approximate matrix decomposition. To appear in: SIAM Journal on Computing.

Digital Library

[11]

P. Drineas, I. Kerenidis, and P. Raghavan. Competitive recommendation systems. In Proceedings of the 34th Annual ACM Symposium on Theory of Computing, pages 82--90, 2002.

Digital Library

[12]

P. Drineas and M. W. Mahoney. A randomized algorithm for a tensor-based generalization of the Singular Value Decomposition. To appear in: Linear Algebra and its Applications.

[13]

P. Drineas and M. W. Mahoney. On the Nyström method for approximating a Gram matrix for improved kernel-based learning. Journal of Machine Learning Research, 6:2153--2175, 2005.

Digital Library

[14]

Y. Freund, R. Iyer, R. E. Schapire, and Y Singer. An efficient boosting algorithm for combining preferences. Journal of Machine Learning Research, 4:933--969, 2003.

Digital Library

[15]

K. Goldberg, T. Roeder, D. Gupta, and C. Perkins. Eigentaste: A constant time collaborative filtering algorithm. Information Retrieval, 4:133--151, 2001.

Digital Library

[16]

G. H. Golub and C. F. Van Loan. Matrix Computations. Johns Hopkins University Press, Baltimore, 1989.

[17]

S. A. Goreinov and E. E. Tyrtyshnikov. The maximum-volume concept in approximation by low-rank matrices. Contemporary Mathematics, 280:47--51, 2001.

[18]

S. A. Goreinov, E. E. Tyrtyshnikov, and N. L. Zamarashkin. A theory of pseudoskeleton approximations. Linear Algebra and Its Applications, 261:1--21, 1997.

[19]

R. A. Harshman and M. E. Lundy. The PARAFAC model for three-way factor analysis and multidimensional scaling. In H. G. Law, C. W. Snyder Jr., J. Hattie, and R. P. McDonald, editors, Research Methods for Multimode Data Analysis, pages 122--215. Praeger, 1984.

[20]

J. Håstad Tensor rank is NP-complete. Journal of Algorithms, 11(2):644--654, 1990.

Digital Library

[21]

R. Jin, L. Si, and C. X. Zhai. Preference-based graphic models for collaborative filtering. In Proceedings of the 19th Conference in Uncertainty in Artificial Intelligence, pages 329--336, 2003.

Digital Library

[22]

R. Jin, L. Si, C. X. Zhai, and J. Callan. Collaborative filtering with decoupled models for preferences and ratings. In Proceedings of the 12th International Conference on Information and Knowledge Management, pages 309--316, 2003.

Digital Library

[23]

J. Kleinberg and M. Sandler. Using mixture models for collaborative filtering. In Proceedings of the 36th Annual ACM Symposium on Theory of Computing, pages 569--578, 2004.

Digital Library

[24]

P. M. Kroonenberg and J. De Leeuw. Principal component analysis of three-mode data by means of alternating least squares algorithms. Psychometrika, 45(1):69--97, 1980.

[25]

J. B. Kruskal. Three-way arrays: Rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics. Linear Algebra and Its Applications, 18:95--138, 1977.

[26]

J. B. Kruskal. Rank, decomposition, and uniqueness for 3-way and N-way arrays. In R. Coppi and S. Bolasco, editors, Multiway Data Analysis, pages 7--18. Elesvier Science Publishers, 1989.

Digital Library

[27]

R. Kumar, P. Raghavan, S. Rajagopalan, and A. Tomkins. Recommendation systems: a probabilistic analysis. In Proceedings of the 39th Annual IEEE Symposium on Foundations of Computer Science, pages 664--673, 1998.

Digital Library

[28]

L. De Lathauwer, B. De Moor, and J. Vandewalle. A multilinear singular value decomposition. SIAM Journal on Matrix Analysis and Applications, 21(4):1253--1278, 2000.

Digital Library

[29]

L. De Lathauwer, B. De Moor, and J. Vandewalle. On the best rank-1 and rank-(R1, R2, .., RN)approximation of higher-order tensors. SIAM Journal on Matrix Analysis and Applications, 21(4):1324--1342, 2000.

Digital Library

[30]

D. Leibovici and R. Sabatier. A singular value decomposition of a k-way array for a principal component analysis of multiway data, PTA-k. Linear Algebra and Its Applications, 269:307--329, 1998.

[31]

L.-H. Lim and G. H. Golub. Tensors for numerical multilinear algebra: ranks and basic decompositions. Technical Report 05-02, Stanford University SCCM, Stanford, CA, April 2005.

[32]

G. Linden, B. Smith, and J. York. Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet Computing, 7:76--80, 2003.

Digital Library

[33]

M. Maggioni, G. L. Davis, F. J. Warner, F. B. Geshwind, A. C. Coppi, R. A. DeVerse,and R. R. Coifman. Algorithms from signal and data processing applied to hyperspectral analysis: Discriminating normal and malignant microarray colon tissue sections using a novel digital mirror device system. Manuscript, 2006.

[34]

M. Maggioni, G. L. Davis, F. J. Warner, F. B. Geshwind, A. C. Coppi, R. A. DeVerse,and R. R. Coifman. Hyperspectral microscopic analysis of normal, benign and carcinoma microarray tissue sections. Manuscript, 2006.

[35]

P. Resnick and H. R. Varian. Recommender systems. Communications of the ACM, 40(3):56--58, 1997.

Digital Library

[36]

B. Sarwar, G. Karypis, J. Konstan, and J. Riedl. Application of dimensionality reduction in recommender system - a case study. In Proceedings of the WebKDD 2000 Workshop, pages 000--000, 2000.

[37]

G. W. Stewart. Four algorithms for the efficient computation of truncated QR approximations to a sparse matrix. Numerische Mathematik, 83:313--323, 1999.

[38]

G. W. Stewart. Error analysis of the quasi-Gram-Schmidt algorithm. Technical Report UMIACS TR-2004-17 CMSC TR-4572, University of Maryland, College Park, MD, 2004.

[39]

L. R. Tucker. Some mathematical notes on three-mode factor analysis. Psychometrika, 31(3):279--311, 1966.

[40]

C. K. I. Williams and M. Seeger. Using the Nyström method to speed up kernel machines. In Annual Advances in Neural Information Processing Systems 13: Proceedings of the 2000 Conference, pages 682--688, 2001.

Cited By

Salut MAnderson D(2023)Tensor Robust CUR for Compression and Denoising of Hyperspectral DataIEEE Access10.1109/ACCESS.2023.329763011(77492-77505)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3297630
Deshpande APratap R(2023)One-Pass Additive-Error Subset Selection for $$\ell _{p}$$ Subspace Approximation and (k, p)-ClusteringAlgorithmica10.1007/s00453-023-01124-085:10(3144-3167)Online publication date: 11-May-2023
https://doi.org/10.1007/s00453-023-01124-0
Qin ZLidiak AGong ZTang GWakin MZhu ZKoyejo SMohamed SAgarwal ABelgrave DCho KOh A(2022)Error analysis of tensor-train cross approximationProceedings of the 36th International Conference on Neural Information Processing Systems10.5555/3600270.3601305(14236-14249)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.5555/3600270.3601305
Show More Cited By

Index Terms

Tensor-CUR decompositions for tensor-based data
1. Information systems
2. Software and its engineering
  1. Software notations and tools
    1. General programming languages
      1. Language features
        Data types and structures

Recommendations

Mode-wise tensor decompositions: multi-dimensional generalizations of CUR decompositions

Low rank tensor approximation is a fundamental tool in modern machine learning and data science. In this paper, we study the characterization, perturbation analysis, and an efficient sampling strategy for two primary tensor CUR approximations, namely ...
Tensor-CUR Decompositions for Tensor-Based Data

Motivated by numerous applications in which the data may be modeled by a variable subscripted by three or more indices, we develop a tensor-based extension of the matrix CUR decomposition. The tensor-CUR decomposition is most relevant as a data analysis ...
Tensor Decompositions and Applications

This survey provides an overview of higher-order tensor decompositions, their applications, and available software. A tensor is a multidimensional or $N$-way array. Decompositions of higher-order tensors (i.e., $N$-way arrays with $N \geq 3$) have ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '06: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining

August 2006

986 pages

ISBN:1595933395

DOI:10.1145/1150402

Conference Chair:
Tina Eliassi-Rad
LLNL
,
General Chair:
Lyle Ungar
University of Pennsylvania
,
Program Chairs:
Mark Craven
University of Wisconsin
,
Dimitrios Gunopulos
University of California, Riverside

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 August 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

KDD06

Sponsor:

KDD06: The 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

August 20 - 23, 2006

PA, Philadelphia, USA

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '24

Sponsor:
sigkdd
sigkdd

The 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

August 25 - 29, 2024

Barcelona , Spain

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
841
Total Downloads

Downloads (Last 12 months)62
Downloads (Last 6 weeks)8

Reflects downloads up to

Other Metrics

View Author Metrics

Citations

Cited By

Salut MAnderson D(2023)Tensor Robust CUR for Compression and Denoising of Hyperspectral DataIEEE Access10.1109/ACCESS.2023.329763011(77492-77505)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3297630
Deshpande APratap R(2023)One-Pass Additive-Error Subset Selection for $$\ell _{p}$$ Subspace Approximation and (k, p)-ClusteringAlgorithmica10.1007/s00453-023-01124-085:10(3144-3167)Online publication date: 11-May-2023
https://doi.org/10.1007/s00453-023-01124-0
Qin ZLidiak AGong ZTang GWakin MZhu ZKoyejo SMohamed SAgarwal ABelgrave DCho KOh A(2022)Error analysis of tensor-train cross approximationProceedings of the 36th International Conference on Neural Information Processing Systems10.5555/3600270.3601305(14236-14249)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.5555/3600270.3601305
Ida YKanai SFujiwara YIwata TTakeuchi KKashima HDaumé HSingh A(2020)Fast deterministic CUR matrix decomposition with accuracy assuranceProceedings of the 37th International Conference on Machine Learning10.5555/3524938.3525365(4594-4603)Online publication date: 13-Jul-2020
https://dl.acm.org/doi/10.5555/3524938.3525365
Traoré ABérar MRakotomamonjy AWallach HLarochelle HBeygelzimer Ad'Alché-Buc FFox E(2019)SingleshotProceedings of the 33rd International Conference on Neural Information Processing Systems10.5555/3454287.3454853(6304-6315)Online publication date: 8-Dec-2019
https://dl.acm.org/doi/10.5555/3454287.3454853
Baladevi CHarikumar S(2018)Semantic Representation of Documents Based on Matrix Decomposition2018 International Conference on Data Science and Engineering (ICDSE)10.1109/ICDSE.2018.8527824(1-6)Online publication date: Aug-2018
https://doi.org/10.1109/ICDSE.2018.8527824
Kim MCandan K(2016)Decomposition-by-normalization (DBN)Data Mining and Knowledge Discovery10.1007/s10618-015-0401-630:1(1-46)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.1007/s10618-015-0401-6
Papalexakis EMitchell TSidiropoulos NFaloutsos CTalukdar PMurphy B(2016)Turbo-SMTStatistical Analysis and Data Mining10.1002/sam.113159:4(269-290)Online publication date: 1-Aug-2016
https://dl.acm.org/doi/10.1002/sam.11315
Papalexakis EFaloutsos CSidiropoulos N(2015)ParCubeACM Transactions on Knowledge Discovery from Data10.1145/272998010:1(1-25)Online publication date: 22-Jul-2015
https://dl.acm.org/doi/10.1145/2729980
Nguyen NDrineas PTran T(2015)Tensor sparsification via a bound on the spectral norm of random tensors: Algorithm 1.Information and Inference10.1093/imaiai/iav0044:3(195-229)Online publication date: 12-May-2015
https://doi.org/10.1093/imaiai/iav004
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents