- Author:
- Nathan Srebro,
- Supervisor:
- Tommi S. Jaakkola
Matrices that can be factored into a product of two simpler matrices can serve as a useful and often natural model in the analysis of tabulated or high-dimensional data. Models based on matrix factorization (Factor Analysis, PCA) have been extensively used in statistical analysis and machine learning for over a century, with many new formulations and models suggested in recent years (Latent Semantic Indexing, Aspect Models, Probabilistic PCA, Exponential PCA, Non-Negative Matrix Factorization and others). In this thesis we address several issues related to learning with matrix factorizations: we study the asymptotic behavior and generalization ability of existing methods, suggest new optimization methods, and present a novel maximum-margin high-dimensional matrix factorization formulation. (Copies available exclusively from MIT Libraries, Rm. 14-0551, Cambridge, MA 02139-4307. Ph. 617-253-5668; Fax 617-253-1690.)
Cited By
- Hazan E, Kalai A, Kanade V, Mohri C and Sun Y Partial matrix completion Proceedings of the 37th International Conference on Neural Information Processing Systems, (30134-30145)
- Kadri H, Ayache S, Huusari R, Rakotomamonjy A and Ralaivola L Partial trace regression and low-rank Kraus decomposition Proceedings of the 37th International Conference on Machine Learning, (5031-5041)
- Cai C, Poor H and Chen Y Uncertainty quantification for nonconvex tensor completion Proceedings of the 37th International Conference on Machine Learning, (1271-1282)
- Yu Y, Peng J and Yue S (2019). A new nonconvex approach to low-rank matrix completion with application to image inpainting, Multidimensional Systems and Signal Processing, 30:1, (145-174), Online publication date: 1-Jan-2019.
- Min J, Jin K, Unser M and Ye J (2018). Grid-Free Localization Algorithm Using Low-Rank Hankel Matrix for Super-Resolution Microscopy, IEEE Transactions on Image Processing, 27:10, (4771-4786), Online publication date: 1-Oct-2018.
- Huang S and Wolkowicz H (2018). Low-rank matrix completion using nuclear norm minimization and facial reduction, Journal of Global Optimization, 72:1, (5-26), Online publication date: 1-Sep-2018.
- Hutchinson B, Ostendorf M and Fazel M (2015). A sparse plus low-rank exponential language model for limited resource scenarios, IEEE/ACM Transactions on Audio, Speech and Language Processing, 23:3, (494-504), Online publication date: 1-Mar-2015.
Yue Y, Wang C, El-Arini K and Guestrin C Personalized collaborative clustering Proceedings of the 23rd international conference on World wide web, (75-84)- Huang B, Ma S and Goldfarb D (2013). Accelerated Linearized Bregman Method, Journal of Scientific Computing, 54:2-3, (428-453), Online publication date: 1-Feb-2013.
- Grosse R, Salakhutdinov R, Freeman W and Tenenbaum J Exploiting compositionality to explore a large space of model structures Proceedings of the Twenty-Eighth Conference on Uncertainty in Artificial Intelligence, (306-315)
- Narita A, Hayashi K, Tomioka R and Kashima H Tensor factorization using auxiliary information Proceedings of the 2011 European conference on Machine learning and knowledge discovery in databases - Volume Part II, (501-516)
- Narita A, Hayashi K, Tomioka R and Kashima H Tensor factorization using auxiliary information Proceedings of the 2011th European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part II, (501-516)
- Shi X and Yu P (2011). Limitations of matrix completion via trace norm minimization, ACM SIGKDD Explorations Newsletter, 12:2, (16-20), Online publication date: 31-Mar-2011.
- Raymond R and Kashima H Fast and scalable algorithms for semi-supervised link prediction on static and dynamic graphs Proceedings of the 2010 European conference on Machine learning and knowledge discovery in databases: Part III, (131-147)
- Raymond R and Kashima H Fast and scalable algorithms for semi-supervised link prediction on static and dynamic graphs Proceedings of the 2010 European conference on Machine learning and knowledge discovery in databases: Part III, (131-147)
- Raymond R and Kashima H Fast and scalable algorithms for semi-supervised link prediction on static and dynamic graphs Proceedings of the 2010th European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part III, (131-147)
- Seldin Y and Tishby N (2010). PAC-Bayesian Analysis of Co-clustering and Beyond, The Journal of Machine Learning Research, 11, (3595-3646), Online publication date: 1-Mar-2010.
- Seldin Y and Tishby N Multi-classification by categorical features via clustering Proceedings of the 25th international conference on Machine learning, (920-927)
- Salakhutdinov R, Mnih A and Hinton G Restricted Boltzmann machines for collaborative filtering Proceedings of the 24th international conference on Machine learning, (791-798)
- Pereira F and Gordon G The support vector decomposition machine Proceedings of the 23rd international conference on Machine learning, (689-696)
- Srebro N and Shraibman A Rank, trace-norm and max-norm Proceedings of the 18th annual conference on Learning Theory, (545-560)
- Tian D, Mansour H and Vetro A Depth-weighted group-wise principal component analysis for video foreground/background separation 2015 IEEE International Conference on Image Processing (ICIP), (3230-3234)
Recommendations
Localization of Matrix Factorizations
Matrices with off-diagonal decay appear in a variety of fields in mathematics and in numerous applications, such as signal processing, statistics, communications engineering, condensed matter physics, and quantum chemistry. Numerical algorithms dealing ...
Convex and Semi-Nonnegative Matrix Factorizations
We present several new variations on the theme of nonnegative matrix factorization (NMF). Considering factorizations of the form X=FG^T, we focus on algorithms in which G is restricted to containing nonnegative entries, but allowing the data matrix X to ...