Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Sampling-based Sublinear Low-rank Matrix Arithmetic Framework for Dequantizing Quantum Machine Learning

Published: 27 October 2022 Publication History
  • Get Citation Alerts
  • Abstract

    We present an algorithmic framework for quantum-inspired classical algorithms on close-to-low-rank matrices, generalizing the series of results started by Tang’s breakthrough quantum-inspired algorithm for recommendation systems [STOC’19]. Motivated by quantum linear algebra algorithms and the quantum singular value transformation (SVT) framework of Gilyén et al. [STOC’19], we develop classical algorithms for SVT that run in time independent of input dimension, under suitable quantum-inspired sampling assumptions. Our results give compelling evidence that in the corresponding QRAM data structure input model, quantum SVT does not yield exponential quantum speedups. Since the quantum SVT framework generalizes essentially all known techniques for quantum linear algebra, our results, combined with sampling lemmas from previous work, suffice to generalize all prior results about dequantizing quantum machine learning algorithms. In particular, our classical SVT framework recovers and often improves the dequantization results on recommendation systems, principal component analysis, supervised clustering, support vector machines, low-rank regression, and semidefinite program solving. We also give additional dequantization results on low-rank Hamiltonian simulation and discriminant analysis. Our improvements come from identifying the key feature of the quantum-inspired input model that is at the core of all prior quantum-inspired results: ℓ2-norm sampling can approximate matrix products in time independent of their dimension. We reduce all our main results to this fact, making our exposition concise, self-contained, and intuitive.

    References

    [1]
    Scott Aaronson. 2015. Read the fine print. Nat. Phys. 11, 4 (2015), 291.
    [2]
    Dorit Aharonov, Jordan Cotler, and Xiao-Liang Qi. 2022. Quantum algorithmic measurement. Nat. Commun. 13, 1 (2022), 1–9.
    [3]
    Dorit Aharonov and Amnon Ta-Shma. 2003. Adiabatic quantum state generation and statistical zero knowledge. In Proceedings of the 35th ACM Symposium on the Theory of Computing. ACM, Association for Computing Machinery, New York, NY, 20–29. arXiv:quant-ph/0301023.
    [4]
    Alexei B. Aleksandrov and Vladimir V. Peller. 2010. Operator hölder–zygmund functions. Adv. Math. 224, 3 (2010), 910–966.
    [5]
    Alexei B. Aleksandrov and Vladimir V. Peller. 2011. Estimates of operator moduli of continuity. J. Funct. Anal. 261, 10 (2011), 2741–2796.
    [6]
    Joran van Apeldoorn and András Gilyén. 2019. Improvements in quantum SDP-solving with applications. In Proceedings of the 46th International Colloquium on Automata, Languages, and Programming (ICALP’19). 99:1–99:15.
    [7]
    Joran van Apeldoorn, András Gilyén, Sander Gribling, and Ronald de Wolf. 2020. Quantum SDP-solvers: Better upper and lower bounds. Quantum 4 (2020), 230.
    [8]
    Sanjeev Arora and Satyen Kale. 2016. A combinatorial, primal-dual approach to semidefinite programs. J. ACM 63, 2 (2016), 12:1–12:35.
    [9]
    Juan Miguel Arrazola, Alain Delgado, Bhaskar Roy Bardhan, and Seth Lloyd. 2020. Quantum-inspired algorithms in practice. Quantum 4 (2020), 307.
    [10]
    Peter N. Belhumeur, João P. Hespanha, and David J. Kriegman. 1997. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. IEEE Trans. Pattern Anal. Mach. Intell. 19, 7 (1997), 711–720.
    [11]
    R. Bellman. 1997. Introduction to Matrix Analysis (2nd ed.). Society for Industrial and Applied Mathematics, Philadelphia, PA.
    [12]
    Dominic W. Berry, Andrew M. Childs, and Robin Kothari. 2015. Hamiltonian simulation with nearly optimal dependence on all parameters. In Proceedings of the 56th IEEE Symposium on Foundation of Computer Science (FOCS’15). 792–809.
    [13]
    Rajendra Bhatia. 1997. Matrix Analysis. Graduate Texts in Mathematics, Vol. 169. Springer.
    [14]
    Peter Borwein and Tamás Erdélyi. 1995. Polynomials and Polynomial Inequalities. Graduate Texts in Mathematics, Vol. 161. Springer, New York, NY.
    [15]
    Fernando G. S. L. Brandão, Amir Kalev, Tongyang Li, Cedric Yen-Yu Lin, Krysta M. Svore, and Xiaodi Wu. 2019. Quantum SDP solvers: Large speed-ups, optimality, and applications to quantum learning. In Proceedings of the 46th International Colloquium on Automata, Languages, and Programming (ICALP’19). 27:1–27:14.
    [16]
    Fernando G. S. L. Brandão and Krysta M. Svore. 2017. Quantum speed-ups for solving semidefinite programs. In Proceedings of the 58th IEEE Symposium of Foundatoin of Computer Science (FOCS’17). 415–426.
    [17]
    Daan Camps and Roel Van Beeumen. 2020. Approximate quantum circuit synthesis using block encodings. Phys. Rev. A 102, 5 (2020), 052411.
    [18]
    Shantanav Chakraborty, András Gilyén, and Stacey Jeffery. 2019. The power of block-encoded matrix powers: Improved regression techniques via faster Hamiltonian simulation. In Proceedings of the 46th International Colloquium on Automata, Languages, and Programming (ICALP’19). 33:1–33:14.
    [19]
    Zhihuai Chen, Yinan Li, Xiaoming Sun, Pei Yuan, and Jialin Zhang. 2019. A quantum-inspired classical algorithm for separable non-negative matrix factorization. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. AAAI Press, Palo Alto, CA, 4511–4517.
    [20]
    Nadiia Chepurko, Kenneth L. Clarkson, Lior Horesh, Honghao Lin, and David P. Woodruff. 2020. Quantum-inspired algorithms from randomized numerical linear algebra. In International Conference on Machine Learning. PMLR, 3879–3900.
    [21]
    Nai-Hui Chia, András Gilyén, Han-Hsuan Lin, Seth Lloyd, Ewin Tang, and Chunhao Wang. 2020. Quantum-inspired algorithms for solving low-rank linear equation systems with logarithmic dependence on the dimension. In Proceedings of the 31st International Symposium on Algorithms and Computation (ISAAC’20). 47:1–47:17.
    [22]
    Nai-Hui Chia, Tongyang Li, Han-Hsuan Lin, and Chunhao Wang. 2020. Quantum-inspired sublinear algorithm for solving low-rank semidefinite programming. In Proceedings of the 45th International Symposium on Mathematical Foundations of Computer Science (MFCS’20). 23:1–23:15.
    [23]
    Nai-Hui Chia, Han-Hsuan Lin, and Chunhao Wang. 2018. Quantum-inspired sublinear classical algorithms for solving low-rank linear systems. arXiv:1811.04852. Retrieved from https://arxiv.org/abs/1811.04852.
    [24]
    Carlo Ciliberto, Mark Herbster, Alessandro Davide Ialongo, Massimiliano Pontil, Andrea Rocchetto, Simone Severini, and Leonard Wossnig. 2018. Quantum machine learning: A classical perspective. Proc. Roy. Soc. A: Math. Phys. Eng. Sci. 474, 2209 (January 2018), 20170551.
    [25]
    Kenneth L. Clarkson and David P. Woodruff. 2017. Low-rank approximation and regression in input sparsity time. J. ACM 63, 6, Article 54 (January 2017), 45 pages.
    [26]
    Michael B. Cohen, Cameron Musco, and Christopher Musco. 2017. Input sparsity time low-rank approximation via ridge leverage score sampling. In Proceedings of the 28th Annual ACM-SIAM Symposium on Discrete Algorithms. Society for Industrial and Applied Mathematics, Philadelphia, PA, 1758–1777.
    [27]
    Iris Cong and Luming Duan. 2016. Quantum discriminant analysis for dimensionality reduction and classification. New J. Phys. 18, 7 (July 2016), 073011.
    [28]
    Yogesh Dahiya, Dimitris Konomis, and David P. Woodruff. 2018. An empirical evaluation of sketching for numerical linear algebra. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, Association for Computing Machinery, New York, NY, 1292–1300.
    [29]
    Chen Ding, Tian-Yi Bao, and He-Liang Huang. 2021. Quantum-inspired support vector machine. IEEE Trans. Neural Netw. Learn. Syst. (2021), 1–13.
    [30]
    Petros Drineas, Ravi Kannan, and Michael W. Mahoney. 2006. Fast monte carlo algorithms for matrices I: Approximating matrix multiplication. SIAM J. Comput. 36, 1 (2006), 132–157.
    [31]
    Petros Drineas, Iordanis Kerenidis, and Prabhakar Raghavan. 2002. Competitive recommendation systems. In Proceedings of the 34th ACM Symposium on the Theory of Computing (STOC’02). Association for Computing Machinery, New York, NY, 82–90.
    [32]
    Petros Drineas and Michael W. Mahoney. 2007. A randomized algorithm for a tensor-based generalization of the singular value decomposition. Lin. Algebr. Appl. 420, 2–3 (2007), 553–571.
    [33]
    Petros Drineas, Michael W. Mahoney, and S. Muthukrishnan. 2008. Relative-error CUR matrix decompositions. SIAM J. Matrix Anal. Appl. 30, 2 (January 2008), 844–881.
    [34]
    Yuxuan Du, Min-Hsiu Hsieh, Tongliang Liu, and Dacheng Tao. 2020. Quantum-inspired algorithm for general minimum conical hull problems. Phys. Rev. Res. 2, 3 (2020), 033199.
    [35]
    Vedran Dunjko and Peter Wittek. 2020. A non-review of quantum machine learning: Trends and explorations. Quant. Views 4 (March 2020), 32.
    [36]
    Richard P. Feynman. 1951. An operator calculus having applications in quantum electrodynamics. Phys. Rev. 84, 1 (1951), 108–128.
    [37]
    Richard P. Feynman. 1982. Simulating physics with computers. Int. J. Theor. Phys. 21, 6–7 (1982), 467–488.
    [38]
    Alan Frieze, Ravi Kannan, and Santosh Vempala. 2004. Fast monte-carlo algorithms for finding low-rank approximations. J. ACM 51, 6 (2004), 1025–1041.
    [39]
    Michael I. Gil. 2010. Perturbations of functions of diagonalizable matrices. Electr. J. Lin. Algebr. 20 (2010), 303–313.
    [40]
    András Gilyén, Seth Lloyd, and Ewin Tang. 2018. Quantum-inspired low-rank stochastic regression with logarithmic dependence on the dimension. arXiv:1811.04909. Retrieved from https://arxiv.org/abs/1811.04909.
    [41]
    András Gilyén, Yuan Su, Guang Hao Low, and Nathan Wiebe. 2019. Quantum singular value transformation and beyond: Exponential improvements for quantum matrix arithmetics. In Proceedings of the 51st ACM Symposium of the Theory of Computing (STOC’19). 193–204.
    [42]
    Vittorio Giovannetti, Seth Lloyd, and Lorenzo Maccone. 2008. Quantum random access memory. Phys. Rev. Lett. 100, 16 (2008), 160501.
    [43]
    Lov Grover and Terry Rudolph. 2002. Creating superpositions that correspond to efficiently integrable probability distributions. arXiv:quant-ph/0208112. Retrieved from https://arxiv.org/abs/quant-ph/0208112.
    [44]
    Casper Gyurik, Chris Cade, and Vedran Dunjko. 2020. Towards quantum advantage via topological data analysis. arXiv:2005.02607. Retrieved from https://arxiv.org/abs/2005.02607.
    [45]
    Aram W. Harrow, Avinatan Hassidim, and Seth Lloyd. 2009. Quantum algorithm for linear systems of equations. Phys. Rev. Lett. 103, 15 (2009), 150502.
    [46]
    Elad Hazan, Tomer Koren, and Nati Srebro. 2011. Beating SGD: Learning SVMs in sublinear time. In Advances in Neural Information Processing Systems, Vol. 24, J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. Pereira, and K. Q. Weinberger (Eds.). Curran Associates, Inc., Red Hook, NY, 1233–1241.
    [47]
    Hsin-Yuan Huang, Richard Kueng, and John Preskill. 2021. Information-theoretic bounds on quantum advantage in machine learning. Phys. Rev. Lett. 126, 19 (2021), 190505.
    [48]
    Dhawal Jethwani, François Le Gall, and Sanjay K. Singh. 2020. Quantum-inspired classical algorithms for singular value transformation. In Proceedings of the 45th International Symposium on Mathematical Foundations of Computer Science (MFCS’20). 53:1–53:14.
    [49]
    Satyen Kale. 2007. Efficient Algorithms Using the Multiplicative Weights Update Method. Ph.D. Dissertation. Princeton University.
    [50]
    Ravindran Kannan and Santosh Vempala. 2017. Randomized algorithms in numerical linear algebra. Acta Numer. 26 (2017), 95–135.
    [51]
    Robert Karplus and Julian Schwinger. 1948. A note on saturation in microwave spectroscopy. Phys. Rev. 73, 9 (1948), 1020–1026.
    [52]
    Iordanis Kerenidis and Anupam Prakash. 2017. ITCSQuantum recommendation systems. In Proceedings of the 8th Innovations in Theoretical Computer Science Conference (ITCS). 49:1–49:21.
    [53]
    Iordanis Kerenidis and Anupam Prakash. 2020. Quantum gradient descent for linear systems and least squares. Phys. Rev. A 101, 2 (2020), 022316.
    [54]
    James R. Lee, Prasad Raghavendra, and David Steurer. 2015. Lower bounds on the size of semidefinite programming relaxations. In Proceedings of the 47th Symposium on the Theory of Computing (STOC’15). 567–576.
    [55]
    Seth Lloyd. 1996. Universal quantum simulators. Science 273, 5278 (1996), 1073–1078.
    [56]
    Seth Lloyd, Silvano Garnerone, and Paolo Zanardi. 2016. Quantum algorithms for topological and geometric analysis of data. Nat. Commun. 7 (2016), 10138.
    [57]
    Seth Lloyd, Masoud Mohseni, and Patrick Rebentrost. 2013. Quantum algorithms for supervised and unsupervised machine learning. arXiv:1307.0411. Retrieved from https://arxiv.org/abs/1307.0411.
    [58]
    Seth Lloyd, Masoud Mohseni, and Patrick Rebentrost. 2014. Quantum principal component analysis. Nat. Phys. 10 (2014), 631–633.
    [59]
    Guang Hao Low and Isaac L. Chuang. 2017. Optimal hamiltonian simulation by quantum signal processing. Phys. Rev. Lett. 118, 1 (2017), 010501.
    [60]
    Michael W. Mahoney. 2011. Randomized algorithms for matrices and data. Found. Trends Mach. Learn. 3, 2 (2011), 123–224.
    [61]
    Michael W. Mahoney, Mauro Maggioni, and Petros Drineas. 2008. Tensor-CUR decompositions for tensor-based data. SIAM J. Matrix Anal. Appl. 30, 3 (2008), 957–987.
    [62]
    John M. Martyn, Zane M. Rossi, Andrew K. Tan, and Isaac L. Chuang. 2021. Grand unification of quantum algorithms. Phys. Rev. X 2, 4 (2021), 040203.
    [63]
    Colin McDiarmid. 1989. On the Method of Bounded Differences. Cambridge University Press, Cambridge, UK, 148–188.
    [64]
    Victor Y. Pan and Zhao Q. Chen. 1999. The complexity of the matrix eigenproblem. In Proceedings of the 31st ACM Symposium on the THeory of Computing (STOC’99). 507–516.
    [65]
    Anupam Prakash. 2014. Quantum Algorithms for Linear Algebra and Machine Learning. Ph.D. Dissertation. University of California at Berkeley.
    [66]
    John Preskill. 2018. Quantum computing in the NISQ era and beyond. Quantum 2 (2018), 79.
    [67]
    Patrick Rebentrost and Seth Lloyd. 2018. Quantum computational finance: quantum algorithm for portfolio optimization. arXiv:1811.03975. Retrieved from https://arxiv.org/abs/1811.03975.
    [68]
    Patrick Rebentrost, Masoud Mohseni, and Seth Lloyd. 2014. Quantum support vector machine for big data classification. Phys. Rev. Lett. 113, 13 (2014), 130503.
    [69]
    Patrick Rebentrost, Maria Schuld, Leonard Wossnig, Francesco Petruccione, and Seth Lloyd. 2019. Quantum gradient descent and newton’s method for constrained polynomial optimization. New J. Phys. 21, 7 (2019), 073023.
    [70]
    Mark Rudelson and Roman Vershynin. 2007. Sampling from large matrices: An approach through geometric functional analysis. J ACM 54, 4 (July 2007), 21–es.
    [71]
    Alessandro Rudi, Leonard Wossnig, Carlo Ciliberto, Andrea Rocchetto, Massimiliano Pontil, and Simone Severini. 2020. Approximating hamiltonian dynamics with the Nyström method. Quantum 4 (2020), 234.
    [72]
    Peter W. Shor. 1997. Polynomial-time algorithms for prime factorization and discrete logarithms on a quantum computer. SIAM J. Comput. 26, 5 (1997), 1484–1509.
    [73]
    Zhao Song, David Woodruff, and Huan Zhang. 2016. Sublinear time orthogonal tensor decomposition. In Advances in Neural Information Processing Systems, Vol. 29. Curran Associates, Inc., Red Hook, NY, 793–801. http://papers.nips.cc/paper/6496-sublinear-time-orthogonal-tensor-decomposition.pdf.
    [74]
    Ewin Tang. 2019. A quantum-inspired classical algorithm for recommendation systems. In Proceedings of the 51st ACM Symposium on the Theory of Computing (STOC’19). 217–228.
    [75]
    Ewin Tang. 2021. Quantum principal component analysis only achieves an exponential speedup because of its state preparation assumptions. Phys. Rev. Lett. 127, 6 (2021), 060503.
    [76]
    Terence Tao. 2010. 254A, Notes 3a: Eigenvalues and Sums of Hermitian Matrices. Retrieved from https://terrytao.wordpress.com/2010/01/12/254a-notes-3a-eigenvalues-and-sums-of-hermitian-matrices/.
    [77]
    Maarten Van den Nest. 2011. Simulating quantum computers with probabilistic methods. Quant. Inf. Comput. 11, 9&10 (2011), 784–812. arXiv:0911.1624.
    [78]
    Michael D. Vose. 1991. A linear algorithm for generating random numbers with a given distribution. IEEE Trans. Softw. Eng. 17, 9 (1991), 972–975.
    [79]
    Max Welling. 2009. Fisher Linear Discriminant Analysis. Retrieved from https://www.ics.uci.edu/welling/teaching/273ASpring09/Fisher-LDA.pdf.
    [80]
    David P. Woodruff. 2014. Sketching as a tool for numerical linear algebra. Found. Trends Theor. Comput. Sci. 10, 1–2 (2014), 1–157.
    [81]
    Leonard Wossnig, Zhikuan Zhao, and Anupam Prakash. 2018. Quantum linear system algorithm for dense matrices. Phys. Rev. Lett. 120, 5 (2018), 050502.
    [82]
    Hayata Yamasaki, Sathyawageeswar Subramanian, Sho Sonoda, and Masato Koashi. 2020. Learning with optimized random features: Exponential speedup by quantum machine learning without sparsity and low-rank assumptions. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (Eds.). Vol. 33. Curran Associates, Inc., Red Hook, NY, 13674–13687.
    [83]
    Zhikuan Zhao, Jack K. Fitzsimons, and Joseph F. Fitzsimons. 2019. Quantum-assisted gaussian process regression. Phys. Rev. A 99, 5 (2019), 052331.

    Cited By

    View all
    • (2024)Fixed-precision randomized quaternion singular value decomposition algorithm for low-rank quaternion matrix approximationsNeurocomputing10.1016/j.neucom.2024.127490580(127490)Online publication date: May-2024
    • (2024)Quantum-inspired algorithm for truncated total least squares solutionJournal of Computational and Applied Mathematics10.1016/j.cam.2024.116042451(116042)Online publication date: Dec-2024
    • (2023)Higher-Order Topological Kernels via Quantum Computation2023 IEEE International Conference on Quantum Computing and Engineering (QCE)10.1109/QCE57702.2023.00076(621-629)Online publication date: 17-Sep-2023
    • Show More Cited By

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image Journal of the ACM
    Journal of the ACM  Volume 69, Issue 5
    October 2022
    420 pages
    ISSN:0004-5411
    EISSN:1557-735X
    DOI:10.1145/3563903
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 27 October 2022
    Online AM: 10 August 2022
    Accepted: 07 June 2022
    Revised: 20 May 2022
    Received: 20 August 2020
    Published in JACM Volume 69, Issue 5

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Quantum-inspired classical algorithms
    2. theoretical machine learning
    3. quantum computing
    4. sublinear algorithms

    Qualifiers

    • Research-article
    • Refereed

    Funding Sources

    • Scott Aaronson’s Vannevar Bush Faculty Fellowship from the US Department of Defense
    • Samsung Electronics Co., Ltd., for the project “The Computational Power of Sampling on Quantum Computers”
    • Institute for Quantum Information and Matter, an NSF Physics Frontiers Center
    • EU’s Horizon 2020 Marie Skłodowska-Curie program
    • IBM PhD Fellowship, QISE-NET Triplet Award
    • U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, Quantum Algorithms Teams program
    • National Science Foundation Graduate Research Fellowship Program

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)369
    • Downloads (Last 6 weeks)18

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Fixed-precision randomized quaternion singular value decomposition algorithm for low-rank quaternion matrix approximationsNeurocomputing10.1016/j.neucom.2024.127490580(127490)Online publication date: May-2024
    • (2024)Quantum-inspired algorithm for truncated total least squares solutionJournal of Computational and Applied Mathematics10.1016/j.cam.2024.116042451(116042)Online publication date: Dec-2024
    • (2023)Higher-Order Topological Kernels via Quantum Computation2023 IEEE International Conference on Quantum Computing and Engineering (QCE)10.1109/QCE57702.2023.00076(621-629)Online publication date: 17-Sep-2023
    • (2023)The unified effect of data encoding, ansatz expressibility and entanglement on the trainability of HQNNsInternational Journal of Parallel, Emergent and Distributed Systems10.1080/17445760.2023.223116338:5(362-400)Online publication date: 17-Jul-2023

    View Options

    Get Access

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media