research-article

Public Access

High-Dimensional Similarity Search with Quantum-Assisted Variational Autoencoder

Authors:

Ramakrishna Nemani,

Eleanor RieffelAuthors Info & Claims

KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

Pages 956 - 964

https://doi.org/10.1145/3394486.3403138

Published: 20 August 2020 Publication History

Abstract

Recent progress in quantum algorithms and hardware indicates the potential importance of quantum computing in the near future. However, finding suitable application areas remains an active area of research. Quantum machine learning is touted as a potential approach to demonstrate quantum advantage within both the gate-model and the adiabatic schemes. For instance, the Quantum-assisted Variational Autoencoder (QVAE) has been proposed as a quantum enhancement to the discrete VAE. We extend on previous work and study the real-world applicability of a QVAE by presenting a proof-of-concept for similarity search in large-scale high-dimensional datasets. While exact and fast similarity search algorithms are available for low dimensional datasets, scaling to high-dimensional data is non-trivial. We show how to construct a space-efficient search index based on the latent space representation of a QVAE. Our experiments show a correlation between the Hamming distance in the embedded space and the Euclidean distance in the original space on the Moderate Resolution Imaging Spectroradiometer (MODIS) dataset.Further, we find real-world speedups compared to linear search and demonstrate memory-efficient scaling to half a billion data points.

Supplementary Material

MP4 File (3394486.3403138.mp4)

Our SIGKDD 2020 Virtual Conference oral presentation of our work on High-Dimensional Similarity Search with Quantum-Assisted Variational Autoencoder.

Download
189.41 MB

References

[1]

J. Biamonte, P. Wittek, N. Pancotti, P. Rebentrost, N. Wiebe, and S. Lloyd, "Quantum machine learning," Nature, vol. 549, no. 7671, pp. 195--202, 2017.

[2]

S. Lloyd, M. Mohseni, and P. Rebentrost, "Quantum principal component analysis," Nature Physics, vol. 10, no. 9, pp. 631--633, 2014.

[3]

P. Rebentrost, M. Mohseni, and S. Lloyd, "Quantum support vector machine for big data classification," Physical review letters, vol. 113, no. 13, p. 130503, 2014.

[4]

A. Mott, J. Job, J.-R. Vlimant, D. Lidar, and M. Spiropulu, "Solving a higgs optimization problem with quantum annealing for machine learning," Nature, vol. 550, no. 7676, pp. 375--379, 2017.

[5]

W. Vinci, L. Buffoni, H. Sadeghi, A. Khoshaman, E. Andriyash, and M. H. Amin, "A path towards quantum advantage in training deep generative models with quantum annealers," arXiv preprint arXiv:1912.02119, 2019.

[6]

A. Khoshaman, W. Vinci, B. Denis, E. Andriyash, and M. H. Amin, "Quantum variational autoencoder," Quantum Science and Technology, vol. 4, no. 1, p. 014001, 2018.

[7]

J. T. Rolfe, "Discrete Variational Autoencoders," arXiv:1609.02200 [cs, stat], Sept. 2016. arXiv: 1609.02200.

[8]

J. Preskill, "Quantum computing in the nisq era and beyond," Quantum, vol. 2, p. 79, 2018.

[9]

F. Arute, K. Arya, R. Babbush, D. Bacon, J. C. Bardin, R. Barends, R. Biswas, S. Boixo, F. G. Brandao, D. A. Buell, et al., "Quantum supremacy using a programmable superconducting processor," Nature, vol. 574, no. 7779, pp. 505--510, 2019.

[10]

D. P. Kingma and M. Welling, "Auto-encoding variational bayes," arXiv preprint arXiv:1312.6114, 2013.

[11]

M. H. Amin, E. Andriyash, J. Rolfe, B. Kulchytskyy, and R. Melko, "Quantum boltzmann machine," Physical Review X, vol. 8, no. 2, p. 021050, 2018.

[12]

P. Long and R. Servedio, "Restricted Boltzmann Machines are Hard to Approximately Evaluate or Simulate," in ICML 2010 - Proceedings, 27th International Conference on Machine Learning, pp. 703--710, Aug. 2010.

[13]

W. Li, Y. Zhang, Y. Sun, W. Wang, W. Zhang, and X. Lin, "Approximate Nearest Neighbor Search on High Dimensional Data -- Experiments, Analyses, and Improvement (v1.0)," arXiv:1610.02455 [cs], Oct. 2016.

[14]

A. Camerra, J. Shieh, T. Palpanas, T. Rakthanmanon, and E. Keogh, "Beyond one billion time series: indexing and mining very large time series collections with isax2+," Knowledge and information systems, vol. 39, no. 1, pp. 123--151, 2014.

[15]

J. Johnson, M. Douze, and H. Jégou, "Billion-scale similarity search with GPUs," arXiv:1702.08734 [cs], Feb. 2017.

[16]

D. E. Yagoubi, R. Akbarinia, F. Masseglia, and D. Shasha, "RadiusSketch: Massively distributed indexing of time series," in 2017 IEEE International Conference on Data Science and Advanced Analytics (DSAA), pp. 262--271, IEEE, 2017.

[17]

T.-T. Do, A.-D. Doan, and N.-M. Cheung, "Learning to Hash with Binary Deep Neural Network," arXiv:1607.05140 [cs], July 2016. arXiv: 1607.05140.

[18]

N. Srivastava, E. Mansimov, and R. Salakhutdinov, "Unsupervised Learning of Video Representations using LSTMs," arXiv:1502.04681 [cs], Feb. 2015. arXiv: 1502.04681.

[19]

M. A. Carreira-Perpinan and R. Raziperchikolaei, "Hashing with binary autoencoders," in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (Boston, MA, USA), pp. 557--566, IEEE, June 2015.

[20]

Y. Cao, M. Long, J. Wang, and H. Zhu, "Correlation autoencoder hashing for supervised cross-modal search," in Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval - ICMR '16, pp. 197--204, ACM Press, 2016.

[21]

S. S. Board, N. R. Council, et al., Earth science and applications from space: national imperatives for the next decade and beyond. National Academies Press, 2007.

[22]

R. Wright, L. Flynn, H. Garbeil, A. Harris, and E. Pilger, "Automated volcanic eruption detection using modis," Remote sensing of environment, vol. 82, no. 1, pp. 135--155, 2002.

[23]

X. Zhang, M. A. Friedl, C. B. Schaaf, A. H. Strahler, J. C. Hodges, F. Gao, B. C. Reed, and A. Huete, "Monitoring vegetation phenology using modis," Remote sensing of environment, vol. 84, no. 3, pp. 471--475, 2003.

[24]

L. A. Remer, Y. Kaufman, D. Tanré, S. Mattoo, D. Chu, J. V. Martins, R.-R. Li, C. Ichoku, R. Levy, R. Kleidman, et al., "The modis aerosol algorithm, products, and validation," Journal of the atmospheric sciences, vol. 62, no. 4, pp. 947--973, 2005.

[25]

S. Boriah, V. Kumar, C. Potter, M. Steinbach, and S. Klooster, "Land cover change detection using data mining techniques," Technical Report March 14, 2008.

[26]

K. Steinhaeuser, A. R. Ganguly, and N. V. Chawla, "Multivariate and multiscale dependence in the global climate system revealed through complex networks," Climate dynamics, vol. 39, no. 3--4, pp. 889--895, 2012.

[27]

C. Kamath, "On mining scientific datasets," in Data Mining for Scientific and Engineering Applications, pp. 1--21, Springer, 2001.

[28]

D. Zhou, A. Gozolchiani, Y. Ashkenazy, and S. Havlin, "Teleconnection paths via climate network direct link detection," Physical review letters, vol. 115, no. 26, p. 268501, 2015.

[29]

P. Indyk and R. Motwani, "Approximate nearest neighbors: towards removing the curse of dimensionality," in Proceedings of the thirtieth annual ACM symposium on Theory of computing, pp. 604--613, ACM, 1998.

[30]

A. Gionis, P. Indyk, and R. Motwani, "Similarity search in high dimensions via hashing," in Vldb, vol. 99, pp. 518--529, 1999.

Digital Library

[31]

M. D. Hoffman, D. M. Blei, C. Wang, and J. Paisley, "Stochastic variational inference," The Journal of Machine Learning Research, vol. 14, no. 1, pp. 1303--1347, 2013.

Digital Library

[32]

A. H. Khoshaman and M. Amin, "Gumbolt: extending gumbel trick to boltzmann priors," in Advances in Neural Information Processing Systems, pp. 4061--4070, 2018.

[33]

M. D. Hoffman and M. J. Johnson, "Elbo surgery: Yet another way to carve up the variational evidence lower bound," in Workshop in Advances in Approximate Bayesian Inference, NIPS, vol. 1, p. 2, 2016.

[34]

C. K. Sø nderby, T. Raiko, L. Maalø e, S. r. K. Sø nderby, and O. Winther, "Ladder Variational Autoencoders," in Advances in Neural Information Processing Systems 29 (D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, eds.), pp. 3738--3746, Curran Associates, Inc., 2016.

[35]

F. P. Casale, A. Dalca, L. Saglietti, J. Listgarten, and N. Fusi, "Gaussian Process Prior Variational Autoencoders," in Advances in Neural Information Processing Systems 31 (S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, eds.), pp. 10369--10380, Curran Associates, Inc., 2018.

[36]

G. E. Hinton, "Training products of experts by minimizing contrastive divergence," Neural computation, vol. 14, no. 8, pp. 1771--1800, 2002.

Digital Library

[37]

T. Tieleman and G. Hinton, "Using fast weights to improve persistent contrastive divergence," in Proceedings of the 26th Annual International Conference on Machine Learning, pp. 1033--1040, 2009.

[38]

D. Koller, N. Friedman, L. Getoor, and B. Taskar, "Graphical models in a nutshell," Introduction to statistical relational learning, pp. 13--55, 2007.

[39]

E. G. Rieffel, S. Hadfield, T. Hogg, S. Mandrà, J. Marshall, G. Mossi, B. O'Gorman, E. Plamadeala, N. M. Tubman, D. Venturelli, et al., "From ans"atze to z-gates: a NASA view of quantum computing," arXiv preprint arXiv:1905.02860, 2019.

[40]

T. F. Rønnow, Z. Wang, J. Job, S. Boixo, S. V. Isakov, D. Wecker, J. M. Martinis, D. A. Lidar, and M. Troyer, "Defining and detecting quantum speedup," Science, vol. 345, no. 6195, pp. 420--424, 2014.

[41]

H. G. Katzgraber, F. Hamze, and R. S. Andrist, "Glassy chimeras could be blind to quantum speedup: Designing better benchmarks for quantum annealing machines," Physical Review X, vol. 4, no. 2, p. 021008, 2014.

[42]

R. Biswas, Z. Jiang, K. Kechezhi, S. Knysh, S. Mandra, B. O'Gorman, A. Perdomo-Ortiz, A. Petukhov, J. Realpe-Gómez, E. Rieffel, et al., "A NASA perspective on quantum computing: Opportunities and challenges," Parallel Computing, vol. 64, pp. 81--98, 2017.

Digital Library

[43]

S. H. Adachi and M. P. Henderson, "Application of quantum annealing to training of deep neural networks," arXiv preprint arXiv:1510.06356, 2015.

[44]

D. P. Kingma and J. Ba, "Adam: A Method for Stochastic Optimization," arXiv:1412.6980 [cs], Jan. 2017.

[45]

E. Gull, A. J. Millis, A. I. Lichtenstein, A. N. Rubtsov, M. Troyer, and P. Werner, "Continuous-time monte carlo methods for quantum impurity models," Reviews of Modern Physics, vol. 83, no. 2, p. 349, 2011.

[46]

C. O. Justice, E. Vermote, J. R. Townshend, R. Defries, D. P. Roy, D. K. Hall, V. V. Salomonson, J. L. Privette, G. Riggs, A. Strahler, et al., "The moderate resolution imaging spectroradiometer (modis): Land remote sensing for global change research," IEEE transactions on geoscience and remote sensing, vol. 36, no. 4, pp. 1228--1249, 1998.

[47]

R. Solano, K. Didan, A. Jacobson, and A. Huete, "Modis vegetation index user's guide (mod13 series)," Vegetation Index and Phenology Lab, The University of Arizona, pp. 1--38, 2010.

[48]

Y. A. Malkov and D. A. Yashunin, "Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs," arXiv:1603.09320 [cs], Aug. 2018.

[49]

J. Marshall, A. Di Gioacchino, and E. G. Rieffel, "The Perils of Embedding for Sampling Problems," arXiv:1909.12184 [cond-mat, physics:quant-ph], Sept. 2019.

Cited By

Romano YPrimack HVaknin TMeirzada IKarpas IFurman DTradonsky CShlomi R(2024)Quantum sparse codingQuantum Machine Intelligence10.1007/s42484-023-00128-x6:1Online publication date: 30-Jan-2024
https://doi.org/10.1007/s42484-023-00128-x
Tian JSun XDu YZhao SLiu QZhang KYi WHuang WWang CWu XHsieh MLiu TYang WTao D(2023)Recent Advances for Quantum Neural Networks in Generative LearningIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.3272029(1-20)Online publication date: 2023
https://doi.org/10.1109/TPAMI.2023.3272029
Asanjan ABrady LIzquierdo ZAaron Lott PMemarzadeh MSuri NBell DRieffel EGrabbe S(2023)Quantum-Assisted Variational Segmentation for Image-to-Image Wildfire Detection Using Satellite DataIGARSS 2023 - 2023 IEEE International Geoscience and Remote Sensing Symposium10.1109/IGARSS52108.2023.10282464(624-626)Online publication date: 16-Jul-2023
https://doi.org/10.1109/IGARSS52108.2023.10282464
Show More Cited By

Index Terms

High-Dimensional Similarity Search with Quantum-Assisted Variational Autoencoder

Recommendations

Scalable variational quantum circuits for autoencoder-based drug discovery
DATE '22: Proceedings of the 2022 Conference & Exhibition on Design, Automation & Test in Europe

The de novo design of drug molecules is recognized as a time-consuming and costly process, and computational approaches have been applied in each stage of the drug discovery pipeline. Variational autoencoder is one of the computer-aided design methods ...
High-Dimensional Vector Similarity Search: From Time Series to Deep Network Embeddings
SIGMOD '20: Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data

Similarity search is an important and challenging problem that is typically modeled as nearest neighbor search in high dimensional space, where objects are represented as high dimensional vectors and their (dis)similarity is evaluated using a distance ...
VAEPP: Variational Autoencoder with a Pull-Back Prior
Neural Information Processing
Abstract
Many approaches to training generative models by distinct training objectives have been proposed in the past. Variational Autoencoder (VAE) is an outstanding model of them based on log-likelihood. In this paper, we propose a novel learnable prior, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

August 2020

3664 pages

ISBN:9781450379984

DOI:10.1145/3394486

General Chairs:
Rajesh Gupta
UC San Diego, USA
,
Yan Liu
USC, USA
,
Program Chairs:
Mohak Shah
LG Electronics, USA
,
Suju Rajan
Linkedin, USA
,
Publications Chairs:
Jiliang Tang
Michigan State, USA
,
B. Aditya Prakash
Georgia Tech, USA

Copyright © 2020 ACM.

Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 August 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NASA's Advanced Information Systems Technology Program
AFRL Information Directorate
Intelligence Advanced Research Projects Activity

Conference

KDD '20

Sponsor:

KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

July 6 - 10, 2020

CA, Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
880
Total Downloads

Downloads (Last 12 months)195
Downloads (Last 6 weeks)24

Reflects downloads up to 02 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Romano YPrimack HVaknin TMeirzada IKarpas IFurman DTradonsky CShlomi R(2024)Quantum sparse codingQuantum Machine Intelligence10.1007/s42484-023-00128-x6:1Online publication date: 30-Jan-2024
https://doi.org/10.1007/s42484-023-00128-x
Tian JSun XDu YZhao SLiu QZhang KYi WHuang WWang CWu XHsieh MLiu TYang WTao D(2023)Recent Advances for Quantum Neural Networks in Generative LearningIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.3272029(1-20)Online publication date: 2023
https://doi.org/10.1109/TPAMI.2023.3272029
Asanjan ABrady LIzquierdo ZAaron Lott PMemarzadeh MSuri NBell DRieffel EGrabbe S(2023)Quantum-Assisted Variational Segmentation for Image-to-Image Wildfire Detection Using Satellite DataIGARSS 2023 - 2023 IEEE International Geoscience and Remote Sensing Symposium10.1109/IGARSS52108.2023.10282464(624-626)Online publication date: 16-Jul-2023
https://doi.org/10.1109/IGARSS52108.2023.10282464
Jiang JChu C(2023)Classifying and Benchmarking Quantum Annealing Algorithms Based on Quadratic Unconstrained Binary Optimization for Solving NP-Hard ProblemsIEEE Access10.1109/ACCESS.2023.331820611(104165-104178)Online publication date: 2023
https://doi.org/10.1109/ACCESS.2023.3318206
Mishra IMulay PBhagat N(2022)Novel model for predicting the future volume of research articles on applications of Quantum DotsCOLLNET Journal of Scientometrics and Information Management10.1080/09737766.2022.206309116:1(187-213)Online publication date: 27-Jul-2022
https://doi.org/10.1080/09737766.2022.2063091
Hu RTang ZWu EMo QYang RLi J(2022)RDC-SAL: Refine distance compensating with quantum scale-aware learning for crowd counting and localizationApplied Intelligence10.1007/s10489-022-03238-452:12(14336-14348)Online publication date: 1-Sep-2022
https://dl.acm.org/doi/10.1007/s10489-022-03238-4
Qin JWang WXiao CZhang YWang YZhu FChin Ooi BMiao CWang HSkrypnyk IHsu WChawla S(2021)High-Dimensional Similarity Query Processing for Data ScienceProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining10.1145/3447548.3470811(4062-4063)Online publication date: 14-Aug-2021
https://dl.acm.org/doi/10.1145/3447548.3470811

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents