Abstract
Hyperdimensional computing (HDC) is a method to perform classification that uses binary vectors with high dimensions and the majority rule. This approach has the potential to be energy-efficient and hence deemed suitable for resource-limited platforms due to its simplicity and massive parallelism. However, in order to achieve high accuracy, HDC sometimes uses hypervectors with tens of thousands of dimensions. This potentially negates its efficiency advantage. In this paper, we examine the necessity of such high dimensions and conduct a detailed theoretical analysis of the relationship between hypervector dimensions and accuracy. Our results demonstrate that as the dimension of the hypervectors increases, the worst-case/average-case HDC prediction accuracy with the majority rule decreases. Building on this insight, we develop HDC models that use binary hypervectors with dimensions orders of magnitude lower than those of state-of-the-art HDC models while maintaining equivalent or even improved accuracy and efficiency. For instance, on the MNIST dataset, we achieve 91.12% HDC accuracy in image classification with a dimension of only 64. Our methods perform operations that are only 0.35% of other HDC models with dimensions of 10,000. Furthermore, we evaluate our methods on ISOLET, UCI-HAR, and Fashion-MNIST datasets and investigate the limits of HDC computing https://github.com/zhangluyan9/EffHDC.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Asgarinejad, F., Thomas, A., Rosing, T.: Detection of epileptic seizures from surface EEG using hyperdimensional computing. In: 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 536–540. IEEE (2020)
Bengio, Y., Léonard, N., Courville, A.: Estimating or propagating gradients through stochastic neurons for conditional computation (2013)
Chuang, Y.C., Chang, C.Y., Wu, A.Y.A.: Dynamic hyperdimensional computing for improving accuracy-energy efficiency trade-offs. In: 2020 IEEE Workshop on Signal Processing Systems (SiPS), pp. 1–5. IEEE (2020)
Duan, S., Liu, Y., Ren, S., Xu, X.: LeHDC: learning-based hyperdimensional computing classifier. arXiv preprint arXiv:2203.09680 (2022)
Frady, E.P., Kleyko, D., Kymn, C.J., Olshausen, B.A., Sommer, F.T.: Computing on functions using randomized vector representations. arXiv preprint arXiv:2109.03429 (2021)
Hassan, E., Halawani, Y., Mohammad, B., Saleh, H.: Hyper-dimensional computing challenges and opportunities for AI applications. IEEE Access 10, 97651–97664 (2021)
Hsieh, C.Y., Chuang, Y.C., Wu, A.Y.A.: FL-HDC: hyperdimensional computing design for the application of federated learning. In: 2021 IEEE 3rd International Conference on Artificial Intelligence Circuits and Systems (AICAS), pp. 1–5. IEEE (2021)
Imani, M., et al.: QuantHD: a quantization framework for hyperdimensional computing. IEEE Trans. Comput.-Aided Des. Integr. Circu. Syst. 39(10), 2268–2278 (2019)
Imani, M., et al.: A framework for collaborative learning in secure high-dimensional space. In: 2019 IEEE 12th International Conference on Cloud Computing (CLOUD), pp. 435–446. IEEE (2019)
Imani, M., Messerly, J., Wu, F., Pi, W., Rosing, T.: A binary learning framework for hyperdimensional computing. In: 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 126–131. IEEE (2019)
Imani, M., et al.: SearcHD: a memory-centric hyperdimensional computing with stochastic training. IEEE Trans. Comput.-Aided Des. Integr. Circ. Syst. 39(10), 2422–2433 (2019)
Neubert, P., Schubert, S., Protzel, P.: An introduction to hyperdimensional computing for robotics. KI-Künstliche Intelligenz 33(4), 319–330 (2019)
Rahimi, A., Kanerva, P., Rabaey, J.M.: A robust and energy-efficient classifier using brain-inspired hyperdimensional computing. In: Proceedings of the 2016 International Symposium on Low Power Electronics and Design, pp. 64–69 (2016)
Salamat, S., Imani, M., Khaleghi, B., Rosing, T.: F5-HD: fast flexible FPGA-based framework for refreshing hyperdimensional computing. In: Proceedings of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, pp. 53–62 (2019)
Schlegel, K., Neubert, P., Protzel, P.: A comparison of vector symbolic architectures. Artif. Intell. Rev. 55(6), 4523–4555 (2022)
Schmuck, M., Benini, L., Rahimi, A.: Hardware optimizations of dense binary hyperdimensional computing: Rematerialization of hypervectors, binarized bundling, and combinational associative memory. ACM J. Emerg. Technol. Comput. Syst. (JETC) 15(4), 1–25 (2019)
Tax, D.M., Duin, R.P.: Using two-class classifiers for multiclass classification. In: 2002 International Conference on Pattern Recognition, vol. 2, pp. 124–127. IEEE (2002)
Thomas, A., Dasgupta, S., Rosing, T.: Theoretical foundations of hyperdimensional computing. arXiv preprint arXiv:2010.07426 (2020)
Yu, T., Zhang, Y., Zhang, Z., De Sa, C.: Understanding hyperdimensional computing for parallel single-pass learning. arXiv preprint arXiv:2202.04805 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yan, Z., Wang, S., Tang, K., Wong, WF. (2023). Efficient Hyperdimensional Computing. In: Koutra, D., Plant, C., Gomez Rodriguez, M., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Research Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14170. Springer, Cham. https://doi.org/10.1007/978-3-031-43415-0_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-43415-0_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43414-3
Online ISBN: 978-3-031-43415-0
eBook Packages: Computer ScienceComputer Science (R0)
