Abstract
Restricted Boltzmann machines are a generative neural network. They summarize their input data to build a probabilistic model that can then be used to reconstruct missing data or to classify new data. Unlike discrete Boltzmann machines, where the data are mapped to the space of integers or bitstrings, continuous Boltzmann machines directly use floating point numbers and therefore represent the data with higher fidelity. The primary limitation in using Boltzmann machines for big-data problems is the efficiency of the training algorithm. This paper describes an efficient deterministic algorithm for training continuous machines.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abu-halaweh, N. M., & Harrison, R. W. (2009). Practical fuzzy decision trees. In 2009 IEEE symposium on computational intelligence and data mining (pp. 211–216).
Abu-halaweh, N. M., & Harrison, R. W. (2010). Identifying essential features for the classification of real and pseudo microRNAs precursors using fuzzy decision trees. In 2010 IEEE symposium on computational intelligence in bioinformatics and computational biology (pp. 1–7).
Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798–1828.
Chen, H., & Murray, A. F. (2003). Continuous restricted boltzmann machine with an implementable training algorithm. IEE Proceedings—Vision, Image and Signal Processing, 150(3), 153–158.
Durham, E. E. A., Yu, X., & Harrison, R. W. (2014). FDT 2.0: Improving scalability of the fuzzy decision tree induction tool—integrating database storage. In 2014 IEEE symposium on computational intelligence in healthcare and e-health (CICARE) (pp. 187–190).
Fischer, A., & Igel, C. (2014). Training restricted boltzmann machines: An introduction. Pattern Recognition, 47(1), 25–39.
Harrison, R., McDermott, M., & Umoja, C. (2017). Recognizing protein secondary structures with neural networks. In 28th international workshop on database and expert systems applications (DEXA) 2017 . (pp. 62–68).
Harrison, R. W., & Freas, C. (2017). Fuzzy restricted boltzmann machines. In North American fuzzy information processing society annual conference. (pp. 392–398). Cham: Springer.
Hebb, D. O. (1949). Organization of behavior. New York: Wiley.
Hinton, G. E. (2002). Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8), 1771–1800.
Hinton, G. (2010). UTML TR 2010003 A Practical Guide to Training Restricted Boltzmann Machines. http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf.
Hinton, G. E. (2002). Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8), 1771–1800.
LeCunn, Y., Cortes, C., & Burges, C. J. C. (2017). http://yann.lecun.com/exdb/mnist/.
Lera, G., & Pinzolas, M. (2002). Neighborhood based levenberg-marquardt algorithm for neural network training. IEEE Transactions on Neural Networks, 13(5), 1200–1203.
Lichman, M. (2013). UCI machine learning repository. http://archive.ics.uci.edu/ml.
Marquardt, D. (1963). An algorithm for least-squares estimation of nonlinear parameters. Journal of the Society for Industrial and Applied Mathematics, 11(2), 431–441. https://doi.org/10.1137/0111030.
McMillen, T., Simen, P., & Behseta, S. (2011). Hebbian learning in linear nonlinear networks with tuning curves leads to near-optimal, multi-alternative decision making. Neural Networks, 24(5), 417–426.
Salakhutdinov, R., & Hinton, G. E. (2012). An efficient learning procedure for deep Boltzmann machines. Neural Computation, 24(8), 1967–2006.
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324.
Yu, X., Weber, I., & Harrison, R. (2013). Sparse representation for HIV-1 protease drug resistance prediction. In Proceedings of the 2013 SIAM international conference on data mining. (pp. 342–349). Philadelphia: SIAM.
Acknowledgements
The research was supported in part by the National Institutes of Health Grant GM062920.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Harrison, R.W. Continuous restricted Boltzmann machines. Wireless Netw 28, 1263–1267 (2022). https://doi.org/10.1007/s11276-018-01903-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11276-018-01903-6