Abstract
This article proposes an approach to reducing fully connected neural networks using classical and modified pretraining of deep neural networks. The authors have demonstrated that this approach can significantly reduce the number of parameters of the trained neural network with little or no reduction in the generalizing ability. The capabilities of the proposed method are demonstrated on classical computer vision datasets.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.REFERENCES
S. Bozinovski and F. Ante, “The influence of pattern similarity and transfer learning upon the training of a base perceptron B2,” in Proc. Symp. Informatica, Bled, 1976 (1976), pp. 3–121.
C. W. Chai, “ProdSumNet: Reducing model parameters in deep neural networks via product-of-sums matrix decompositions” (2018). arXiv:1809.02209 [cs.LG]
V. A. Golovko, “From multilayer perceptrons to neural networks of deep thrust: Paradigms of learning and application,” in Lectures on Neuroinformatics (Mosk. Inzh.-Fiz. Inst., Moscow, 2015), pp. 47–84.
V. Golovko, A. Kroshchanka, V. Turchenko, S. Jankowski, and D. Treadwell, “A new technique for restricted Boltzmann machine learning,” IEEE 8th Int. Conf. on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), Warsaw, 2015 (IEEE, 2015), pp. 182–186. https://doi.org/10.1109/IDAACS.2015.7340725
V. Golovko, A. Kroshchanka, and D. Treadwell, “The nature of unsupervised learning in deep neural networks: A new understanding and novel approach,” Opt. Memory Neural Networks 25, 127–141 (2016). https://doi.org/10.3103/S1060992X16030073
G. E. Hinton, P. Dayan, B. J. Frey, and R. M. Neal, “The “wake-sleep” algorithm for unsupervised neural networks,” Science 268, 1158–1161. https://doi.org/10.1126/science.7761831
G. E. Hinton, S. Osindero, and Y.-W. Teh, “A fast learning algorithm for deep belief nets,” Neural Comput. 18, 1527–1554 (2006). https://doi.org/10.1162/neco.2006.18.7.1527
A. Krizhevsky, “Learning multiple layers of features from tiny images,” in Technical Report (2009), pp. 32–33.
A. Kroshchanka and V. Golovko, “The reduction of fully connected neural network parameters using the pre-training technique,” in 11th IEEE Int. Conf. on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), Cracow, 2021 (IEEE, 2021), Vol. 2, pp. 937–941. https://doi.org/10.1109/IDAACS53288.2021.9661015
K. Kwon and J. Chung, “Reducing parameters of neural networks via recursive tensor approximation,” Electronics 11, 214 (2022). https://doi.org/10.3390/electronics11020214
Y. LeCun, C. Cortes, and C. J. C. Burges, “The MNIST database of handwritten digits” (2013). http://yann.lecun.com/exdb/mnist/. Cited January 5, 2022.
V. Nair and G. E. Hinton, “Rectified linear units improve restricted Boltzmann machines,” in Proc. 27th Int. Conf. on Machine Learning, Haifa, Israel, 2010 (Omnipress, 2010), pp. 807–814.
D. E. Rumelhart and J. L. McClelland, “Information processing in dynamical systems: Foundations of harmony theory,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition (MIT Press, 1986), pp. 194–281.
Funding
This work was supported by the Belarusian Republican Foundation for Basic Research BRFFR, project F22KI-046.
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
COMPLIANCE WITH ETHICAL STANDARDS
This article is a completely original work of its authors; it has not been published before and will not be sent to other publications until the PRIA Editorial Board decides not to accept it for publication.
Conflict of Interests
The process of writing and the content of the article do not give grounds for raising the issue of a conflict of interest.
Additional information
Aliaksandr Kroshchanka received Bachelor’s degree in 2008 and MS degree in 2009 from Pushkin Brest State University. At present he works as a senior lecturer in the Intelligence Information Technologies Department of the Brest State Technical University. Research interests: artificial intelligence, neural networks, deep learning, computer vision, integrated AI systems. He has published more than 40 scientific papers.
Prof. Vladimir Golovko received ME degree in Computer Engineering in 1984 from Bauman Moscow State Technical University. In 1990 he received a PhD degree from the Belarus State Technical University and in 2003 he received a Doctoral science degree in Computer Science from the United Institute of Informatics Problems of the National Academy of Sciences (Belarus). At present he works as head of the Intelligence Information Technologies Department and Laboratory of Artificial Neural Networks of Brest State Technical University and Professor of Akademia Bialska Nauk Stosowanych im. Jana Pawła II. His research interests include artificial intelligence, neural networks, deep learning, autonomous learning robots, signal processing, and intrusion and epilepsy detection. He has published more than 400 scientific papers.
Dr. Marta Chodyka. Doctor of Technical Sciences in the field of computer science, specializing in image analysis, databases, computer networks, software engineering: Lodz University of Technology, Faculty of Electrical Engineering, Electronics, Computer Science and Automation 2013. Master of Science in Computer Science, specialization in software engineering: Lublin University of Technology, Faculty of Electrical Engineering and Computer Science, Institute of Computer Science 2005.
Rights and permissions
About this article
Cite this article
Kroshchanka, A.A., Golovko, V.A. & Chodyka, M. Method for Reducing Neural-Network Models of Computer Vision. Pattern Recognit. Image Anal. 32, 294–300 (2022). https://doi.org/10.1134/S1054661822020146
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1134/S1054661822020146