A Novel Training Mechanism for Extending Convolutional Neural Network
Pages 166 - 170
Abstract
Convolutional Neural Network (CNN) has obtained great success in the computer vision domain in the recent years. These CNN models adopt deeper neural network architecture to achieve high recognition accuracy, the training costs of time and dataset are dramatically increased. While the recognizing categories are expanded, the CNN architecture needs to be modified, the whole CNN model requires to be retrained. Transfer learning method is adopted to save the training cost by migrating part of learned weights, from the existed CNN model to the target CNN model with expanded recognizing categories. However, the requirement of modifying neural network architecture still consumes huge amount of the training cost. This paper presents a new training mechanism, called Extended Learning, to solve the above problems. By using the proposed Partially Back-Propagation Operation, the CNN model can expand new classification categories without modifying the architecture of the CNN model, the learning weights from previously training results can be retained, the training cost of time and dataset can be reduced accordingly. The experimental result shows that the proposed extended learning method can save 16.7% training image count compared to the transfer learning method, with the target accuracy of 0.75.
References
[1]
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770--778).
[2]
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and F.-F. Li. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3), 211--252.
[3]
K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
[4]
I. Goodfellow, Y. Bengio, and A. Courville. Deep learning. MIT press, 2016.
[5]
S. J. Pan and Q. Yang. A survey on transfer learning. IEEE Trans. on Knowledge and Data Engineering, 22(10), 1345--1359.
[6]
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1--9).
[7]
M. Oquab, L. Bottou, I. Laptev, and J. Sivic. Learning and transferring mid-level image representations using convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1717--1724).
[8]
A. Krizhevsky, I. Sutskever, G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097--1105).
[9]
J. Yoon, E. Yang, J. Lee, and S. J. Hwang. Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547.
[10]
TensorFlow (July 02, 2019). TensorFlow Transfer Learning Using Pretrained ConvNets. Retrieved from: https://www.tensorflow.org/tutorials/images/transfer_learning#prepare_training_and_validation_cats_and_dogs_datasets/.
[11]
M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D. G. Murray, B. Steiner, P. Tucker, V. Vasudevan, P. Warden, M. Wicke, Y. Yu, and X. Zheng. Tensorflow: A system for large-scale machine learning."In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16) (pp. 265--283).
[12]
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, Liang-Chieh Chen. Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4510--4520).
[13]
D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning representations by back-propagating errors. Cognitive modeling, 5(3), 1.
[14]
A. Krizhevsky and G. E. Hinton. Learning multiple layers of features from tiny images. (Vol. 1, No. 4, p. 7). Technical report, University of Toronto.
[15]
J. T. Springenberg, A. Dosovitskiy, T. Brox, and M. Riedmiller. Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806.
Index Terms
- A Novel Training Mechanism for Extending Convolutional Neural Network
Recommendations
Fruit category classification via an eight-layer convolutional neural network with parametric rectified linear unit and dropout technique
AbstractIn this paper, we apply an improved deep convolutional neural network (CNN) in fruit category classification, which is a hotspot in computer vision field. We created an 8-layer deep convolutional neural network, and utilized parametric rectified ...
Towards dropout training for convolutional neural networks
Recently, dropout has seen increasing use in deep learning. For deep convolutional neural networks, dropout is known to work well in fully-connected layers. However, its effect in convolutional and pooling layers is still not clear. This paper ...
Research on improved wavelet convolutional wavelet neural networks
AbstractConvolutional neural network (CNN) is recognized as state of the art of deep learning algorithm, which has a good ability on the image classification and recognition. The problems of CNN are as follows: the precision, accuracy and efficiency of ...
Comments
Information & Contributors
Information
Published In
February 2020
607 pages
ISBN:9781450376426
DOI:10.1145/3383972
Copyright © 2020 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
In-Cooperation
- Shenzhen University: Shenzhen University
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 26 May 2020
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICMLC 2020
ICMLC 2020: 2020 12th International Conference on Machine Learning and Computing
February 15 - 17, 2020
Shenzhen, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 43Total Downloads
- Downloads (Last 12 months)1
- Downloads (Last 6 weeks)0
Reflects downloads up to 11 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in