Abstract
Fractional calculus has been found to be a promising area of research for information processing and modeling of some physical systems. In this paper, we propose a fractional gradient descent method for the backpropagation (BP) training of neural networks. In particular, the Caputo derivative is employed to evaluate the fractional-order gradient of the error defined as the traditional quadratic energy function. Simulation has been implemented to illustrate the performance of presented fractional-order BP algorithm on large dataset: MNIST.
J. Wang—This work was supported in part by the National Natural Science Foundation of China (No. 61305075), the China Postdoctoral Science Foundation (No. 2012M520624), the Natural Science Foundation of Shandong Province (Nos. ZR2013FQ004, ZR2013DM015), the Specialized Research Fund for the Doctoral Program of Higher Education of China (No. 20130133120014) and the Fundamental Research Funds for the Central Universities (Nos. 14CX05042A, 15CX05053A, 15CX02079A, 15CX08011A).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Chen, X.: Application of fractional calculus in BP neural networks (in Chinese). Master thesis. Nanjing Forestry University, Nanjing, Jiangsu (2013)
Zhang, S., Yu, Y., Wang, H.: Mittag-Leffler stability of fractional-order Hopfield neural networks. Nonlinear Anal. Hybri. 16, 104–121 (2015)
Chen, B., Chen, J.: Global \(O(t^{-\alpha })\) stability and global asymptotical periodicity for a non-autonomous fractional-order neural networks with time varying delays. Neural Netw. 73, 47–57 (2016)
Rakkiyappan, R., Sivaranjani, R., Velmurugan, G., Cao, J.: Analysis of global \(O(t^{-\alpha })\) stability and global asymptotical periodicity for a class of fractional-order complex-valued neural networks with time varying dela. Neural Netw. 77, 51–69 (2016)
Rakkiyappan, R., Cao, J., Velmurugan, G.: Existence and uniform stability analysis of fractional-order complex-valued neural networks with time delays. IEEE Trans. Neural Netw. Learn. 26, 84–97 (2015)
Xiao, M., Zheng, W., Jiang, G., Cao, J.: Undamped oscillations generated by Hopf Bifurcations in fractional-order recurrent neural networks with Caputo derivative. IEEE Trans. Neural Netw. Learn. 26, 3201–3214 (2015)
Wang, H., Yu, Y., Wen, G.: Stability analysis of fractional-order Hopfield neural networks with time delays. Neural Netw. 55, 98–109 (2014)
Wu, A., Zhang, J., Zen, Z.: Dynamic behaviors of a class of memristor-based Hopfield networks. Phys. Lett. A. 375, 1661–1665 (2011)
Wu, A., Wen, S., Zen, Z.: Anti-synchronization control of a class of memristive recurrent neural networks. Commun. Nonlinear Sci. 18, 373–385 (2013)
Pu, Y., Zhou, J., Zhang, Y., Zhang, N., Huang, G., Siarry, P.: Fractional extreme value adaptive training method: fractional steepest descent approach. IEEE Trans. Neural Netw. Learn. 26, 653–662 (2015)
Love, E.R.: Fractional derivatives of imaginary order. J. Lond. Math. Soc. 3, 241–259 (1971)
Oldham, K.B., Spanier, J.: The Fractional Calculus: Theory and Applications of Differentiation and Integration to Arbitrary Order. Academic, Cambridge (1974)
Mcbride, A.C.: Fractional Calculus. Halsted, USA (1986)
Nishimoto, K.: Fractional Calculus: Integrations and Differentiations of Arbitrary Order. New Haven University Press, New Haven (1989)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back propagating errors. Nature 323, 533–536 (1986)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Yang, G., Zhang, B., Sang, Z., Wang, J., Chen, H. (2017). A Caputo-Type Fractional-Order Gradient Descent Learning of BP Neural Networks. In: Cong, F., Leung, A., Wei, Q. (eds) Advances in Neural Networks - ISNN 2017. ISNN 2017. Lecture Notes in Computer Science(), vol 10261. Springer, Cham. https://doi.org/10.1007/978-3-319-59072-1_64
Download citation
DOI: https://doi.org/10.1007/978-3-319-59072-1_64
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-59071-4
Online ISBN: 978-3-319-59072-1
eBook Packages: Computer ScienceComputer Science (R0)