Abstract
Overfitting is a common problem that is faced when dealing with neural networks, especially as computers continue to get more powerful, and we have the capability to train larger networks with many free parameters. As a result there is a pressing need to develop and explore different techniques to reduce overfitting; we explore the impact of different regularization terms, and their combinations, in the training phase of a single-perceptron neural network.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5, 115–133 (1943)
Rosenblatt, F.: Principles of neurodynamics: perceptrons and the theory of brain mechanisms. In: Palm, G., Aertsen, A. (eds.) Brain Theory. Springer, Berlin, Heidelberg (1962)
LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Back-propagation applied to handwritten zip code recognition. Neural Comput. 1(4), 541–551 (1989)
Zhang, C., Bengio, S., Hardt, M., Recht, B., Vinyals, O.: Understanding deep learning requires rethinking generalization. In: International Conference on Learning Representations, ICLR 2017 (2017)
Louizos, C., Welling, M., Kingma, D.P.: Learning sparse neural networks through \(L_{0}\) regularization. In: International Conference on Learning Representations, ICLR 2018 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Boreland, B., Kunze, H., Levere, K.M. (2021). Comparing Regularization Techniques Applied to a Perceptron. In: Kilgour, D.M., Kunze, H., Makarov, R., Melnik, R., Wang, X. (eds) Recent Developments in Mathematical, Statistical and Computational Sciences. AMMCS 2019. Springer Proceedings in Mathematics & Statistics, vol 343. Springer, Cham. https://doi.org/10.1007/978-3-030-63591-6_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-63591-6_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-63590-9
Online ISBN: 978-3-030-63591-6
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)