Abstract
This paper is concerned with the problem of Reinforcement Learning (RL) in large or continuous spaces. Function approximation is the main method to solve such kind of problem. We propose using neural networks as function approximators in this paper. Then we experiment with three kind of neural networks in Mountain-Car task and illustrate comparisons among them. The result shows that CMAC and Fuzzy ARTMAP perform better than BP in Reinforcement Learning with Function Approximation (RLFA).
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. Bradford Books, MIT (1998)
Baird, L.C.: Residual Algorithms: Reinforcement Learning with Function Approximation. In: Prieditis, A., Russell, S. (eds.) Proceedings of Twelfth International Conference on Machine Learning, July 9-12, Morgan Kaufman Publishers, San Francisco (1995)
Gaskett, C., Wettergreen, D., Zelinsky, A.: Q-Learning in Continuous State and Action Spaces. In: Australian Joint Conference on Artificial Intelligence, pp. 417–428 (1999)
Munos, R.: A convergent Reinforcement Learning algorithm in the continuous case: the Finite-Element Reinforcement Learning. In: Proceedings of IJCAI 1997, pp. 826–831. Morgan Kaufmann, San Francisco (1997)
Albus, J.S.: A new approach to manipulator control:the cerebellar model articulation controller (CMAC). Journal of Dynamic Systems, Measurement, and Control, Trans. ASME, Series G 97 (1975)
Sutton, R.S.: Generalization in Reinforcement learning: Successful Examples Using Sparse Coarse Coding. In: Advances in Neural Information Processing Systems 8, pp. 1038–1044. The MIT Press, Cambridge (1996)
Andonie, R., Sasu, L.: A Modified Fuzzy ARTMAP Architecture for Incremental Learning Function Approximation. In: Castillo, O. (ed.) Proceedings of the IASTED International Conference on Neural Networks and Computational Intelligence (NCI 2003), Cancun, Mexico, pp. 124–129. ACTA Press, Anaheim (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wang, D., Gao, Y., Yang, P. (2005). Applying Neural Network to Reinforcement Learning in Continuous Spaces. In: Wang, J., Liao, X., Yi, Z. (eds) Advances in Neural Networks – ISNN 2005. ISNN 2005. Lecture Notes in Computer Science, vol 3496. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11427391_99
Download citation
DOI: https://doi.org/10.1007/11427391_99
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-25912-1
Online ISBN: 978-3-540-32065-4
eBook Packages: Computer ScienceComputer Science (R0)