Abstract
The purpose of this article is to present a novel learning paradigm that extracts reward-related low-dimensional state space by combining correlation-based learning like Input Correlation Learning (ICO learning) and reward-based learning like Reinforcement Learning (RL). Since ICO learning can quickly find a correlation between a state and an unwanted condition (e.g., failure), we use it to extract low-dimensional feature space in which we can find a failure avoidance policy. Then, the extracted feature space is used as a prior for RL. If we can extract proper feature space for a given task, a model of the policy can be simple and the policy can be easily improved. The performance of this learning paradigm is evaluated through simulation of a cart-pole system. As a result, we show that the proposed method can enhance the feature extraction process to find the proper feature space for a pole balancing policy. That is it allows a policy to effectively stabilize the pole in the largest domain of initial conditions compared to only using ICO learning or only using RL without any prior knowledge.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Pfeifer, R., Lungarella, M., Iida, F.: Self-Organization, Embodiment, and Biologically Inspired Robotics. Science 318(5853), 1088–1093 (2007)
Porr, B., Wörgötter, F.: Strongly Improved Stability and Faster Convergence of Temporal Sequence Learning by Using Input Correlations Only. Neural Comput. 18(6), 1380–1412 (2006)
Phon-Amnuaisuk, S.: Learning Cooperative Behaviours in Multiagent Reinforcement Learning. In: Leung, C.S., Lee, M., Chan, J.H. (eds.) ICONIP 2009. LNCS, vol. 5863, pp. 570–579. Springer, Heidelberg (2009)
Kolter, J.Z., Ng, A.Y.: Policy Search via the Signed Derivative. In: The Proc. of Robotics: Science and Systems (RSS), p. 27 (2009)
Melo, F.S., Lopes, M., Santos-Victor, J., Ribeiro, M.I.: A Unified Framework for Imitation-Like Behaviours. In: The Proc. of 4th International Symposium on Imitation in Animals and Artifacts, pp. 241–250 (2007)
Doya, K.: Reinforcement Learning in Continuous Time and Space. Neural Comput. 12, 219–245 (2000)
Barto, A.G., Sutton, R.S., Anderson, C.: Neuron-Like Adaptive Elements That Can Solve Difficult Learning Control Problems. IEEE Transactions on Systems, Man, and Cybernetics 13, 834–846 (1983)
Pasemann, F.: Evolving Neuropolicys for Balancing an Inverted Pendulum. Network: Computation in Neural Systems 9, 495–511 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Manoonpong, P., Wörgötter, F., Morimoto, J. (2010). Extraction of Reward-Related Feature Space Using Correlation-Based and Reward-Based Learning Methods. In: Wong, K.W., Mendis, B.S.U., Bouzerdoum, A. (eds) Neural Information Processing. Theory and Algorithms. ICONIP 2010. Lecture Notes in Computer Science, vol 6443. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17537-4_51
Download citation
DOI: https://doi.org/10.1007/978-3-642-17537-4_51
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-17536-7
Online ISBN: 978-3-642-17537-4
eBook Packages: Computer ScienceComputer Science (R0)