Teamwork Formation for Keepaway in Robotics Soccer (Reinforcement Learning Approach)

Abstract

In this paper, we discuss guidelines for a reward design problem that defines when and what amount of reward should be given to the agents, within the context of reinforcement learning approach. We take keepaway soccer as a standard task of multiagent domain which requires skilled teamwork. The difficulties of designing reward for good teamwork are due to its features as follows: i) since it is a continuing task which has no explicit goal, it is hard to tell when reward should be given to the agents, ii) since it is a multiagent cooperative task, it is hard to make a fair share of the reward for each agent’s contribution. Through some experiments, we show that reward design have a major effect on the agent’s behavior, and introduce the reward function that makes agents perform keepaway successfully.