CMUnited-97 - RoboCup-97 Small-Robot World Champion Team
CMUnited-97 - RoboCup-97 Small-Robot World Champion Team
CMUnited-97 - RoboCup-97 Small-Robot World Champion Team
Articles
CMUNITED-97
RoboCup-97
Small-Robot World Champion Team
■ Robotic soccer is a challenging research domain teammates and opponents. Opportunities, and
that involves multiple agents that need to collab- indeed demands, for innovative and novel
orate in an adversarial environment to achieve techniques abound.
specific objectives. In this article, we describe CMU- One of the advantages of robotic soccer is
NITED, the team of small robotic agents that we
that it enables the direct comparison of differ-
developed to enter the RoboCup-97 competition.
ent systems; they can be matched against each
We designed and built the robotic agents, devised
the appropriate vision algorithm, and developed other in competitions. We have been pursuing
and implemented algorithms for strategic collabo- research in the robotic soccer domain within
ration between the robots in an uncertain and the RoboCup initiative (Kitano, Kuniyoshi, et
dynamic environment. The robots can organize al. 1997), which, in 1997, included a simulator
themselves in formations, hold specific roles, and league and small-size and medium-size robot
pursue their goals. In game situations, they have leagues. We have been doing research exten-
demonstrated their collaborative behaviors on sively in the simulator league, developing
multiple occasions. We present an overview of the learning techniques and team strategies in sim-
vision-processing algorithm that successfully
ulation (Stone and Veloso 1998a, 1998d).
tracks multiple moving objects and predicts trajec-
Many of these team strategies were directly
tories. The article then focuses on the agent behav-
iors, ranging from low-level individual behaviors incorporated into the robotic system described
to coordinated, strategic team behaviors. CMUNITED here. We are currently also applying machine-
won the RoboCup-97 small-robot competition at learning techniques to acquire hard-to-tune
the Fifteenth International Joint Conference on boundary behaviors for the real robots.
Artificial Intelligence in Nagoya, Japan. This article describes the overall architecture
of our small-size robotic soccer team. The com-
bination of robust hardware, real-time vision,
P
roblem solving in complex domains and intelligent control represented a signifi-
often involves multiple agents, dynamic cant challenge that we were able to successfully
environments, and the need for learning meet. The work described in this article is fully
from feedback and previous experience. Robot- implemented as our CMUNITED-97 RoboCup
ic soccer is an example of such complex tasks team. CMUNITED-97 won the RoboCup-97 small-
for which multiple agents need to collaborate robot competition at the Fifteenth Internation-
in an adversarial environment to achieve spe- al Joint Conference on Artificial Intelligence
cific objectives. Robotic soccer offers a chal- (IJCAI-97) in Nagoya, Japan. Our team scored a
lenging research domain to investigate a large total of 13 goals and only suffered 1. Figure 1
spectrum of issues relevant to the development shows a picture of our robotic agents.1
of complete autonomous agents (Asada et al. The specific contributions of the CMUNITED-
1998; Kitano, Tambe, et al. 1997). 97 robot team include the following:
The fast-paced nature of the domain neces- First is a demonstration of a complete inte-
sitates real-time sensing coupled with quick gration of perception, action, and cognition in
behaving and decision making. The behaviors a team of multiple robotic agents.
and decision-making processes can range from Second is a set of robust behaviors for indi-
the most simple reactive behaviors, such as vidual agents. Each agent is equipped with
moving directly toward the ball, to arbitrarily skills that enable it to effectively perform indi-
complex reasoning procedures that take into vidual and collaborative actions.
account the actions and perceived strategies of Third is reliable perception through the use
Copyright © 1998, American Association for Artificial Intelligence. All rights reserved. 0738-4602-1998 / $2.00 FALL 1998 61
Articles
of a Kalman-Bucy filter. Sensing through our each robot and the ball. This information is
vision-processing algorithm allows for color- sent to an off-board controller and distributed
based tracking of multiple moving objects and to the different agent algorithms. Each agent
prediction of object movement, particularly evaluates the world state and uses its strategic
the ball, even when inevitable sharp trajectory knowledge to decide what to do next. Actions
changes occur. are motion commands that are sent by the off-
Fourth is multiagent strategic reasoning. board controller through radio communica-
Collaboration between robots is achieved tion. Commands can be broadcast or sent
through (1) a flexible role-based approach by directly to individual agents. Each robot has an
which the task space is decomposed, and identification binary code that is used on
agents are assigned subtasks; (2) a flexible team board to detect commands intended for the
structure by which agents are organized in for- robot. This complete system is fully imple-
mations, and homogeneous agents flexibly mented.
switch roles within formations; and (3) alter- Although it might be possible to fit an on-
native plans allowing for collaboration (for board vision system onto robots of small size,
example, passing to a teammate or shooting at in the interest of being able to quickly move
the goal directly) that are controlled by prede- on to strategic multiagent research issues, we
fined metrics and are evaluated in real time. opted for a global vision system. It is part of
our ongoing research to also investigate and
develop teams of robots capable of local per-
Real-Time Perception for ception (Shen et al. 1998; Mataric 1995). Part
Multiple Agents of our challenge in developing approaches to
individual robot autonomy will consist of
The small-size robot league setup is viewed as combining different sources of perception,
an overall complete autonomous framework namely, local sensing, and targeted and broad-
composed of the physical navigational robotic casted communication.
agents, a video camera overlooking the playing The fact that perception is achieved by a
field connected to a centralized interface com- video camera that overlooks the complete field
puter, and several clients as the minds of the offers an opportunity to get a global view of
small-size robot players. Figure 2 sketches the the world state. Although this setup simplifies
building blocks of the architecture. the sharing of information among multiple
The complete system is fully autonomous, agents, it presents a challenge for reliable and
consisting of a well-defined and challenging real-time processing of the movement of mul-
processing cycle. The global vision algorithm tiple moving objects—in our case, the ball, five
perceives the dynamic environment and agents on our team, and five agents on the
processes the images, giving the positions of opposing team.
62 AI MAGAZINE
Articles
Action Client
Object
Code Module
Positions
Coaching/
Perceiving/
Transmitting
Client
Robot-specific Interface
Module
Action code
Client
Module
Client
Module
FALL 1998 63
Articles
64 AI MAGAZINE
Articles
Single-Agent Behaviors
To be able to successfully collaborate, agents
require robust basic skills, including the ability Ball Robot
to go to a given place on the field, the ability
to direct the ball in a given direction, and the
ability to intercept a moving ball. All these
Intermediate
skills must be executed while the robot avoids
Targets
obstacles such as the walls and other robots. A
The robot’s hardware includes two motors
that allow it to turn on itself. The front and the
back of the robots are also absolutely equiva-
lent in terms of navigation. Through these two
features, the robots can therefore efficiently
switch direction by turning, at most, 90°. Ball Final Ball Target
If a robot is to accurately direct the ball
toward a target position, it must be able to Line b
approach the ball from a specified direction.
Using the ball prediction from the vision sys- Line a
tem, the robot aims at a point on the far side Intermediate D
of the target position. The robots are equipped Target
with two methods of doing so: (1) ball collec- Predicted
tion, moving behind a ball and knocking it Interception
toward the target, and (2) ball interception, Point
waiting for the ball to cross its path and then Robot
intercepting the moving ball toward the target. B
When using the ball-collection behavior,
the robot considers a line from the target posi-
tion to the ball’s current or predicted position, Figure 4. Single-Agent Behaviors to Enable Team Collaboration.
depending on whether the ball is moving. The A. Ball collection (aiming for a pass or the goal).
robot then plans a path to a point on the line B. Ball interception (receiving a pass).
and behind the ball such that it does not hit
the ball on the way and such that it ends up
tually position itself in a place from which it
facing the target position. Finally, the robot
can intercept the ball toward the target.
accelerates to the target. Figure 4a illustrates
this behavior. Multiagent Behaviors
When using the ball-interception behavior
Although the single-agent behaviors are effec-
(figure 4b), however, the robot considers a line
from itself to the target position and deter- tive when just a single robot is on the field, if
mines where the ball’s path will intersect this all five robots were simultaneously chasing the
line. The robot then positions itself along this ball and trying to shoot it at the goal, chaos
line so that it will be able to accelerate to the would result. To achieve coordinated multia-
point of intersection at the same time that the gent behavior, we organize the five robots into
ball arrives. a flexible team structure.
In practice, the robot chooses between its The team structure, or formation, defines a
two ball-handling routines based on whether set of roles, or positions, with associated be-
the ball will eventually cross its path at a point haviors. The robots are then dynamically
such that the robot could intercept it toward mapped into the positions.
the goal. Thus, the robot gives precedence to Each robot is equipped with the knowledge
the ball-interception routine, only using ball required to play any position in each of several
collection when necessary. When using ball formations. The positions indicate the areas of
collection, it actually aims at the ball’s predict- the field that the robots should move to in the
ed location a fixed time in the future to even- default situation. There are also different active
FALL 1998 65
Articles
modes that determine when a given robot obstruction-free index of the two line seg-
should move to the ball or do something else ments that the ball must traverse if the receiver
instead. Finally, the robot with the ball choos- is to shoot the ball (lines b and c in figure 6).
es whether to shoot or pass to a teammate In the case of a shot, only one line segment
using a passing evaluation function. must be considered (line a). The value of each
These high-level, multiagent behaviors were possible pass or shot is the product of the rele-
originally developed in simulation and then vant obstruction-free indexes. Robots can be
transferred to the robot-control code. Only the biased toward passing or shooting by further
run-time passing evaluation function was multiplying the values by a factor determined
redefined. Further details, particularly about by the relative proximities of the active robot
the flexible team structures, are available in and the potential receivers to the goal. The
Stone and Veloso (1998b, 1998c). robot chooses the pass or shot with the maxi-
Positions, Formations, and Active mum value. The obstruction-free index of line
Modes Positions are defined as flexible segment l is computed by the algorithm
regions within which the player attempts to shown in figure 5 (variable names correspond
move toward the ball. For example, a robot to those in figure 6).
playing the right-wing (or right-forward) posi- Thus, the obstruction-free index reflects
tion remains on the right side of the field near how easily an opponent could intercept the
the opponents’ goal until the ball comes pass or the subsequent shot. The closer the
toward it. Positions are classified as defender, opponent is to the line and the farther it is
midfielder, or forward based on the locations from the ball, the better chance it has of inter-
of these regions. They are also given behavior cepting the ball.
specifications in terms of which other posi- The Goalkeeper The goalkeeper robot has
CMUNITED-97 tions should be considered as potential pass both special hardware and special software.
successfully receivers. Thus, it does not switch positions like the oth-
At any given time, each of the robots plays er robots. The goalkeeper’s physical frame is
demonstrated a particular position on the field. However, distinct from that of the other robots in that it
the feasibility each robot has all the knowledge necessary to is as long as allowed under the RoboCup-97
and play any position. Therefore, the robots can— rules (18 centimeters) to block as much of the
and do—switch positions on the fly. For exam- goal as possible. The goalkeeper’s role is to pre-
effectiveness ple, robots A and B switch positions when vent the ball from entering the goal. It stays
of teams of robot A chases the ball into the region of robot parallel and close to the goal, aiming always to
B. Then, robot A continues chasing the ball, be directly even with the ball’s lateral coordi-
multiagent and robot B moves to the position vacated by nate on the field.
robotic A. Ideally, simply staying even with the ball
The predefined positions known to all play- would guarantee that the ball would never get
systems. ers are collected into formations. An example past the goalkeeper. However, because the
of a formation is the collection of positions robots cannot accelerate as fast as the ball can,
consisting of the goalkeeper, one defender, one it would be possible to defeat such a behavior.
midfielder, and two attackers. Another possible Therefore, the goalkeeper continually moni-
formation consists of the goalkeeper, two tors the ball’s trajectory. In some cases, it
defenders, and two attackers. moves to the ball’s predicted destination point
Run-Time Evaluation of Collaborative ahead of time. The decision about when to
Opportunities One of CMUNITED-97’s main move to the predicted ball position is both cru-
features is the robots’ ability to collaborate by cial and difficult, as illustrated in figure 7. Our
passing the ball. The robots use an evaluation goalkeeper robot currently takes into account
function that takes into account teammate the predicted velocity and direction of the ball
and opponent positions to determine whether to select its moves.
to pass the ball or shoot. In particular, as part
of the formation definition, each position has
a set of positions to which it considers passing.
Discussion and Conclusion
For example, a defender might consider pass- CMUNITED-97 successfully demonstrated the fea-
ing to any forward or midfielder, but a forward sibility and effectiveness of teams of multia-
would consider passing to other forwards but gent robotic systems. Within this paradigm,
not backward to a midfielder or defender. one of the major challenges was to close the
For each such position that is occupied by a loop, that is, to integrate all the different mod-
teammate, the robot evaluates the pass to the ules, ranging from perception to strategic mul-
position as well as its own shot. To evaluate tiagent reasoning. CMUNITED is an example of a
each possible pass, the robot computes the fully implemented multiagent system in
66 AI MAGAZINE
Articles
1. obstruction-free-index = 1.
2. For each opponent O:
• Compute the distance x from O to l and the distance y along l to l’s origin, that is,
the end at which the ball will be kicked by the robot (figure 5).
• Define constants min-dist and max-denominator. Opponents farther than min-dist
from l are not considered. When discounting obstruction-free-index in the next step,
the y distance is never considered to be larger than max-denominator. For example,
in figure 5, the opponent near the goal would be evaluated with y = max-denominator,
rather than its actual distance from the ball. The reasoning is that beyond distance
max-denominator, the opponent has enough time to block the ball: the extra distance
is no longer useful.
• If x < min-dist and x < y,
obstruction-free-index *= x/MIN(max-denominator, y).
3. Return obstruction-free-index.
Figure 5. Algorithm for the Run-Time Evaluation of Collaboration Opportunities (Pass or Shoot).
Teammate
line c
line b
line a
Robot x
y
Ball
Opponent
which the loop is closed. In addition, we resources to build a single team of five robots,
implemented interesting strategic behaviors, with one spare. Therefore, we offer a restricted
including agent collaboration and real-time evaluation of CMUNITED based on the results of
evaluation of alternative actions. 4 effective 10-minute games that were played
It is generally difficult to accumulate signif- at RoboCup-97. We also include anecdotal evi-
icant scientific results to test teams of robots. dence of the multiagent capabilities of the
Realistically, extended runs are prohibited by CMUNITED-97 robotic soccer team.
battery limitations and the difficulty of keep- The CMUNITED-97 robot team played games
ing many robots operational concurrently. against robot teams from the Nara Institute of
Furthermore, to date, we have only had the Science and Technology (NAIST), Japan; Uni-
FALL 1998 67
Articles
fast
slow
Ball A Ball B
Figure 7. Goalkeeping.
68 AI MAGAZINE
Articles
Naval Research, under contract N00014-95-1- width Communication for Real-Time Strategic Team-
0591. Views and conclusions contained in this work. Artificial Intelligence. Forthcoming.
document are those of the authors and should Stone, P., and Veloso, M. 1998c. The CMUNITED-97
not be interpreted as necessarily representing Simulator Team. In RoboCup-97: Robot Soccer World
official policies or endorsements, either Cup I, ed. H. Kitano, 389–397. Berlin: Springer Ver-
expressed or implied, of the Air Force, the lag.
Department of the Navy, the Office of Naval Stone, P., and Veloso, M. 1998d. Using Decision Tree
Confidence Factors for Multiagent Control. In
Research, or the U.S. government.
RoboCup-97: Robot Soccer World Cup I, ed. H. Kitano,
Notes 99–111. Berlin: Springer Verlag.
Veloso, M.; Stone, P.; Han, K.; and Achim, S. 1998.
1. For the hardware description of our robots, see
The CMUNITED-97 Small-Robot Team. In RoboCup-97:
Veloso et al. (1998).
Robot Soccer World Cup I, ed. H. Kitano, 242–256.
Berlin: Springer Verlag.
References
Manuela Veloso is associate pro-
Asada, M.; Kuniyoshi, Y.; Drogoul, A.; Asama, H.;
fessor of computer science at
Mataric, M.; Duhaut, D.; Stone, P.; and Kitano, H.
Carnegie Mellon University. She
1998. The RoboCup Physical Agent Challenge: Phase
received her Ph.D. in computer
1. Applied Artificial Intelligence 12:251–264.
science from Carnegie Mellon in
Asada, M.; Noda, S.; Tawaratumida, S.; and Hosoda, 1992. She received a B.S. in electri-
K. 1996. Purposive Behavior Acquisition for a Real cal engineering in 1980 and an
Robot by Vision-Based Reinforcement Learning. M.Sc. in electrical and computer
Machine Learning 23:279–303. engineering in 1984 from the
Han, K., and Veloso, M. 1998. Reactive Visual Con- Instituto Superior Tecnico in Lisbon as well as an
trol of Multiple Nonholonomic Robotic Agents. M.A. in computer science in 1986 from Boston Uni-
Paper presented at the International Conference on versity. Veloso researches in the area of AI. Her long-
Robotics and Automation, May, Leuven, Belgium. term research goal is the effective construction of
Kalman, R. E., and Bucy, R. S. 1961. New Results in intelligent agents where cognition, sensors, and
Linear Filter and Prediction Theory. Journal of Basic action are combined to address planning, execution,
Engineering, March, 95–108. and learning tasks. In 1995, she received a National
Kitano, H.; Kuniyoshi, Y.; Noda, I.; Asada, M.; Mat- Science Foundation Career Award to pursue her
subara, H.; and Osawa, H. 1997. RoboCup: A Chal- research interests in autonomous robotic agents.
lenge Problem for AI. AI Magazine 18(1): 73–85. Veloso’s team of soccer robots won the RoboCup-97
competition. She was awarded the Allen Newell
Kitano, H.; Tambe, M.; Stone, P.; Veloso, M.; Corade-
Medal for Excellence in Research in 1997.
schi, S.; Osawa, E.; Matsubara, H.; Noda, I.; and Asa-
da, M. 1997. The RoboCup Synthetic Agent Chal-
lenge 97. In Proceedings of the Fifteenth Peter Stone is a Ph.D. candidate in
International Joint Conference on Artificial Intelli- the Computer Science Department
gence, 24–29. Menlo Park, Calif.: International Joint at Carnegie Mellon University. He
Conferences on Artificial Intelligence. received his B.S. from the Univer-
sity of Chicago in 1993. He is on
Mataric, M. J. 1995. Designing and Understanding
the organizing committee of
Adaptive Group Behavior. Adaptive Behavior 4(1).
RoboCup—the Robotic Soccer
Sahota, M. K.; Mackworth, A. K.; Barman, R. A.; and World Cup. His research interests
Kingdon, S. J. 1995. Real-Time Control of Soccer- include planning and machine
Playing Robots Using Off-Board Vision: The DYNAMITE learning, particularly in multiagent systems. His e-
Test Bed. In Proceedings of the IEEE International mail address is pstone@cs.cmu.edu.
Conference on Systems, Man, and Cybernetics,
3690–3663. New York: IEEE Computer Society.
Kwun Han is currently a Ph.D.
Sargent, R.; Bailey, B.; Witty, C.; and Wright, A. 1997.
candidate in computer science at
Dynamic Object Capture Using Fast Vision Tracking.
Carnegie Mellon University. He
AI Magazine 18(1): 65–72.
received his B.Sc. with honors in
Shen, W.-M.; Adibi, J.; Adobbati, R.; Cho, B.; Erdem, computer science from Brown
A.; Moradi, H.; Salemi, B.; and Tejada, S. 1998. Build- University in 1996. His current
ing Integrated Mobile Robots for Soccer Competi- research interests include multia-
tion. Paper presented at the International Confer- gent systems, machine learning,
ence on Robotics and Automation, May, Leuven, mobile robotics, robotic soccer,
Belgium. and computer vision.
Stone, P., and Veloso, M. 1998a. A Layered Approach
to Learning Client Behaviors in the RoboCup Soccer
Server. Applied Artificial Intelligence 12(2–3): 165–188.
Stone, P., and Veloso, M. 1998b. Task Decomposi-
tion, Dynamic Role Assignment, and Low-Band-
FALL 1998 69