INTELLIGENCE FOR HUMAN-ASSISTANT
PLANETARY SURFACE ROBOTS
Robert Hirsh, NASA Johnson Space Center, USA
Jeffrey Graham, S&K Technologies, Inc., USA
Kimberly Tyree, NASA Johnson Space Center, USA
Maarten Sierhuis, RIACS/NASA Ames Research Center, USA
William J. Clancey, NASA Ames Research Center, USA
1. INTRODUCTION
Robots will require intelligence to succeed in the uncertain and changing environment on lunar
and planetary surfaces. Even with humans directly involved in controlling such robots, individual
robotic intelligence is still needed. In fact, robotic intelligence may be even more necessary in
human-robot collaborative work than for robots operating alone. In addition to knowledge of
terrain and task, human-interacting robots also need knowledge of human needs and the ability to
respond to human commands, movements, and dynamics.
Planetary surface robots are necessary to perform the more dangerous tasks and to provide a
level of physical capability that space-suited humans cannot provide during an EVA (Extra
Vehicular Activity) on a planetary surface. The research goal of the ERA (EVA Robotic Assistant)
Project in the Automation, Robotics, and Simulation Division (ARSD) at NASA’s Johnson Space
Center (JSC) is to develop robots capable of providing effective and time-saving assistance to
astronauts [1]. Our approach is centered on Earth-based field testing with humans and robots
working together, in order to best learn what intelligence, capabilities, mechanical design and
support systems should be used on future planetary robots. Highly focused remote field tests are
combined with continual local subsystem testing, robot hardware and software development, and
consultation with exploration scenario experts (i.e. NASA astronauts, scientists, engineers, and
university research professors).
The central premise in developing effective human-assistant planetary surface robots is that
robotic intelligence is needed. The exact type, method, forms and/or quantity of intelligence is an
open issue being explored on the ERA project, as well as others. In addition to field testing,
theoretical research into this area can help provide answers on how to design future planetary
robots. Many fundamental intelligence issues are discussed by Murphy [2], including (a) learning,
(b) planning, (c) reasoning, (d) problem solving, (e) knowledge representation, and (f) computer
vision (stereo tracking, gestures). The new “social interaction/emotional” form of intelligence that
some consider critical to Human Robot Interaction (HRI) can also be addressed by humanassistant planetary surface robots, as human operators feel more comfortable working with a
robot when the robot is verbally (or even physically) interacting with them. Arkin [3] and Murphy
are both proponents of the hybrid deliberative-reasoning/reactive-execution architecture as the
best general architecture for fully realizing robot potential, and the robots discussed herein
implement a design continuously progressing toward this hybrid philosophy.
The remainder of this chapter will describe the challenges associated with robotic assistance to
astronauts, our general research approach, the intelligence incorporated into our robots, and the
results and lessons learned from over six years of testing human-assistant mobile robots in field
settings relevant to planetary exploration. The chapter concludes with some key considerations
for future work in this area.
2. PROBLEM STATEMENT
2.1 The Problem
When humans travel to live and work on the Moon or Mars, they will have many tasks to
perform. They will need to transform their base camp into a safe habitat, possibly including the
placement, construction, and installation of power sources, communication relays, in-situ
resource extraction facilities, greenhouses, and “safe havens” to retreat into during times of
intense solar activity. They will need to maintain and repair this entire infrastructure for the
duration of their stay – a vital task which will become increasingly difficult with longer duration
missions. In addition to these housekeeping tasks, the astronauts will need to pursue their goals of
exploration and discovery in the surrounding environment. One of the biggest challenges to this
slate of tasks is that humans will be limited to working in spacesuits when outside. Suited
astronauts will have limited visibility, dexterity, and strength, as well as limited work durations
due to fatigue and limited air and power reserves. Additionally, the astronauts will have a limited
ability to carry and use all the tools and instruments required to safely perform their work.
Thus, robots enter the picture to provide much-needed support [4]. The key aspect of our
research is HRI, in the context of teaming to perform EVA tasks. Our approach is a top-down
holistic strategy involving all salient aspects of realistic scenarios, including analog surface
environments, habitats, advanced space suit subjects, remote science teams, and autonomous
robots. However, robots also introduce a new host of constraints and challenges, and this is the
source of our research topic.
An important aspect of HRI research is understanding and identifying the best compliment of
humans and robots for a given task—just as one would want the “best” team for the job, we need
to choose the best mix, types and quantities of robots and humans. One approach is to somehow
measure the effectiveness of various team compositions on a given set of tasks. Even this is
difficult, since tasks must be reduced into primitive actions that can be performed by several
candidate teammates, and then performed by the various humans and robots while being able to
measure their effectiveness. Task decomposition into primitive actions is discussed in various
sources including [5], and potential HRI metrics in [6, 7, 8].
2.2 The Motivation
Robotic intelligence and autonomy are needed to allow effective HRI to take place. For robots
to work effectively with humans on planetary surfaces, they require some level of intelligence in
at least four areas: two-way communication with humans, safety awareness, self-maintenance,
and self-monitoring. While some aspects of these of these requirements can be accomplished with
simple, straight-forward software and hardware, our field research to date has shown that the
more intelligence that robots possess, the more effective they will be in their tasks.
To work alongside humans, robots will need to communicate with them in intuitive (and
spacesuit compatible) ways. Astronauts will be communicating with each other as well as with
robots, so similar language, communication protocols, and voice loop hardware for both human
and robot agents will reduce confusion and allow more flexibility. Robots will need to distinguish
commands directed toward them from all other communication. Robots will need to respond to
various sources of commands, whether from speech, gesture, or direct computer controls, and
whether from nearby or distance sources. Space-suited astronauts will need to issue commands
with some glove-friendly method, such as voice or gestures. Many commands will require
confirmation of some type to prevent execution of misinterpreted commands caused by the noisy
environment, echoes, static, or recognition errors. Such confirmation will need to be
understandable by the humans, and responsive to the current amount of activity on the voice loop.
Some commands may also require or naturally involve gesturing to indicate the subject or object
of the command (deictic gestures). All commands need to be succinct and yet still clear, even
when detailed instructions are required.
Robots will need to respond to human commands and at the same time, maintain their own
safety and prior task assignments. In Figure 1, a robot is shown during field tests in a canyon in
Utah, USA. The robot is pulling a trailer and avoiding obstacles, while at the same time following
the astronaut and being responsive to new voice commands such as taking snapshots. This
situation also introduces another constraint: robots need to be safe for humans to work with side
by side, especially when the humans are in spacesuits in a hazardous environment.
Figure 1. Robot performing multiple parallel tasks during an extra-vehicular activity simulation
Other constraints deal with maintenance of the robots. Such maintenance needs to be userfriendly, considering the scarcity and possible lack of any “shirt-sleeve” repair environments.
Remote maintenance or even self maintenance is highly useful, especially for minor issues such
as power cycling components and restarting subsystems.
Monitoring of robot status needs to be simple and not time-consuming, as astronauts inside
and outside the habitat have little extra time to devote to such monitoring. Astronaut EVA time in
particular is very limited and precious. And because of time delays, even an army of humans
staffing an Earth-based mission control will not be able to provide the timely responses for
missions to remote sites such as Mars. Robot self-monitoring would be best, with the robot able
to respond, adjust, or repair itself autonomously, or else to alert humans if functionality is at risk
of failing, has failed, or can continue in a degraded mode. Even local monitoring by astronauts in
the vicinity may not be consistently available due to the fragility of the newly established,
resource-limited infrastructure. Thus, at times, a robot will need to be able to carry out tasks on its
own during periods without any external monitoring or control.
Other authors also discuss the need for autonomy and robot intelligence in future planetary
exploration missions. An extensive survey determined that key challenges for robots include
robustness, human-robot interaction, and the ability for robots to handle mission-level objectives
[9] [10]. Common sense reasoning, the understanding of human intentions, and the ability to
execute complex plans in uncertain environments are all needed for human-assistant exploration
robots. Longer duration missions in which human operational control is minimized will require
more knowledge and understanding on the part of the robot. While human supervision and
anomaly response will still be needed, systems must be automated more and more in the future in
order to reduce the overhead for human crew members [11].
All these constraints and challenges will require intelligence to some degree on the part of the
robot. The more intelligence the robot is given, the more efficient and productive such robots will
be. Ultimately, robots need to operate on the same level as humans to be the most accepted and
utilized during complex, joint human-robot space exploration missions.
3. RESEARCH APPROACH
Aside from knowing the issues and goals described above, we must determine how to begin
work in this research area. Without the resources to experiment in space as a first step, we must
locate and use effective test-beds here on Earth. The questions which need to be asked include:
• What robotic tasks or capabilities are really useful to astronauts?
• What level and type of interaction is best or most useful?
• What is the best mix of robots and humans to successfully execute a given task?
• If needs vary between individuals, what range of behaviors are expected?
We maintain these questions are best answered using the “practice like you play” approach –
by field testing realistic scenarios in space-relevant environments, with experts in the fields of
exploration, science, and habitat construction serving as astronaut test subjects (including actual
astronauts). Additional guidance comes from continual discussions with exploration scenario
developers and astronauts, and researchers in related areas such as advanced spacesuit design,
voice and data communications, infrastructure development, human-robot interaction, and
logistical and mission support.
3.1 Robotic Capability Requirements
Identification of desirable tasks and capabilities for robotic assistants is coupled with ongoing
improvement of already developed capabilities. Robustness and reliability are key components of
the overall utility of such robots. Without reliable behavior, humans will not be able to depend on
their robotic assistants and will naturally tend to seek out alternative means of support, even if
those means are less productive, less efficient, or introduce greater risk to the humans. The
reliability needed is not only in hardware, but in software as well. Adaptability is a partner to this
type of reliability, as it requires the ability to adapt to subtask failures or plan changes. One of the
key advantages of human explorers is their ability to quickly evaluate situations and to change
their plans to fit any new facts or issues that arise [4]. Robots assisting humans will need this
same ability to adapt.
While robots operating alone can perform tasks at their own rate, human-assistant robots need
to perform at a speed or pace matching that of human activity. Just as with reliability, without
robots that can keep up, astronauts will seek out other tools to use. Part of our research is
developing robots that can provide the needed services at the needed time.
Robot Capabilities Matrix
ROBOT PLATFORM
Boudreaux
Thibodeaux
SCOUT
Teleoperation
√
√
√
Full autonomy
√
√
√
Point-to-point navigation
√
√
√
Human following with stereo vision
CAPABILITIES
ONBOARD COMPUTING
OPERATING SYSTEMS
PROGRAMMING LANGUAGES
COMMUNICATIONS APIs
SENSORS
X (possible with added HW)
√
√
Obstacle avoidance
√
√
√
Autonomous Scouting and Mapping
√
√
×
Manipulation (robotic arm, hand)
√
√
×
Trailer-towing
√
√
√
Tool carrying
√
√
√
Passenger Carrying
×
×
√
Watching
√
√
√
Natural Language Understanding
√
√
√
Speech Synthesis
√
√
√
EVA traversal maps, MET
√
√
√
Differential (Skid) Steering
√
×
×
All-Wheel-Drive
√
√
√
Front-wheel steer
×
√
√
Rear-wheel steer
×
√
√
Night-time driving
×
×
√
√
√
√
Video Feedback
√
√
√
Wireless connectivity
√
√
√
Laptop1
1.9GhZ P4M
1.9GhZ P4M
2.4GhZ P4
Laptop2
1.9GhZ P4M
1.9GhZ P4M
2.4GhZ P4
Laptop3
Open
Open
3.6Ghz P4
Laptop4
×
Open
Open
Linux
√
√
√
Windows XP
×
×
√
C++
√
√
√
Java
√
√
√
CORBA (primary)
√
√
√
NDDS
×
×
√
Raw Sockets
√
×
√
Differential GPS
√
√
√
Inertial Navigation Unit
√
√
√
Voltage/Current/Power
√
√
√
Velocity
√
√
√
1394/Firewire Cameras
√
√
√
×
Steering motor temperature
×
√
×
×
√
Steering limit switches
×
√
Web Camera
×
×
√
Pan-tilt cameras
√
√
√
√ (shared)
7DOF Arm+Barrett 3Finger Hand
Drive motors
Steering motors
Software power control of devices
SPECIFICATIONS
√
Offboard camera control
Drive motor temperature
ACTUATORS
√
Human following with GPS
√
√
×
√
√
√
√
√
×
2.75
Height (m)
1.9
1.9
Width (m)
0.95
1.25
2
Length (m)
1.25
1.5
2.75
Weight Ready-To-Roll (kg)
1150
180
180
Max Velocity (MPH)
2
20
7
Approximate Available Payload (kg)
45
90
90
Endurance (hours)
4
TBD
4
Ground Clearance (cm)
35
25
33
Onboard Crew Size
0
0
2
Towing capacity (kg, at ½ max slope)
70
TBD
TBD
Max obstacle size (cm)
35
35
35
Max slope (degrees)
40
20
10
Table 1. ERA Robot Capabilities Matrix
Physically and mechanically, ERAs must possess the strength, endurance, speed and
traversability that matches or exceeds their human teammates. Table 1 outlines the capabilities of
three such robots used in our research.
3.2 Robotic Intelligence Requirements
Robotic intelligence can help in achieving these goals of reliability and maintaining a proper
pace. In order for a robot to be reliable in an unknown environment such as on a planetary
surface, that robot must have adaptable software which can determine how to respond in
unforeseen situations. Such adaptability is a form of intelligence involving reasoning, decisionmaking and fault tolerance. When humans are involved as well, robotic intelligence often needs
to adapt to whatever course of action the humans decide to pursue at any given time. For
maintaining a proper pace, intelligence can allow a robot to plan ahead in order to help
compensate for any physical limitations. If a robot cannot travel as fast, or along the same
obstacle-strewn course as a human astronaut, intelligence can aid the robot in several ways. The
robot could predict when it will be needed and arrange its tasks such that it can be where it is
needed when it is needed. The robot could plan its own paths, travel at its own rate, and meet up
with the human astronauts at a later time without having to be guided directly by the astronauts as
they travel. The robot could also plan an alternate, more manageable route, and meet the human
astronauts beyond any impassable terrain.
Independence is another key issue for human-assistant space robots, closely linked to the need
for intelligence. As mentioned earlier, astronauts’ time is very precious and limited. Earth-based
controllers are limited by time delays and often possess inadequate sensor feedback. The more
robots can do on their own, the more helpful they will be to astronauts and their exploration
goals.
3.3 Practical Issues and Constraints
In order to identify, develop, and test the requirements identified above, appropriate field
testing is needed. While initial theoretical and laboratory based testing is necessary and useful,
along with discussions with experts in multiple subjects, in order to move robotic assistants into
practical use, field testing in relevant environments is required. Space-relevant, or planetaryanalog, sites here on Earth are chosen for various reasons. Lists of such sites have been compiled,
with pros and cons identified for each [12, 13, 14]. Basically, the terrain and environment should
be applicable to the specific tests to be performed. For mobility tests, rough terrain is needed. For
communication tests, remote areas with obscuring hills and hidden canyons may provide the
desired challenges. For logistical tests of science and exploration goals, areas with interesting
science are needed. Some of the most severe or persistent issues that are found in cooperative
field testing with humans and robots involve communications, power, dust, and thermal
variations. Choosing field test sites that can stress these areas produces more data relevant to and
experience in dealing with these issues.
In addition to an applicable environment, remoteness in general can provide an added benefit
for field testing: the enforcement of self-sufficiency. Identifying what equipment,
communications, and capabilities are really needed for a remote space robot is one of the primary
accomplishments of many field tests, often providing surprising results. Challenges are frequently
uncovered that would not be found in a laboratory or simulation setting, specifically relating to
terrain, logistics, and human interaction needs.
In addition to providing a test environment for robots, field testing provides operational
training for humans. For future missions, humans may control robots from a long distance (such
as from Earth), from a nearby but still remote location (such as from an orbital station or ground
habitat), or from the immediate neighborhood. All three control locations present different needs,
and all three can be, and must be, tested in the field with as much fidelity as possible. By
participating in field testing, not only can operational crew gain an understanding of what is
needed, but researchers can gain an understanding of what interfaces and capabilities will be
needed in the future.
Thus, our approach has been to perform as much field testing in planetary-analog sites as
possible, interspersed with local subsystem testing in our outdoor Rock Yard as needed (see
Figure 2). The field testing provides information about what issues we need to address more
closely and what capabilities we need to develop for further testing.
Figure 2. Boudreaux (left) and SCOUT (right) in JSC Rock Yard with suited test subjects
4. IMPLEMENTATION AND RESULTS
4.1 Robots
Our research group in ARSD is currently working
with three robotic test beds for planetary surface
exploration, named Boudreaux, Thibodeaux, and
SCOUT. Boudreaux and SCOUT are shown in Figure 2,
and Thibodeaux is shown in Figure 3. Boudreaux and
Thibodeaux are four-wheeled, battery powered robots
meant to serve as ERAs. They have a range of
capabilities meant to assist astronauts as they set up a
base camp and explore their surroundings. Both of these
robots are approximately the size of a typical ATV (All
Terrain Vehicle). Each robot has a suite of sensors,
including cameras, lasers, GPS units, and dynamic
measurement units for angular pose. A seven degree of
Figure 3. Thibodeaux in the JSC
freedom manipulator arm and three-fingered Barrett®
Rock Yard
hand can be mounted to either robot. Either robot can also
pull a trailer with supplies and tools. They carry multiple computers on board, have speech
synthesizers, and can perform speech recognition.
SCOUT stands for Science, Crew, Operations, and Utility Testbed. This robot is designed to
carry two suited crewmembers, and can be operated in multiple modes: manual driving,
teleoperated driving, and autonomous driving. During autonomous activity, this robot can
perform many of the same tasks that the ERA robots can, though it is, of course, larger in size.
SCOUT measures approximately 3 by 2 meters. This robot also has cameras, lasers, GPS units,
and an IMU (Inertial Measurement Unit), and carries multiple computers on board. SCOUT also
has lights for nighttime operation, for the ease of human drivers. SCOUT is a newer project, and
thus has not been involved in remote field testing, other than an initial foray for information
gathering only. Some preliminary work has also been done with the Robonaut [40], project
dealing with the potential for a mobile, highly dexterous robot for human-robot interaction. Some
autonomous abilities also used in the ERA and SCOUT projects have been implemented for use
during human interactive tasks with Robonaut, and future plans include much more work in this
area [15] [16]. Although much local testing has been carried out with SCOUT and Robonaut, the
remainder of this section will concentrate on the remote field testing done with the ERA robots.
4.2 Robotic Assistant Control Architecture
A high-level overview of the ERA robot architecture is shown in Figure 4 to illustrate its basic
design and capabilities. The individual objects in the figure are discussed in the following
paragraphs. Our system is similar to other multi-layer autonomous robotic architectures (such as
Remote Agents [17] or 3T [18]), with the significant difference that we may use the space-suited
astronaut (that the robot is assisting) as the highest level of intelligence in our system. The human
may command the robot to perform a certain task, and the robot works on that task until finished
or receipt of another command from the human interrupts that task.
The software processes (servers) that interface with actuators/sensors (the grey ovals at the
bottom of the figure) represent the lowest layer of the architecture. These servers actually
communicate with robot hardware to read sensors and/or command actuators. Low-level actuator
servers are designed to control a physical actuator (such as the drive motors, pan tilt units,
computerized on/off control, etc). Generally, these servers take a myopic view of the world, and
simply try to safely control their device as they are directed, without reasoning why the action
was requested. Because these servers are connected to physical devices, they should be
commanded by only one behavior at a time to prevent contention. Low-level sensor/data servers
obtain and process data such as GPS location, temperatures, laser range measurements, battery
status, power usage, etc., and make it available to other servers upon request. These types of
servers can simultaneously communicate with all the servers who desire data from the particular
sensor (i.e., no resource contention issues).
The servers which talk to multiple sensors/actuators to perform more sophisticated actions
(which we call behaviors) are indicated by the blue ovals. Some behaviors currently in the ERA
architecture are (1) deploy, (2) follow, (3) photograph, (4) sample, (5) say, (6) scout, and (7)
watch. Note: other architectures frequently have a separate obstacle avoidance behavior, but in
the ERA architecture this functionality is a modular, configurable component used by other move
behaviors such as follow or scout.
As an example of a behavior from our architecture, consider (3) from the list above. When this
behavior is activated, the robot is being told to take a photograph of a particular location. This
behavior must find the current robot and target locations, and then calculate the pan/tilt values to
“point” the camera toward that specified location. The behavior must then command the pan/tilt
unit to the specified angles, and then wait for that movement to finish. Finally, the behavior must
obtain the resulting image from the camera (now that it is pointed in the right direction), and
return that image to whoever initiated that behavior request (most likely another behavior or the
Executor). Note that multiple behaviors can receive data from the same sensor without
contention, but when multiple behaviors utilize the same actuator, contention will exist when both
behaviors are simultaneously active.
This contention between simultaneously active behaviors is resolved in the Executor process
(red oval), which monitors and arbitrates resource contention using a priority scheme. Each
requested action is received in a command structure also containing a desired priority (possibly
originating from a human or external agent). The command maps to a set of behaviors to produce
the desired result. If other behaviors are active, are in resource contention or behavioral conflict,
and have lower priority, the Executor suspends them in favor of the higher-priority command and
its behavior set. Besides behavioral commands, commands to handle reporting and mode
switching are defined. The ERA robots may be configured into “safe”,”teleoperated”, or
“autonomous” modes.
In addition to conflict resolution and mode changing, the Executor also serves as the single
point for interfacing with external influencing agents such as space-suited humans and software
systems. A well-defined set of commands and replies is used by these external entities to
command or monitor the autonomous robots. This interface and command set has been used
successfully over several years of field tests and is evolving as the robot capabilities and scenario
complexities increase.
Figure 4. Control Architecture used in the ERA and SCOUT projects. The Seven areas of AI are
from Barr [31]. The location of these types of AI within our architecture is called out in the
individual process ovals in {braces}.
The ERA architecture currently lacks its own high-level planner and cannot coordinate a team
of humans and robots nor provide procedure tracking or mission-data dissemination to the outside
world. These critical functionalities and their requisite intelligence methods are currently
provided by the external software system known as Mobile Agents software Architecture (or
MAA), currently under development at NASA Ames Research Center. We will briefly describe
the MAA here; for a detailed overview of the Mobile Agents project and MAA we point the
reader to Clancey, et al [19].
The MAA is a multi-agent software architecture developed using the Brahms multi-agent
modeling language and the Java programming language [20, 21, 22]. The Brahms language
incorporates theories of activities [23, 24], situated cognition [25], situated action [26], work
practice modeling [21], and distributed AI and multi-agent systems [27, 28]. Belief-based agents
use situated-action and production rules to act and reason in the world. Agents can communicate
with each other through a communicative act, in which beliefs are transferred to receiver agents.
Agents communicate with each other using speech acts. Speech acts are represented as objects
with a well-defined communication protocol. The meaning and intention of the speech act objects
are represented as beliefs in the agents. By communicating these beliefs, agents can communicate
their intentions and state to one another. The MAA defines a number of speech act types, based
on the FIPA Communicative Act Library [29]. Currently the MAA supports the message types:
subscribe, inform, request, accept, and failure.
The MAA consists of distributed agents that serve external entities, such as astronauts, ERA
robots and mission operation systems. Each external entity has its own Personal Agent (PA) that
enables the entity to cooperate with the other entities in the architecture, by communicating
speech acts to their respective PA’s. The PA’s for each entity can be seen as a) the teamwork
coordinator and for the non-human entities b) the deliberator and plan executor. People interact
with their PA via a speech dialog system [30]. In other words, the MAA enables the coordination
of the work of people, robots and mission operation systems during a mission. From a functional
perspective the MAA enables the ERAs to work together in a team of astronauts and other robots
on a surface exploration mission through:
1) Use of Activity models (an EVA plan) to establish context for data records and expectations
for location and duration of astronaut work. Future iterations of the architecture this will lead
to a representation of meaning of activity, e.g., "survey," to establish expectations about
astronaut actions.
2) Distributed agent architecture allows flexible configuration for managing and automating
aspects of the workflow (e.g., storing data in a database with appropriately generated
descriptors -- combining data about astronaut location, activity in process, etc. -- and sending
e-mail notification to relevant remote participants, e.g., a remote science team).
3) Heuristic interpretation of telemetry information (e.g., biosensors) to generate alerts (again
stored, forwarded, broadcast as necessary) -- along with contextual interpretations (e.g., heart
rate expectation depending on astronaut actions, such as climbing a hill) as described in #1.
4) Contextual disambiguation of voice commands (e.g. “[Robot name] join me”, “Stop working
with me”, “[Robot name] come here”, “What is the next activity”).
5) Automated association of data records (e.g., voice annotations and photos) based on context
(e.g. “Take a panorama and label it picture at work site four”)
6) Integration of external systems (e.g., cameras, database) through "communication agents"
that mediate command and data flow between people and their tools (including robots).
7) Integration of diverse data sources (e.g., GPS, agent registry, inter-agent communication
protocols) that enables continuous assignment of resources to astronauts (e.g., following or
tracking an astronaut).
In Figure 4 the various forms of Artificial Intelligence [31] are mapped onto the processes in
the software architectures for ERA and MAA. As both continue evolving, we expect to improve
the existing intelligence methods and implement new ones into the overall architecture, leading to
more possibilities in the field.
4.3 Field Tests
The ERA robots have participated in multiple remote field tests to date, in several locations in
the USA. Most notably, including Arizona (Meteor Crater, Joseph City, Cinder Lake, and SP
Mountain), and Utah (at the Mars Desert Research Station near Hanksville). Most of these field
tests last for two weeks and involve partners from other research groups. The participants from
NASA Johnson Space Center, in addition to ARSD, include the Advanced Suit group from the
Crew and Thermal Systems Division, personnel from the Mission Operations Directorate, the
Exploration Systems Engineering Division, and the Astronaut Office. NASA Ames Research
Center participants include Mobile Agents, Brahms multi-agent simulation environment, Mobile
Exploration (MEX) wireless communication system [32], RIALIST spoken dialogue, and Science
Organizer [41] groups. Communications experts from NASA Glenn Research Center and NASA
Kennedy Space Center often participate in these field tests. Finally, various universities have
been involved with a range of contributions: crew member biosensors, science sensor
deployments, and suited crew member test subjects. An overview of this field testing is given in
Table 2.
4.3.1 Arizona
Our field tests in Arizona usually take place in early September, and have been occurring
regularly since 2000. The Meteor Crater area was also used during the Apollo era for astronaut
training in lunar driving and geology work. The type of terrain here varies from flat, hard ground
to sloping, dusty, sometimes rocky ground. The terrain is generally mild in the areas in which we
operate, with few obstructions to line of sight operations. The large open areas do allow greatly
extended test runs compared to the limited area we have available for local testing near Johnson
Space Center. This allows us to focus on human-robot interactions during nearly hour-long
scenarios with suited crewmembers, and multi-hour tests with shirtsleeve crewmembers.
The ERA field tests at Meteor Crater involve human crew members with high-fidelity
spacesuits. This has allowed us to understand constraints dealing with pressurized suits, such as
the inherent noise from air handling equipment, the human dexterity and strength limitations
imposed by the bulky, pressurized suit, and the limited amount of time the human crew member
has for conducting activities.
Scenarios have included logistical construction tasks, such as deploying power cables (Figure
5a) and flexible solar panels. These tasks are carried out with both human and robot team
members. Humans have provided route guidance for cable deployment and secured the solar
panels to the ground after deployment. Boudreaux provided the strength to carry all the material,
and provided a fixed heading for precise deployment of the solar panel.
Figure 5. ERA Field Testing: (a) power cable deploy, (b) geology traverse, (c) science instrument
(geophone) deploy
Science experiments have also been enacted in these field tests. The ERA has pulled a
science trailer with various tools, autonomously following behind an astronaut in order to provide
access to equipment needed for analysis of samples in the field. Tools for geology investigations
have also been carried by the robot. A science sensor (a field spectroradiometer) was pulled along
behind the robot as it performed a search pattern, automatically gathering data for further analysis
of the soil [33]. Geophones, which help to determine ground composition, have been deployed by
Boudreaux using several methods: cooperatively by carrying the sensors for the astronaut to use
as they progress along a straight line, or individually by deploying the sensors itself using its
manipulator arm (Figure 5c).
4.3.2 Utah
The Mars Society’s Mars Desert Research Station, near Hanksville, Utah, provides an
excellent field test site for human-robot interaction. The terrain here varies from flat, hard ground,
to rolling hills with loose gravel, to rocky ground and deep canyons. Many obstructions to line of
sight can be found, and the varied terrain provides nearly every type of testing ground desired.
ERA tests here focus on the integration of robots with the human crew. A cylindrical, twostory habitat allows the human crew to live and work in the environment. The robots can be sent
out by themselves to scout around and provide imagery for the human crew in the habitat to use
in identifying prime exploration spots. EVAs with humans and robots then take place for further
exploration. The robots serve as pack mules, safety monitors, information providers (such as
location and distance from habitat), data gatherers (such as panoramic imagery), and
communication relays.
Figure 6. ERA Field Testing in Utah.
4.4 Lessons Learned
Lessons learned during our field testing fall into logistical, operational, and functional
categories. Logistical lessons include such practical considerations as having radios for voice
communication, tents for sheltering support crews, adequate work environment, and spare robot
parts. Operational lessons include performing pre-deployment check lists (e.g. check for loose
wiring, fasteners, and connectors, and having all spare batteries fully-charged in case they are
needed during a test) and continuous monitoring the robot (e.g. temperature, remaining battery
life, and power usage) during all tests. Also to prevent electrical damage to components, every
device should have a properly rated fuse specific to that device to prevent damage should
electrical problems arise. Functional lessons include the requirements for performance and
capability, relating both to the physical characteristics of the robots and to the onboard
intelligence they possess.
Robotic intelligence is involved in some form in most of the field test scenarios described in
section 4.3. This section will concentrate on the functional lessons learned with regard to robotic
intelligence requirements for human-assistant planetary activities. The main categories of
intelligence include: directional knowledge, task knowledge, physical interaction with humans,
and verbal interaction with humans.
Date/Location
Febuary
1999
Silver Lake, CA
Challenge
Scenarios
Construction tasks
September
2000.
Cinder Lake,
Meteor Crater,
and SP Mountain, AZ
Solar Panel Deploy.
Power cable deploy.
Geology assistant.
Stereo-vision tracking
September,
2002.
Meteor crater
and
Joseph City, AZ
Geophone
Science Collection tasks.
Initial testing of Thibodeaux. Deploy/Assistance.
Geology Assistant.
Five minute comm delay
over satellite. Fuel cell
testing. Interface testing with
Mobile Agents infrastructure
April
2003.
Hanksville
(MDRS), UT
Integration with Mobile
Agents Infrastructure.
Terrain with hills and
canyons, distance, wireless
communications
Science Collection tasks.
Interface testing with Mobile
Agents infrastructure. Initial
testing of a SCOUT vehicle
Conquer Lith Canyon. Involve
Remote Science teams
September,
2003.
Meteor crater,
AZ
May
2004.
Hanksville
(MDRS), UT
Robot
Results
Astronaut spent a great amount of
Teleoperated rover as Ames'
Initial field test with robot
Marsokhod time waiting for robot to catch up
scout, videographer,
and suited test subject in
analog environment. Robot field science assistant
following using stereo-vision
Lessons Learned
Resulting Improvement
Robot too slow. Ground clearance severely limits robot Purchase new mobile robotic platform. Increase
ground clearance
mobility in tested terrain: need minimum of 10 inch
clearance
Boudreaux Major success in all scenarios
Need alternative methods of tracking astronaut. Need Differential GPS on robot. Infopak with dpgs for
astronaut. Begain new robot architecture based on
absolute localization method. Need better battery
management. Need better voice and data comm. Need CORBA, C++, Linux. Start Thibodeaux
to replace Mobility Software since cannot change it as
required.Robotic base needs improving: shock
mounting, more speed, better traction, stronger. Need
arm+hand manipulator to improve HRI.
Boudreaux, Failed to pull heavy geology trailer
Thibodeaux uphill
Need a single-point of interaction with robots instead of
exposing many lower-level processos to external
collaborators. Need improved arm hardware and
kinematics. Need better speech recognition and
natural language understanding
Added second GPS unit to obtain heading estimate,
COMM outages require robot autonomy to recover,
many Hardware issues required a power cycle to fix, added a "power box" to allow computer control of
"hot swap" of batteries would allow for longer missions. power to several devices, modified power bus to
allow 2 strings of batteries to be used in parallel
(allowing for hot swaps)
Need better power regulation/monitoring of power bus. Improved power buss to eliminate power spikes from
Failed to pull heavy geology trailer
uphill, several hardware failures due to
the motors from reaching other devices
poor voltage regulation of motor
"noise".
Major success in all scenarios.
Need more field spares. Need better e-kill. Need global Acquired more field spares. Purchased improved einformation distributed to each robot. Robots need to kill. Added data distribution functionality to
architecture. Refactored/multi-threaded executor
be able to perform multiple parallel behaviors. Need
and high-level behaviors to allow pause,resume.stop,
better power information. Need better data comm.
Obstacle Avoidance must not count tracked astronaut prioritized execution of multiple simultaneous
behaviors. Added battery meter. Replaced wireless
comm system with more powerful mesh-technology
t
Modified
the motor controller interface for Thibodeaux
Funding and schedule must generously allow for
Significant hardware and software
Geology assistant.
Comm Relay. Still
imagery
Boudreaux Periodic COMM outages temporarily
prevented remote commanding,
heading estimate was not accurate
enough for tracking
Geology Assistant.
Spectrograph data
collection
Boudreaux,
Thibodeaux
Boudreaux
Geology assistant.
Comm Relay. Video
Survellience,
Panoramic Image
Generation, situational
awareness
Deployment of COMM Boudreaux,
Thibodeaux issues. Arm broken prior to RDR
Repeater,
deploy. Thibodeaux unused.
Still/Panorama Image
Significant effort required to debug and
capture, Autonomous
test refactored high-level interface to
scouting, Geology
mobile agents. Used many field spares
assistant
April
2005.
Hanksville
(MDRS), UT
Mult-robot cooperation.
Teamwork model. Multiple
simultaneous behaviors
May
2005.
JSC Sites
Stereo-vision tracking, SCOUT
Test SCOUT vehicle with
space-suited, onboard crew autonomous point-topoint navigation,
on long-distance trek.
obstacle avoidance,
suit ingress/egress,
teleoperation
Created interface spec and a single-threaded
“executor” to handle interfacing with external agents.
Arm hw & sw upgraded, 6axis force-torque sensor
acquired. Replaced ViaVoice with Nuance and
collaborated with experts on more powerful and
flexible grammar
robotic development and field test preparedness (i.e.
don’t plan on development in the field). Flexibility and
contingency plans are required for continuing field
tests in the face of failures and weather.
Suited Astronut tracking using stereo Startup/Shutdown proceedures need to be more
simple (and always followed). Antenna height is very
vsion, obstacle avoidance was
confused by vegitation, velocity control important to maintaining COMM coverage
was not properly tuned
Table 2. ERA Field Test History
to be simpler and more reliable.
Reworked velocity control loop to better maintain
speed
4.4.1 Directional Knowledge
Directional knowledge can be simple (following a compass heading from its sensor) or
complex (following a human). Human following may not require any prior terrain knowledge on
the part of the robot, but takes advantage of the human capability of picking out a navigable trail
quickly and easily, even in new terrain. The robot’s job is just to follow the human. Stereo vision
and laser sensors have been used to perform human following, and robotic intelligence is needed
to process the raw data to filter out the background and identify the human. This can be
accomplished by any number of vision systems/techniques, but whichever vision system is
selected needs to be reliable and operate in all conditions. This means that the system should
work in low and high light conditions, for astronauts of varying heights, and suits of various
colors (since dirt will change the suits appearance over time). Additionally, the human tracking
system needs to track the human in cluttered environments, not just in open areas.
Autonomous obstacle avoidance can be performed by the robot, if desired, to make certain that
the ground the astronaut chooses is also safe for the robot. This not only allows the robot to
ensure its own safety, but also allows the human to act more freely, without considering the
terrain limitations of the robot. This enables the human to concentrate on his or her own work,
limiting the amount of time the crew needs to focus on the robot. By preventing robots from
hindering or burdening human crew members, robots can provide added capability and efficiency
rather than added work. Indeed, some of the strongest feedback we received from our first field
test with a suited astronaut concerned the speed of the robot. Because the robot traveled so slowly
compared to the human, the human frequently had to wait for the robot to catch up. Making
robots an asset, and not a liability, on planetary missions will be of utmost importance for
astronauts to accept them as team members.
Current obstacle avoidance capabilities require slower motion on the part of the robot, and are
not guaranteed in all terrain situations. Though still a work in progress, obstacle avoidance has
been successfully used in our Utah testing, and has made apparent the desirability of a more
autonomous, intelligent robot during joint human-robot activities. Again, the obstacle avoidance
system needs to be reliable, since an error could lead to the loss of the robot. During our field
tests, we have a human safety officer watching the robot at all times who has the ability to
remotely halt the robot at any time if the robot makes an error in path selection. This “safety net”
will obviously not be available to a real planetary robot, so planetary robots will need to operate
in a manner that ensures safe behavior at all times.
4.4.2 Task Knowledge
Task knowledge can require many different types of intelligence. For the geophone
deployment by the manipulator arm, the robot needed to know how to grasp and move the sensor
in order to securely place it in the ground. For the science sensor pulled along in a trailer, the
robot needed to know how to perform the search pattern. These are typical robotic activities, but
they do require intelligence, especially when working in outdoor, unstructured terrain. For
instance, performing a search pattern requires knowledge of the capabilities of the robot, such as
how precisely it is able to maneuver. Exact following of a path is often impossible due to small
rocks or other terrain features which push the robot off course, or cause it to lose traction. Some
level of intelligence is needed to enable the robot to decide when to proceed onward to the next
leg of the pattern. This decision point includes knowledge of the robot’s location relative to the
path as well as the time it has taken to reach its current location. Robotic intelligence enables the
robot to decide whether to continue along the current portion of the search pattern, to proceed on
to the next leg despite not having finished the current leg, or to abort the pattern completely.
4.4.3 Human Interaction
Physical and verbal interaction with humans is a key component of our research. Field tests
have shown what humans expect from a robot, and what they require in order to work together
effectively with a robot. Safety is a primary concern, and so human following includes a buffer
zone around the robot. If the human moves too close, the robot will back away. Due to a lack of
verbal communication with the robot, this behavior produced frustration on the part of the human
crewmember during one field test. The robot was pulling a trailer with tools, and when the
astronaut needed to use one of those tools, he approached the robot, but without telling the robot
to stop following him first. So the robot continually backed away from the astronaut when he
tried to approach.
Proper verbal interaction would have prevented this situation. The robot needed the
intelligence to respond to any of several phrases which meant “stop,” instead of only responding
to one particular word. The robot might also need to express its current task or behavior mode
more frequently in order to let the human know what the robot thought it was doing. On the other
side, though, continual utterances of the robot’s thought processes would annoy the human crew
members so much that they would turn off their radios and not listen to the robot anymore.
Additional intelligence is needed to enable the robot to behave in a more human manner when
interacting with humans. Awareness of the current situation and desires of the human would help
the robot in adapting its behavior to the moment. This is one of the goals of the Utah test
participants.
Communication dropouts during Utah field testing have shown the need for more autonomy
and intelligence on the part of the robots. A robot must be able to carry out its currently assigned
task, such as traveling to a given waypoint, without having a communication link to the habitat or
anywhere else. Often, short or even long term communication dropouts have occurred in our field
tests while the robot was in transit. The robot must be able to continue either to its goal, or to
return to its original location in order to regain communication. Thus, we have developed our
software architecture to provide onboard all the information the robot needs to complete its
traverse to a given goal point: self-localization, goal location, navigation, and obstacle
avoidance. In the future, the goal point itself can be determined by the robot when needed.
Preliminary work has been done to enable the robot to determine a new goal point based on
communication signal strength.
When performing a communication relay function between the habitat and the EVA crew
members, the robot also must possess enough intelligence to maintain a link with both sides.
Onboard intelligence is obviously needed in this case, as the crucial decisions must be made when
the robot is in communication with any humans. The benefit of using a robot for this task instead
of a fixed relay is that the robot can maneuver as needed when the astronauts change position.
The winding canyons of Utah have shown that a single relay is not sufficient for covering all the
areas the astronaut-geologists wish to explore. A robot provides a more reasonable solution than
having innumerable fixed relays to cover every angle. The goal of such an autonomous capability
is that communication between remote field teams and a habitat is seamless and unnoticed – the
humans do not have to worry or think about this task, but just allow the robots to perform their
duty.
Finally, interaction between robots and humans is a primary area of research in the Utah field
tests. The robots need to act as a member of the crew, responding to verbal commands, providing
feedback, and not annoying or interfering with the human crew so much that the robots prove
more of a hindrance than a help [34]. It is clear that individual humans prefer to hear different
amounts of communication from a robot during EVA’s. Some want to know every move the
robot is going to make, and some want to hear only high level data. Allowing a robot to be
configured (or even better, to learn and predict) to communicate in a mode that is preferred by
each astronaut will allow the robot interact more successfully with humans. To aid in this goal,
intelligence is needed in the form of a better understanding of human interaction, and even human
motivation, etiquette, and goals. The NASA Ames Mobile Agents group has been doing research
in this area for some time ([35, 36]), and our joint field tests have been a proving ground for
much of this work.
5. DISCUSSION
There are many issues raised by including human-robot teams in the exploration of planetary
surfaces. The safety of the crew must be considered above all other concerns. While robotic
intelligence can add to the safety of robot operations, the complexity involved also introduces
additional risks, due to incomplete testing of all possible situations and due to the inherent fact
that the robots will be operating more on their own. The safe operation of the robot can be
ensured (as much as is possible) by performing extensive testing and development of the robots
here on Earth, and by having multiple levels of safety built into the robots. As an example, we
have both hardware and software “kill” capabilities on our robots. The software can tell the robots
to stop moving, but if for some reason the software malfunctions or is unable to command a stop,
the hardware can physically remove electrical power from the motors. “Kill buttons,” usually
large red buttons, can be pressed even by space suited personnel, and provide an additional path
to stop the robot in case any primary control mechanisms should fail. Having redundant methods
for commanding (or stopping) a robot will be a requirement for all planetary robotic assistants.
The robots discussed in this chapter are all designed to be robotic testbeds, and as such are not
intended to be actual flight units. These robots replicate the functionality of actual flight units, but
avoid the added cost and time involved with building an actual flight unit. This is a fundamental
design decision, and one that we feel allows our research to move forward in a more productive
manner. Since we are testing new robotic technologies, we are helping to determine which
techniques are useful and feasible, and which are not. By determining which functions are most
promising, we can help to ensure that when the final flight units are constructed, they will be as
capable as possible.
6. CONCLUSIONS AND FUTURE WORK
We have acquired a basic understanding of the types and functionalities of robots needed to
successfully assist astronauts in the exploration of planetary surfaces. However, much work
remains to be done in successfully utilizing human robot teams effectively [39]. Additional field
testing needs to be performed on Earth, with increasingly realistic scenarios, environments, and
robots. As plans for actual future missions solidify, analog field tests must simulate those
missions. Test participation will be required from all types of personnel expected to take part in
those missions. ERA-type robots must continue their iterative improvement, evolving from Earthbased test platforms into space-qualified hardware.
Our near-term plans include both hardware and software related improvements. In terms of
hardware, we plan to integrate an improved seven degree-of-freedom manipulator arm, enabling
additional mission scenarios. We will also utilize two complete robots in cooperative and parallel
field test activities.
In terms of software and intelligence, we will be integrating gestures as an additional form of
HRI. We will also be developing a Vehicle Health Management system to improve robustness
and allow for operation in degraded modes. We will be adding more deliberative processes, such
as a sequencer and a planner. We will be improving behaviors and algorithms for functionalities
such as obstacle avoidance. Finally, we will be developing and recording more detailed metrics,
both for HRI and autonomy evaluation. Some simple examples of HRI metrics include:
• number of commands given (True Positives, True Negatives, False Positives, False
Negatives)
• number of confirmations required
• number of gestures given and/or recognized
• distance humans are tracked
• number of scenarios accomplished
We are aware of the current research of Goodrich [6], Fong [7], Scholtz [8], and Rodriquez [37],
and we would like to include their HRI metrics as well. To facilitate the collection of HRI
metrics, composite (complex) ERA commands and behaviors will be decomposed into more
elemental “primitive” forms, then layered and instrumented to facilitate the collection of metrics.
Examples of autonomy metrics include:
• distance traveled autonomously
• number of waypoints traversed
• number of human interventions required
• number of hardware failures
• number of battery swaps
• number of device upsets/resets
• number of software upsets/resets
• number of times the robot is remotely disabled by the safety officer (E-stops)
As our robotic development continues with an eye toward lunar and Martian missions [38],
several key points need to be kept in mind:
1. Reliability
Robots must be reliable to be useful on the moon or Mars. A robot that is not operational
might as well not be on the surface, because it will not help complete any mission goals.
We need to ensure the robots are dependable team members, or else humans will be
forced to operate without them. Reliable robots will also have the added benefit of lower
overhead associated with their maintenance.
2. KISS Principle (Keep It Simple, Stupid)
Keep the robots simple. Don’t try to make them do everything. A concerted effort is
needed to ensure that “requirements creep” does not make individual robots too
complicated, too fragile, or too expensive.
3. Modularity
The ability to share components, software, and procedures between robots will greatly
decrease cost and training requirements, as well as allow robots to share spare parts. With
such modularity, astronauts will be able to control and maintain all the various robots
with a minimum of training.
4. Human Safety
Robots working side by side with humans, or performing mission critical operations, will
need to have their software and hardware verified to be safe and fault tolerant.
Additionally, if these robots provide functionality to humans such as carrying them,
providing recharging of space suit consumables, or acting as a communications relay,
then the mission plans will need to include contingency plans in case of robot failures.
5. High Level Autonomy
Higher levels of autonomy will help avoid low-level and highly monotonous human
control of robots. This is especially important when communication time delays are
present.
6. Spiral Development
Future robotic missions can always learn from prior missions. Just as we learn from
Earth-based field testing, lunar missions should not be viewed as only an end unto
themselves. They are also as a way of learning more about the true usefulness and
capability of robots to be sent to Mars.
7. ACKNOWLEDGEMENTS
We would like to acknowledge the support of our field test partners, as mentioned in section
4.3. In particular, John Dowding and Rick Alena (NASA/Ames), and Joe Kosmo (NASA/JSC)
and their research groups have contributed greatly to our work and are integral to our success.
This research has been funded by JSC CDDF, Code R (IS – Intelligent Systems program, Office
of Aerospace Technologies), Mobile Agents, and ESMD (Exploration Systems Missions
Directorate).
REFERENCES
[1] R. Burridge and J. Graham, “Providing robotic assistance during extra-vehicular activity”,
Proc. SPIE Vol. 4573, pp. 22, Boston, Massachusetts, USA 2002.
[2] R. R. Murphy, Introduction to AI Robotics, A Bradford Book, MIT Press, Cambridge,
Massachusetts, USA, 2000, pp. 15-16.
[3] R. C. Arkin, Behavior-Based Robotics, MIT Press, Cambridge, Massachusetts, USA, 1998.
[4] F. Slakey and P. Spudis, “Robots vs. Humans: Who Should Explore Space?”, Scientific
American, Vol. 10, 1999, pp. 24-31.
[5] “Robot & Human Surface Operations on Solar System Bodies”,
http://www.lpi.usra.edu/publications/reports/CB-1089/weisbinrobot.pdf
[6] M. Goodrich, D. Olsen, Jr., “Seven Principles of Efficient Human-Robot Interaction”, Proc.
IEEE International Conference on Systems, Man and Cybernetics, 2003, pp. 3943-3948.
[7] T. Fong et al.,”Common Metrics for Human-Robot Interaction”, Proc. IEEE International
Conference on Intelligent Robots and Systems, Sendai, Japan, 2004.
[8] J. Scholtz, “Theory and Evaluation of Human-Robot Interaction”, Proc. 36th Hawaii
International Conference on System Sciences, 2003.
[9] D. Kortenkamp and L. Pedersen, “Worlds To Explore: An Introduction to Current Capabilities
and Key Challenges in Space Robotics”, Tutorial at IROS, Lausanne, Switzerland, 2002.
[10] L. Pedersen, D. Kortenkamp, D. Wettergreen and I. Nourbakhsh, “A Survey of Space
Robotics”, Proc. 7th International Symposium on Artificial Intelligence, Robotics and
Automation in Space (i-SAIRAS-03), Nara, Japan, 2003.
[11] C. Martin, D. Schreckenghost, P. Bonasso, D. Kortenkamp, T. Milam and C. Thronesbery,
“Helping Humans: Agents for Distributed Space Operations”, Proc. 7th International
Symposium on Artificial Intelligence, Robotics and Automation in Space (i-SAIRAS-03),
Nara, Japan, 2003.
[12] W. Clancey, “Principles for integrating Mars analog science, operations, and technology
research,” Workshop on Analog Sites and Facilities for the Human Exploration of the Moon
and Mars, Golden, CO, USA 2003.
[13] J. Clarke and G. Mann, “Jarntimarra 1: Selecting an Australian Mars Analogue Research
Site,” Proc. 2nd Australian Mars Exploration Conference, New South Wales, Australia, 2002.
[14] P. Lee, “Mars on Earth: The NASA Haughton-Mars Project,” SpaceRef.com, 2002, see
website: http://www.spaceref.com/news/viewnews.html?id=459
[15] F. Rehnmark, W. Bluethmann, J. Mehling, R. O. Ambrose, M. Diftler, M. Chu, R.
Necessary, “Robonaut: The ‘Short List’ of Technology Hurdles,” Computer (IEEE Computer
Society), Jan 2005, pp. 28-37.
[16] R.O. Ambrose et al., “Mobile Manipulation Using NASA's Robonaut,” Proc. IEEE
International Conference on Robotics and Automation (ICRA 2004), Vol. 2, New Orleans,
Louisiana, USA 2004, pp. 2104-2109.
[17] B. Pell, D. Bernard, S. Chien, E. Gat, N. Muscettola, P. Nayak, M. Wagner, and B. Williams,
“Remote Agent: To Boldly Go Where No AI System Has Gone Before”, Artificial
Intelligence, Vol 5, Issue 1, March, 1998.
[18] R. Bonasso, R. Firby, E. Gat, D. Kortenkamp, D. Miller, and M. Slack, “Experiences with an
Architecture for Intelligent, Reactive Agents”, Journal of Experimental and Theoretical
Artificial Intelligence, January, 1997.
[19] W. J. Clancey, M. Sierhuis, R. Alena, D. Berrios, J. Dowding, J. S. Graham, K. S. Tyree, R.
L. Hirsh, W. B. Garry, A. Semple, S. J. Buckingham Shum, N. Shadbolt, and S. Rupert,
“Automating CapCom Using Mobile Agents and Robotic Assistants,” Proc. 1st Space
Exploration Conference, Orlando, FL, USA, 2005.
[20] W. J. Clancey, P. Sachs, M. Sierhuis, and R. van Hoof, “Brahms: Simulating practice for
work systems design,” International Journal on Human-Computer Studies, Vol. 49, 1998, pp.
831-865.
[21] M. Sierhuis, “Modeling and Simulating Work Practice; Brahms: A multiagent modeling and
simulation language for work system analysis and design,” University of Amsterdam, The
Netherlands, SIKS Dissertation Series No. 2001-10, 2001.
[22] M. Sierhuis, A. Acquisti, and W. J. Clancey, “Multiagent Plan Execution and Work Practice:
Modeling plans and practices onboard the ISS,” Proc. 3rd International NASA Workshop on
Planning and Scheduling for Space, Houston, TX, USA, 2002.
[23] L. S. Vygotsky, Mind in Society: The Development of Higher Psychological Processes,
Harvard University Press, Cambridge, Massachusetts, USA, 1978.
[24] W. J. Clancey, “Simulating Activities: Relating Motives, Deliberation, and Attentive
Coordination,” Cognitive Systems Research, Vol. 3, 2002, pp. 471-499.
[25] W. J. Clancey, Situated Cognition: On Human Knowledge and Computer Representations,
Cambridge University Press, Cambridge, Massachusetts, USA, 1997.
[26] L. A. Suchman, Plans and Situated Action: The Problem of Human Machine
Communication, Cambridge University Press, Cambridge, Massachusetts, USA, 1987.
[27] L. Gasser, “Social conceptions of knowledge and action: DAI foundations and open
systems,” Artificial Intelligence, Vol. 47, 1991, pp. 107-138.
[28] D. N. Pynadath and M. Tambe, “An Automated Teamwork Infrastructure for Heterogeneous
Software Agents and Humans,” Autonomous Agents and Multi-Agent Systems, Vol. 7, 2003,
pp. 71-100.
[29] FIPA, “FIPA Communicative Act Library Specification,” Vol. 2004: Foundation for
Intelligent Physical Agents, 2002.
[30] J. Dowding and E. Campana, “Spoken Dialogue Systems in MDRS Rotation 29,” Proc. 7th
International Mars Society Convention, Chicago, Illinois, USA, 2004.
[31] A. Barr , E. A. Feigenbaum, Handbook of AI, Addison-Wesley Longman Publishing Co.,
Inc., Boston, MA, USA, 1986.
[32] R. Alena, B. Gilbaugh, B. Glass, and S. P. Braham, “Communication System Architecture
for Planetary Exploration”, Proc. IEEE Aerospace Conference, 2001.
[33] R. Beck et al., “A space-based end-to-end prototype geographic information network for
lunar and planetary exploration and emergency response (2002 and 2003 field experiments),”
Computer Networks, Volume 47, Issue 5, Pages 765-783.
[34] D. Kortenkamp, D. Schreckenghost, and C. Martin. “User Interaction with Multi-Robot
Systems”, Proc. Workshop on Multi-Robot Systems, Washington, DC, USA, 2002.
[35] W. Clancey., M. Sierhuis., C. Kaskiris, and R. van Hoof, “Advantages of Brahms for
Specifying and Implementing a Multi-agent Human-Robotic Exploration System”, Fifteenth
International FLAIRS Conference, St. Augustine, Florida, USA, 2003.
[36] M. Sierhuis, M. Sims, W. Clancey, and P. Lee, “Applying multi-agent simulation to
planetary surface operations”, Proc. COOP' 2000 Workshop on Modeling Human Activity,
Saint-Rafaël, France, 2000.
[37] G. Rodriguez, C. Weisbin, R. Easter, “A New Method to Compare the Performance of
Humans, Robots and Integrated Human-Robot Systems in the 21st Century Space Operations”,
JPL Publication 01-13, Pasadena, CA, USA, October 2001.
[38] J. Gruner, "Lunar Surface Scenarios Study", Exploration Systems Mission Directorate task
RFT0019.05JSC, JSC Publication, Houston, TX, USA, May 2005.
[39] Dr. Ayanna Howard , “Applying Computational Intelligence Techniques for Assessment of
Human-Robot Interactions for Space Missions”, this volume.
[40] Fred Rehnmark, “Characteristics Common to the Utility of Robotic Astronaut Assistants”,
this volume.
[41] Keller, R. M., et al., “SemanticOrganizer: A Customizable Semantic Repository for
Distributed NASA Project Teams,” Int'l Semantic Web Conf. (ISWC-2004), Hiroshima,
Japan, 2004, pp. 767-781.
Field Testing of Intelligent Planetary Surface Robots
Jeffrey Graham, Robert Hirsh, Dr. Kimberly Tyree
NASA Johnson Space Center, Houston, TX
Robots will require intelligence to succeed in the uncertain and changing
environment on lunar and planetary surfaces. Even with humans directly involved in
controlling such robots, intelligence is still needed. Field testing here on Earth has
demonstrated this need, and this chapter will describe the results and lessons learned
from over six years of testing human-assistant mobile robots in field settings relevant
to planetary exploration.
Planetary-analog sites here on Earth are chosen for various reasons. Basically, the
terrain and environment should be applicable to the tests to be performed. For
mobility tests, rough terrain is needed. For communication tests, remote areas with
obscuring hills and hidden canyons may provide the desired challenges. For logistical
tests of science and exploration goals, areas with interesting science are needed.
In addition to an applicable environment, remoteness in general can provide an
added benefit for field testing: the enforcement of self-sufficiency. Identifying what
equipment, communications, and capabilities are needed for a remote space robot is
one of the primary accomplishments of many field tests, often providing surprising
results. Challenges are frequently uncovered that would not be found in a laboratory
or simulation setting, particularly relating to terrain, logistics, and human interaction
needs. Often, robot intelligence provides a solution to these challenges.
Aside from providing a testing environment for robots, field testing provides
operational training for humans. For future missions, humans may control robots
from a long distance (such as from Earth), from a nearby but still remote location
(such as from an orbital station or ground habitat), or from the immediate
neighborhood. All three control locations present different requirements, and all three
need to be tested in the field. By participating in field testing, not only can
operational crew can gain an understanding of what is needed, but researchers can
gain an understanding of what interfaces and intelligent capabilities will be needed in
the future.
Specific case studies of field testing performed by the Extravehicular Activity
(EVA) Robotic Assistant, or ERA project, will be described, including tests in
Arizona and Utah. The types of tasks performed include construction and habitatrelated logistics, science and exploration, interaction with high-fidelity space-suited
personnel, human crew integration, safety monitoring, communication studies, and
others. The lessons learned from these field tests will be detailed, focusing on the
need for robotic intelligence. Finally, future field testing needs will be identified,
with the goal of developing intelligent robots which will enable and enhance the
human exploration of space.