Cisek and Kalaska 2010 - Neural Mechanisms For Interacting With A World Full of Action Choices

NE33CH13-Cisek ARI 22 May 2010 19:1
ANNUAL
REVIEWS Further Neural Mechanisms for
Click here for quick links to
Annual Reviews content online,
including:
Interacting with a World Full
• Other articles in this volume
• Top cited articles of Action Choices
Access provided by WIB6113 - Universitaet Osnabrueck on 05/26/21. For personal use only.
• Top downloaded articles

Annu. Rev. Neurosci. 2010.33:269-298. Downloaded from www.annualreviews.org
• Our comprehensive search

Paul Cisek and John F. Kalaska
Groupe de Recherche sur le Système Nerveux Central (FRSQ), Département de
Physiologie, Université de Montréal, Montréal, Québec H3C 3J7 Canada;
email: paul.cisek@umontreal.ca
Annu. Rev. Neurosci. 2010. 33:269–98 Key Words

First published online as a Review in Advance on neurophysiology, sensorimotor control, decision making, embodied
March 26, 2010
cognition
The Annual Review of Neuroscience is online at
neuro.annualreviews.org
Abstract
This article’s doi:
10.1146/annurev.neuro.051508.135409
The neural bases of behavior are often discussed in terms of perceptual,
cognitive, and motor stages, defined within an information processing
Copyright c 2010 by Annual Reviews.
All rights reserved
framework that was originally inspired by models of human abstract
problem solving. Here, we review a growing body of neurophysiolog-
0147-006X/10/0721-0269$20.00
ical data that is difficult to reconcile with this influential theoretical
perspective. As an alternative foundation for interpreting neural data,
we consider frameworks borrowed from ethology, which emphasize the
kinds of real-time interactive behaviors that animals have engaged in for
millions of years. In particular, we discuss an ethologically-inspired view
of interactive behavior as simultaneous processes that specify potential
motor actions and select between them. We review how recent neuro-
physiological data from diverse cortical and subcortical regions appear
more compatible with this parallel view than with the classical view of
serial information processing stages.
269
movement. Neural correlates of putative

Contents decision variables (such as payoff) appear
to be expressed by the same neurons that
INTRODUCTION . . . . . . . . . . . . . . . . . . 270
encode the attributes (such as direction) of
COGNITIVE NEUROSCIENCE
the potential motor responses used to report
AND THE INFORMATION
the decision, which reside within sensorimotor
PROCESSING FRAMEWORK . . . 271
circuits that guide the on-line execution of
Perceptual Processing . . . . . . . . . . . . . . 272
movements. These data and their implications
Motor Control . . . . . . . . . . . . . . . . . . . . . 273
for the computational mechanisms of decision
Cognitive Functions . . . . . . . . . . . . . . . 273
making have been the subject of several recent
AN ECOLOGICAL
reviews (Glimcher 2003, Gold & Shadlen
PERSPECTIVE . . . . . . . . . . . . . . . . . . . 275
2007, Schall 2004). Here, we consider more
REVISITING NEURAL DATA . . . . . . 279
general questions of what these and other

Frontoparietal Specification
recent data imply for large-scale theories of the

of Potential Actions . . . . . . . . . . . . . 279
neural organization of behavior. Why should
Action Selection Signals from
supposedly cognitive processes take place
Prefrontal and Subcortical
within sensorimotor circuits? Why should in-
Sources . . . . . . . . . . . . . . . . . . . . . . . . . 283
dividual neurons appear to change in time from
Parallel Operation . . . . . . . . . . . . . . . . . 285
encoding sensory qualities to encoding motor
FUTURE DIRECTIONS . . . . . . . . . . . . 288
parameters? What should be the time course
CONCLUSIONS . . . . . . . . . . . . . . . . . . . . 289
of neural processing? Many recent results do
not appear to be compatible with the classical
distinctions between perceptual, cognitive, and
motor systems (Lebedev & Wise 2002); and we
INTRODUCTION consider whether an alternative framework can
In this review, we discuss some potential impli- more readily account for these observations.
cations of recent neurophysiological results to As an alternative perspective, we con-
large-scale theories of behavior. We focus pri- sider frameworks inspired by many decades
marily on data from the cerebral cortex of non- of ethological research focused on natural
human primates, data that are often interpreted animal behavior in the wild. Such behavior
in terms of a theoretical framework influenced involves continuous sensorimotor interaction
by studies of human cognition. In that frame- between an organism and its environment,
work, the brain is seen as an information pro- in contrast with conditions often used in the
cessing system that first transforms sensory in- laboratory in which time is divided into a
formation into perceptual representations, then series of individual trials. Because the brain’s
uses these to construct knowledge about the functional architecture originally evolved to
world and make decisions, and finally imple- serve the needs of interactive behavior, and
ments decisions through action. was strongly conserved during phylogeny, we
However, neurophysiological data we believe an ethological foundation may be more
review below appear to be at odds with many appropriate for understanding neurophysi-
of the assumptions of this influential view. For ological data about voluntary sensorimotor
example, studies on the neural mechanisms of behavior compared to frameworks inspired by
decision making have repeatedly shown that studies of advanced human abilities. Indeed,
correlates of decision processes are distributed we believe that a wide variety of neurophys-
throughout the brain, notably including cor- iological results are more readily interpreted
tical and subcortical regions that are strongly within the perspective of ethologically-based
implicated in the sensorimotor control of theories.
270 Cisek · Kalaska

COGNITIVE NEUROSCIENCE system is formalized in a theoretical framework

AND THE INFORMATION that has dominated psychological thinking and
PROCESSING FRAMEWORK teaching for more than 50 years.
Motor program:
Information processing was established a theoretical
Imagine yourself sitting at a computer, reply- as the theoretical foundation of cognitive representation of
ing to a friend’s email message. From the out- psychology during the mid-twentieth cen- planned movements
side, the act looks simple: A person is seated, tury, when it replaced the then-dominant
hardly moving their body, looking at a com- paradigm of behaviorism. From the infor-
puter screen. After some time, fingers begin mation processing perspective, perception in-
to tap some keys on the keyboard. Of course, volves the construction of increasingly sophis-
on the inside, this behavior involves a large set ticated and abstract internal representations
of incredibly complex processes. These include of the world (Marr 1982) that are used as
visual recognition of the letters displayed on the input to cognitive systems. These cogni-
the screen, parsing of the words and sentences,
tive systems bring salient context-dependent

analysis of their meaning, emotional reactions information together in a temporary work-
to it, consideration of the many factors that in- ing memory buffer (Miller 1956), manipulate
fluence the nature of the answer and how to representations to build complex knowledge
phrase the reply, and finally, the production of ( Johnson-Laird 1988, Pylyshyn 1984), store
precise finger movements to produce a new set and retrieve information from long-term mem-
of letters. We can classify these processes into ory (Newell & Simon 1972), perform de-
three general categories: perception—the pro- ductive reasoning (Smith & Osherson 1995),
cesses that take information from the outside and make decisions (Shafir & Tversky 1995,
world to build knowledge about it; cognition— Tversky & Kahneman 1981). Finally, the mo-
the internal processes of knowledge manip- tor systems are seen simply as tools that
ulation, including semantic analysis, decision implement action plans chosen by cognitive
making, etc.; and action—the control of the processes. They are often conceived accord-
muscular contractions that produce our re- ing to formalisms borrowed from engineering
sponse. In tasks such as replying to an email, control theory, in which a predetermined mo-
these processes are likely performed in a largely tor program or desired trajectory (Keele 1968,
serial manner. We sense the world, think about Miller et al. 1960) is passed to a controller that
it, and then act upon it. executes it via feedforward and feedback control
This informal description of behavior is of- mechanisms.
ten reflected in how we study it in the lab. Per- This classical framework was originally
ceptual scientists study how the brain processes proposed as an explanation of complex human
information to produce internal representa- abilities of abstract problem solving, such as
tions of external phenomena. Cognitive scien- chess playing (Newell & Simon 1972, Pylyshyn
tists study, among other things, how knowledge 1984)—the kinds of problems that require the
is acquired, how memories are stored and re- subject to obtain information about the world
trieved, and how abstract decisions are made and perform a great deal of computation before
between distinct choices such as whether to try taking any external actions. It was not originally
gamble A or B. Motor scientists like us study meant to be a general theory of all behavior.
how a voluntary plan of action is transformed Eventually, however, the architecture of an
into patterns of muscular contraction that move information processing system was seen as so
our limbs or eyes or produce utterances. This powerful that its basic concepts have come to
division of labor among neuroscientists re- influence nearly every domain of brain theory.
flects how we think about behavior and influ- For example, the concept of the bandwidth of
ences how we teach brain science. Indeed, the transmission in information processing chan-
view of the brain as an information processing nels has provided a foundation for theories
www.annualreviews.org • Neural Mechanisms for Interactive Behavior 271

of attention (Broadbent 1958) and working stable at the level of single neurons or neural
memory (Miller 1956), as well as studies of populations.
motor control (Fitts 1954). The project of For example, Ungerleider & Mishkin (1982)
Cognitive
neuroscience: the explaining complex behavior in terms of neural reviewed data indicating that visual informa-
study of the neural mechanisms is often called cognitive neuro- tion in the cerebral cortex diverges into two
substrates of complex science (Albright et al. 2000, Gazzaniga 2000). partially distinct streams of processing: (a) an
behavior, usually based It is an approach that inherits specific concepts occipitotemporal ventral stream in which cells
on concepts from
from cognitive psychology and maps them onto are sensitive to information that pertains to the
cognitive psychology
particular regions of the brain. Today, even identity of objects and (b) an occipitoparietal
Ventral stream: a
those of us who study sensorimotor control dorsal stream in which cells are sensitive to spa-
visual processing
pathway from the have a tendency to phrase the problem as one of tial information. Within each of these, informa-
occipital cortex along transforming input representations to output tion diverges further. There are separate visual
the temporal lobe representations through a series of intermediate streams for processing color, shape, and mo-
Dorsal stream: a processing stages. tion (Felleman & Van Essen 1991); and there
visual processing However, attempts to interpret neural data are multiple representations of space within the
pathway from the from this perspective encounter several chal- posterior parietal cortex (Colby & Goldberg
occipital cortex and
lenges. For example, functions that should 1999, Stein 1992). From the traditional cog-
colliculi to posterior
parietal cortex be unified appear distributed throughout the nitive perspective, the ventral stream builds a
brain, whereas those that should be distinct ap- representation of what is in the environment,
pear to involve the same regions, or even the whereas the dorsal stream builds a represen-
same cells. Below, we discuss several examples tation of where things are. Presumably, all of
of such challenges, which lead us to question these visual substreams must be bound together
whether the basic structure of sensing, thinking, to form a unified representation of the world;
and acting is indeed the optimal blueprint for but whether and how this binding occurs re-
understanding how the brain implements much main unresolved, despite vibrant research ef-
of the real-time interactive behavior whose de- forts (Engel et al. 2001, Shadlen & Movshon
mands drove neural evolution. 1999, Singer 2001).
Furthermore, activity in much of the visual
system appears to be strongly influenced
Perceptual Processing by attentional modulation (Boynton 2005,
Psychological and computational theories Colby & Goldberg 1999, Moran & Desimone
often propose that our perception of the world 1985, Treue 2001), even when a quiescent
is the result of a reconstruction process that monkey spontaneously scans a familiar stable
uses sensory information to build and update environment (Bushnell et al. 1981, Gottlieb
an internal representation of the external world et al. 1998, Mountcastle et al. 1981). This is
(Marr 1982, Riesenhuber & Poggio 1999, usually exhibited as an enhancement of neural
Riesenhuber & Poggio 2002). We usually activity from the regions of space to which
assume that this internal representation must attention is directed and a suppression of activ-
be unified (linking diverse information into ity from unattended regions. Such attentional
a common form available to diverse systems) modulation is found in both the ventral and
and stable (reflecting the stable nature of the dorsal streams and increases as one ascends the
physical world) to be useful for building knowl- visual hierarchy (Treue 2001). Consequently,
edge and making decisions. To date, however, the neural representation of the visual world, at
neural data do not support the existence of least in higher visual areas, appears “dominated
such an internal representation. Indeed, the by the behavioral relevance of the information,
representation of the external world generated rather than designed to provide an accurate
by the most studied sensory modality, the and complete description of it” (Treue 2001,
visual system, appears to be neither unified nor p. 295). Furthermore, because the direction of

attention is frequently shifting from one place heterogeneity at the level of single neurons is
to another, the activity in visual regions is any- difficult to reconcile with the breakdown of be-
thing but stable. It is constantly changing, even havior into perception, cognition, and action.
FEF: frontal eye fields
if one is fixating a completely motionless scene. Instead of encoding the unique and detailed
PMd: dorsal
To summarize, the classical assumption of motor program predicted by classical models
premotor cortex
a unified and stable internal representation (an (Keele 1968, Miller et al. 1960), neural activ-
internal replica of the external world) does not ity in motor regions appears to initially encode
appear to be well supported by the divergence information about relevant stimuli and then
of the visual system and the widespread influ- changes to represent motor variables, such as
ence of attentional and contextual modulation. the direction of movement. For example, dur-
If something that resembles a perception mod- ing visual search tasks, cells in frontal eye fields
ule exists, it overlaps so strongly with cognitive (FEF) initially respond to all salient stimuli,
processes that the distinction between them be- including multiple distracters, but later reflect
comes blurred. only the final selected target (Schall & Bichot
1998). During reach/antireach tasks, neural ac-
tivities in the dorsal premotor cortex (PMd) first
Motor Control appear to encode the location of a stimulus and
According to the information processing view later reflect the movement direction instructed
of voluntary behavior, the role of the motor sys- by that stimulus (Crammond & Kalaska 1994,
tem is to implement the course of action com- Gail et al. 2009). Such findings have often been
manded by the cognitive system. This has led to interpreted as early visual responses, which are
the common assumption that by the time mo- followed by motor activity, but it is unclear how
tor processing begins, cognitive processes have a traditional model could account for both of
decided what to do, and only a single motor these being encoded in the same region, some-
program is prepared before movement initi- times by the same neurons. Additionally, neural
ation (Keele 1968, Miller et al. 1960). How- activity in motor regions appears to be modu-
ever, neural data do not appear to support this lated by a variety of putatively cognitive vari-
assumption. First, many of the same regions ables, as described below. In summary, if an ac-
that appear to be involved in movement plan- tion module exists, then it appears to be closely
ning are also active during movement execution entwined with both perceptual and cognitive
(Alexander & Crutcher 1990b, Crammond & processes (Lebedev & Wise 2002).
Kalaska 2000, Hoshi & Tanji 2007, Kalaska
et al. 1998, Wise et al. 1997). Neural corre-
lates of both planning and execution processes Cognitive Functions
can be found even in the activity of individ- The search for the modules that lie be-
ual cells, whose association with motor out- tween perception and action has been even
put changes in time from abstract aspects of more problematic. According to classical views
the task to limb movement-related parameters (Fodor 1983, Pylyshyn 1984), cognition is sep-
(Cisek et al. 2003, Crammond & Kalaska 2000, arate from sensorimotor control. However, a
Shen & Alexander 1997). Furthermore, when- hallmark executive function, decision making
ever planning activity has been studied in tasks (Tversky & Kahneman 1981), does not appear
that present animals with choices, that same ac- to be localized within particular higher cog-
tivity also appears related to decision making nitive centers such as the primate prefrontal
processes that should have been completed by cortex. Instead, there is growing evidence that
the cognitive system (Cisek & Kalaska 2005, decisions, at least those reported through ac-
Gold & Shadlen 2007, Hoshi & Tanji 2007, tion, are found within the same sensorimo-
Platt & Glimcher 1999, Romo et al. 2004, tor circuits that are responsible for planning
Wallis & Miller 2003). Such functional and executing the associated actions (Cisek &

Kalaska 2005, Gold & Shadlen 2007, Pesaran spatial sensory information on the location of
et al. 2008, Romo et al. 2002, Romo et al. behaviorally salient objects in the environment
2004, Scherberger & Andersen 2007). For ex- (Colby & Duhamel 1996, Colby & Goldberg
LIP: lateral
intraparietal area ample, Romo and colleagues (Hernandez et al. 1999, Stein 1992), strongly modulated by atten-
2002, Romo et al. 2002, Romo et al. 2004) tion and behavioral context (Colby & Duhamel
PPC: posterior
parietal cortex found that during tasks in which a nominally 1996, Colby & Goldberg 1999, Kalaska 1996,
tactile perceptual decision is reported by an Mountcastle et al. 1975). This has led to the
Salience map: a
spatial representation arm movement, correlates of all of the pu- hypothesis that the parietal cortex is involved
of the most salient tative sensory encoding, memory, discrimina- in directing attention to different parts of space
features of the tion, and decision-making processes were much and in constructing a salience map of the en-
environment stronger within premotor regions than in classi- vironment (Constantinidis & Steinmetz 2001,
cal somatic sensory areas. Similarly, when mon- Kusunoki et al. 2000). Presumably, this forms
keys were required to decide whether to hold part of the perceptual representation that serves
or release a lever in response to a sequence as input to the cognitive system. However,

of visual stimuli, neural correlates of the be- there is also strong evidence that parietal cor-
havioral rule (match/nonmatch) and the action tical activity contains representations of action
decision (release/hold) were stronger and ap- intentions (Andersen & Buneo 2003, Colby &
peared earlier in the premotor regions related to Duhamel 1996, Kalaska et al. 1997, Mazzoni
hand movements than in the prefrontal cortex et al. 1996, Platt & Glimcher 1997, Snyder
(Wallis & Miller 2003). Likewise, decisions et al. 2000), which include activity that specifies
about eye movements appear to involve the the direction of intended saccades (Snyder et al.
same circuits that execute eye movements, 2000) and arm reaching movements (Andersen
which include the lateral intraparietal area & Buneo 2003, Buneo et al. 2002, Kalaska &
(LIP) (Dorris & Glimcher 2004, Gold & Crammond 1995), and different subregions of
Shadlen 2007, Platt & Glimcher 1999, Sugrue the PPC are specialized for different effectors
et al. 2004, Yang & Shadlen 2007), the FEF (Calton et al. 2002, Cui & Andersen 2007).
(Coe et al. 2002, Schall & Bichot 1998), and Because action representations are supposedly
the superior colliculus (Basso & Wurtz 1998, activated by the output of the cognitive system,
Carello & Krauzlis 2004, Horwitz et al. 2004, it is difficult to reconcile these findings with the
Thevarajah et al. 2009), which is a brainstem sensory properties of the PPC, which leads to
structure that is just two synapses away from persistent debates about its role. Furthermore,
the motor neurons that move the eye. In all of neural activity in the PPC is also modulated
these cases, the same neurons appear to first re- by a range of variables associated with decision
flect decision-related variables such as the qual- making, such as expected utility (Platt &
ity of evidence in favor of a given choice and Glimcher 1999), local income (Sugrue et al.
then later encode the metrics of the action used 2004), relative subjective desirability (Dorris &
to report the decision (Cisek & Kalaska 2005, Glimcher 2004), and log-likelihood estimates
Kim & Basso 2008, Roitman & Shadlen 2002, (Yang & Shadlen 2007). In short, the PPC
Schall & Bichot 1998, Yang & Shadlen 2007). does not appear to fit neatly into any of the
Consequently, it has proven to be notori- categories of perception, cognition, or action;
ously difficult to assign a specific perceptual, or alternatively, the PPC reflects all categories
cognitive, or motor function to cortical associa- at once without respecting those theoretical
tive regions such as the posterior parietal cortex distinctions. Indeed, it is difficult to see how
(PPC), where cells appear to be related to all neural activity in the PPC can be interpreted
of these functions (Andersen & Buneo 2003, using any of the concepts of classical cognitive
Colby & Duhamel 1996, Colby & Goldberg psychology (Culham & Kanwisher 2001).
1999, Culham & Kanwisher 2001, Kalaska The data and resulting disagreements
& Crammond 1995). The PPC represents reviewed above have motivated us to reflect on

some of our assumptions for interpreting neural organization was being laid down. Throughout
activity. Perhaps specific functions such as per- evolutionary history, organisms and their
ception are not implemented by particular cor- nervous systems have been preoccupied by
tical regions. Instead, they may be implemented almost constant interaction with a complex
by different layers or subnetworks of cells dis- and ever changing environment, which contin-
tributed within many parts of the nervous uously offers a potentially bewildering variety
system. Perhaps distinct roles such as percep- of opportunities and demands for action.
tual or motor representations can be performed Interaction with such an environment cannot
by the same neurons at different times. These be broken down into a sequence of distinct and
possibilities are worth considering and study- self-contained events that each start with a dis-
ing experimentally. Here, however, we explore crete stimulus and end with a specific response,
a different possibility. We consider whether similar to the isolated trials we typically use
the distinctions among perceptual, cognitive, in many psychological or neurophysiological

and motor systems may not reflect the natural experiments. Instead, it involves the continu-
categories of neural computations that underlie ous modification of ongoing actions through
sensory-guided behavior (Hendriks-Jansen feedback control, the continuous evaluation of
1996, Lebedev & Wise 2002). The framework alternative activities that may become available,
of serial information processing may not be and continuous tradeoffs between choosing
the optimal blueprint for the global functional to persist in a given activity and switching
architecture of the brain. Instead, we consider to a different one. The internal processes
whether alternative theoretical frameworks for that are most useful for such behavior may
the large-scale organization of behavior may not be those that first construct an accurate
facilitate interpretations of neural activity. internal description of objective and abstract
knowledge about the world and then reflect
upon it with some introspective, intelligent
AN ECOLOGICAL PERSPECTIVE circuits. Instead, pragmatic processes that
One of the most important facts we know mediate sensorimotor interaction in the here
about the brain is that it evolved. This not and now, on the basis of continuous streams of
only motivates our theories to describe mecha- sensory inputs as well as prior knowledge and
nisms that confer selective advantage, but more experiences, are much more useful for guiding
importantly, it constrains theories to respect interactive behavior (Gibson 1979).
the brain’s phylogenetic history. Contrary to An emphasis on real-time, natural behavior
popular belief, brain evolution has been re- has been the foundation of ethological research
markably conservative. Since the development for a long time (Hinde 1966). In the early twen-
of the telencephalon, the basic outline of the tieth century, researchers such as Von Uexküll,
vertebrate nervous system has been strongly Tinbergen, and Lorenz focused their studies on
conserved (Butler & Hodos 2005, Holland & the observation of animals in the wild rather
Holland 1999, Katz & Harris-Warrick 1999). than in the laboratory. Consequently, instead
Even recently elaborated structures such as the of focusing on how knowledge is represented
mammalian neocortex have homologs among or what variables are included in the motor
nonmammals (Medina & Reiner 2000), and the program, they focused on how competition
topology of neural circuitry is analogous across between potential actions is resolved, how on-
diverse species (Karten 1969). going behavior is fine tuned by feedback mecha-
The conservative nature of brain evolution nisms that operate at multiple hierarchical lev-
motivates us to think about large-scale theories els, and how animals trade off activity against
of neural organization from the perspective of metabolic costs.
the kinds of behaviors that animals engaged in Some of the original founders of psycholog-
many millions of years ago, when that neural ical science also emphasized the importance of

interactions with the environment. For exam- interacts in the real world, such architectures
ple, John Dewey (1896) criticized the view of have simply proven to be more effective than
behavior as a process of receiving a stimulus and serial information processing through distinct
Affordances:
opportunities for producing a response, and wrote that “[w]hat perception, cognition, and action modules.
action defined by the we have is a circuit. . .the motor response deter- These concepts are also becoming increasingly
environment around mines the stimulus, just as truly as sensory stim- influential in a branch of cognitive science that
an animal ulus determines movement.” (p. 363). Similar is sometimes called embodied cognition (Clark
Embodied cognition: emphases on sensorimotor control were made 1997, Klatzky et al. 2008, Núñez & Freeman
a study of cognition by Hughlings Jackson (1884) and Merleau- 2000, Thelen et al. 2001).
that emphasizes its
Ponty (1945), among many others. Perhaps the Such concepts may also be useful for inter-
role in sensorimotor
control and action best known example is the work of the eminent preting neurophysiological data. For example,
psychologist Jean Piaget (1954), who suggested Graziano & Aflalo (2007) proposed that the
that the abstract cognitive abilities of adult hu- multiple motor areas in the precentral gyrus
mans are constructed upon the basis of the sen- may not be organized on the basis of a se-
sorimotor interactions experienced as a child. quential planning and execution architecture,
This is supported by a variety of neural studies, as commonly assumed. Instead, the precentral
which include the classic experiments of Held & gyrus may reflect the animal’s natural behav-
Hein (1963), who found that the visual behavioral repertoire, with different regions that are
ior of newborn kittens did not develop properly specialized for different actions such as bring-
unless they were allowed to exert their own ac- ing objects to the mouth, manipulating objects
tive control upon their visual input. in central vision, climbing, or defensive behav-
The perceptual psychologist James Gibson ior. Although controversial, this conjecture has
was another well-known proponent of an eco- intriguing similarities to theoretical proposals
logical view of behavior. Similar to ethologists, that evolution constructs complex behaviors by
Gibson viewed the constrained environment of using simpler ones as building blocks (Brooks
a typical psychological experiment as conceal- 1991, Hendriks-Jansen 1996). This has clear
ing the true interactive nature of behavior. He ecological advantages because it reflects the
argued that perception is not about passively need for animals to partially plan many different
constructing an internal representation of the classes of potential actions, such as grasping a
world, but rather it is about actively picking up piece of fruit while also being ready to scamper
information of interest to one’s behavior. In- away in case of danger.
spired by earlier work of Gestalt psychologists One particularly important and influential
such as Koffka, he emphasized that the envi- example of how a perspective of interactive be-
ronment contains information relevant for an havior may shed light on neurophysiology is
animal’s activity and that a large part of percep- the work of Melvyn Goodale and David Milner
tion is the accumulation of that information. (1992, Milner & Goodale 1995). As discussed
He defined the concept of affordances (Gibson above, visual processing diverges in the cerebral
1979) as the opportunities for action that the cortex into a ventral stream, where cells are sen-
environment presents to an animal. sitive to stimulus features, and a dorsal stream,
Ethological concepts have been very use- where cells are sensitive to spatial relationships
ful in research on autonomous robotics, which (Ungerleider & Mishkin 1982). Instead of de-
is increasingly abandoning classical serial ar- scribing these, respectively, as the what and
chitectures based on explicit representations where systems, Goodale & Milner suggested
of the environment in favor of hierarchical that the predominant role of the dorsal stream is
control systems in which the basic elements to mediate visually guided behavior. They pro-
are sensorimotor feedback loops (Ashby 1965, posed that the dorsal stream (now often called
Brooks 1991, Hendriks-Jansen 1996, Meyer the how system) is sensitive to spatial infor-
1995, Sahin et al. 2007). For a robot that mation, not to build a representation of the

environment for central knowledge acquisition, affordance competition hypothesis (Cisek

but because spatial information is critical for 2007). This general hypothesis is directly in-
specifying the parameters of potential and on- spired by the work of Gibson, Ashby, Goodale
Action selection: the
going actions. This view explains many other & Milner, Arbib, and many others mentioned process of choosing an
properties of dorsal pathway processing, such as above. It begins with a distinction between action from among
its emphasis on concrete and current informa- two types of problems that animals behaving many possible
tion (Milner & Goodale 1995) and its intimate in the natural environment continuously alternatives
interconnection with frontal regions involved face: deciding what to do and how to do it. Action specification:
in movement control ( Johnson et al. 1996, We can call these the problems of action the process of
specifying the
Wise et al. 1997). From this perspective, pro- selection and action specification. However,
spatiotemporal aspects
cessing in the parietal cortex and reciprocally although traditional psychological theories of possible actions
connected premotor regions is not exclusively assume that selection (decision making) occurs
concerned with descriptive representations of before specification (movement planning), we

objects in the external world but primarily with consider the possibility that, at least during
pragmatic representations of the opportunities natural interactive behavior, these processes
for action that those objects afford (Cisek 2007, operate simultaneously and in an integrated
Colby & Duhamel 1996, Fadiga et al. 2000, manner (Cisek 2007).
Kalaska et al. 1998, Rizzolatti & Luppino 2001). For the particular case of visually-guided
Indeed, parietal activity in both monkeys (Iriki movement, action specification (Figure 1,
et al. 1996, Mountcastle et al. 1975) and humans dark blue lines) may involve the dorsal visual
(Gallivan et al. 2009) is often stronger when ob- stream and a distributed and reciprocally in-
jects are within reach. terconnected network of areas in the posterior
Several groups have developed these ideas parietal and caudal frontal cortex (Andersen
further. For example, Fagg & Arbib (1998) have & Buneo 2003; Andersen et al. 1997; Goodale
suggested that the PPC represents a set of cur- & Milner 1992; Johnson et al. 1996; Kalaska
rently available potential actions, one of which 1996; Kalaska & Crammond 1995; Milner &
is ultimately selected for overt execution. Simi- Goodale 1995; Rizzolatti & Luppino 2001;
larly, we and others have suggested that the dor- Wise et al. 1996, 1997). These circuits perform
sal stream is involved in specifying the param- transformations that convert information about
eters of potential actions, whereas the ventral objects in sensory coordinates into the pa-
stream provides further information for their rameters of actions (Andersen & Buneo 2003,
selection (Andersen & Buneo 2003, Cisek 2007, Andersen et al. 1997, Wise et al. 1997). Along
Kalaska et al. 1998, Passingham & Toni 2001, the way, each area can represent information
Sakagami & Pan 2007). This has much in com- that is pertinent to several potential actions
mon with a long history of proposals, made on simultaneously as patterns of tuned activity
the basis of EEG studies (Coles et al. 1985) and within distributed populations of cells. This
stimulus-response compatibility effects (Korn- forms a representation of possible movements
blum et al. 1990), that neural processing is that is conceptually similar to a probability
continuous and not organized in distinct serial density function (Sanger 2003). Importantly,
stages. It is also similar to the proposal that the these same brain regions ultimately guide the
brain begins to prepare several actions in par- execution of those actions. Because multiple
allel while collecting evidence for selecting be- actions usually cannot be performed at the
tween them (Shadlen et al. 2008), a view that is same time, there is competition between
strongly supported by neurophysiological stud- options, perhaps through mutual inhibition
ies of decision making (Gold & Shadlen 2007, among cells with different tuning properties
Kim & Basso 2008, Ratcliff et al. 2007). (Cisek 2006) and/or through differential se-
Some of these ideas are summarized in lection in corticostriatal circuits (Leblois et al.
Figure 1, which shows what we call the 2006).

Prem
otor
corte
tex x
cor Potential actions
tal
Parie
Attention Selection
rules
Prefrontal
cortex
Behavioral
ream
relevance
al st
Basal
Behavior
Dors
Predicted ganglia
feedback biasing Payoff
Ventral st
ream
Object
Specification
identity
Cerebellum
Temporal Selection
cortex
Motor
command
Visual feedback
Figure 1
Sketch of the affordance competition hypothesis in the context of visually-guided movement. The primate
brain is shown, emphasizing the cerebral cortex, cerebellum, and basal ganglia. Dark blue arrows represent
processes of action specification, which begin in the visual cortex and proceed rightward across the parietal
lobe, and which transform visual information into representations of potential actions. Polygons represent
three neural populations along this route. Each population is depicted as a map where the lightest regions
correspond to peaks of tuned activity, which compete for further processing. This competition is biased by
input from the basal ganglia and prefrontal cortical regions that collect information for action selection (red
double-line arrows). These biases modulate the competition in several loci, and because of reciprocal
connectivity, their influences are reflected over a large portion of the cerebral cortex. The final selected
action is released into execution and causes overt feedback through the environment (dotted blue arrow) as
well as internal predictive feedback through the cerebellum. Modified with permission from Cisek (2007).
If a competition between representations Sakagami & Pan 2007, Tanji & Hoshi 2001,
of potential actions exists in frontoparietal Wise 2008). In turn, the prefrontal areas re-
circuits, then intelligent behavior requires a ceive information pertinent to action selection
way to influence that competition by factors that include object identity from the temporal
related to rewards, costs, risks, or any variable lobe (Pasupathy & Connor 2002, Tanaka
pertinent to making good choices. A variety of et al. 1991) and subjective value from the
brain systems can contribute their votes into orbitofrontal cortex (Padoa-Schioppa & Assad
this selection process simply by biasing activity 2008, Schultz et al. 2000, Wallis 2007). In sum-
within the ongoing frontoparietal competition mary, the hypothesis is that interaction with the
(Figure 1, red double-line arrows). This environment involves continuous and simul-
includes influences from subcortical structures taneous processes of sensorimotor control and
such as the basal ganglia (Mink 1996, Redgrave action selection from among the distributed
et al. 1999, Schultz 2004) and cortical regions representations of a limited number of response
such as the prefrontal cortex (Miller 2000, options. This perspective is consistent with a

large family of computational models of deci- Frontoparietal Specification

sions, which suggest that neural activity related of Potential Actions
to different response choices builds up in sepa-
Following Goodale & Milner (1992, Milner MIP: medial
rate accumulators as a function of the evidence intraparietal area
& Goodale 1995), one can interpret the
for or against those choices until a threshold is
dorsal visual stream as part of the system AIP: anterior
reached that favors one over the others (Gold intraparietal area
for specifying the parameters of potential
& Shadlen 2007, Ratcliff et al. 2007). Whereas
actions using visual information, a process that PMv: ventral
classic decision models have treated the accu- premotor cortex
continues during movement execution (Resulaj
mulators as separate modules that correspond
et al. 2009). As mentioned above, the dorsal
to distinct choices, our hypothesis suggests
stream is not unified but progressively diverges
that they emerge from a continuous population
into parallel subsystems, each specialized
that represents parameters of potential actions
toward the demands of different sensorimotor
(Cisek 2006, Erlhagen & Schöner 2002, Fur-

functions and effectors (Andersen et al. 1997,
man & Wang 2008, Tipper et al. 2000), at least

Colby & Duhamel 1996, Colby & Goldberg
in cases when decisions are reported through
1999, Rizzolatti & Luppino 2001, Stein 1992,
specific actions. Indeed, the neurons that have
Wise et al. 1997). Area LIP represents space
been implicated in the evidence accumulation
in an ego-centered reference frame (Colby &
process are always found within populations
Duhamel 1996, Snyder et al. 1998), is involved
tuned for motor output parameters such as
in control of gaze (Snyder et al. 2000), and is
direction (Glimcher 2003, Gold & Shadlen
interconnected with other parts of the gaze
2007, Kim & Basso 2008, Ratcliff et al. 2007).
control system that includes the FEF and the
We propose that the kind of general theoret-
superior colliculus (Paré & Wurtz 2001). The
ical architecture shown in Figure 1, although
medial intraparietal area (MIP) is involved in
highly simplified, can nevertheless help us in-
the control of arm reaching movements (Cui &
terpret many of the neural data briefly reviewed
Andersen 2007, Kalaska & Crammond 1995,
above, data that have proven difficult to inter-
Pesaran et al. 2008, Scherberger & Andersen
pret within the traditional view of serial infor-
2007, Snyder et al. 2000), represents target
mation processing stages. The rest of this re-
locations with respect to the direction of gaze
view is devoted to discussing data relevant to
and the position of the arm (Buneo et al. 2002),
that claim.
and is interconnected with frontal regions that
are involved in reaching, such as PMd ( Johnson
REVISITING NEURAL DATA et al. 1996, Wise et al. 1997). Neurons in the
In revisiting some of the neural data discussed at anterior intraparietal area (AIP) are involved in
the beginning of this review, we emphasize two grasping (Baumann et al. 2009), their activity
main conjectures: (a) The control of interactive is sensitive to object size and orientation, and
behavior involves competition between paral- they are interconnected with the grasp-related
lel sensorimotor control loops, and (b) neu- ventral premotor cortex (PMv) (Nakamura
ral representations involved in this control are et al. 2001, Rizzolatti & Luppino 2001). To
pragmatic—that is, they are adapted to produce summarize, the dorsal stream diverges into
good control as opposed to producing accurate parallel subsystems, each of which specifies
descriptions of the sensory environment or a the spatial parameters of different kinds of
motor plan. Both of these proposals have been potential actions and plays a direct role in
made repeatedly for over a hundred years of guiding their execution during movement.
research from Dewey to Gibson to Goodale & In such a distributed system, several actions
Milner and others, but they have not often been can be specified simultaneously. For example,
used as the theoretical framework for interpret- if a monkey is presented with a spatial target
ing neurophysiological data. but not instructed about whether an arm or

eye movement is required, neurons begin to and veers away from regions of risk (Trommer-
discharge in both LIP and MIP (Calton et al. shauser et al. 2006). Patients with frontal lobe
2002, Cui & Andersen 2007). Later, if an arm damage often cannot suppress actions associ-
movement is instructed (Calton et al. 2002) or ated with distracters even while they are plan-
autonomously chosen (Cui & Andersen 2007), ning actions directed elsewhere (Humphreys &
the activity becomes stronger in MIP than LIP. Riddoch 2000), and such effects may be the
Conversely, if a saccade is instructed or cho- result of competition among parallel simul-
sen, activity becomes stronger in LIP than MIP. taneous representations of potential actions,
This is consistent with the proposal that before with a bias toward the actions with the high-
the effector is selected, reach and saccade plans est stimulus-response compatibility (Castiello
begin to be specified simultaneously by differ- 1999).
ent parts of the PPC. Indeed, during natural Neurophysiological studies support this in-
activity, eye and hand movements are usually terpretation. For example, partial information
executed in unison. Similar findings have been on possible upcoming movements engages the
reported for decisions regarding hand choice. activity of cells in reach-related regions be-
For example, Hoshi & Tanji (2007) showed that fore the animal selects the movement that will
when monkeys are first presented with a reach be made (Bastian et al. 1998, Kurata 1993,
target location in a bimanual response-choice Riehle & Requin 1989). In particular, when
task without specifying which arm to use, neu- a reach direction is initially specified ambigu-
ral activity in the premotor cortex reflects the ously by sensory information, neural activity
potential movements of both hands until the arises in the motor and premotor cortex that
monkey is instructed about which hand to use. spans the entire angular range of potential di-
Simultaneous specification of multiple rections. Later, when the direction is specified
potential actions can occur even within the more precisely, the directional spread of pop-
same effector system (Basso & Wurtz 1998, ulation activity narrows to reflect this choice
Bastian et al. 1998, Baumann et al. 2009, Cisek (Bastian et al. 1998). Neural correlates of mul-
& Kalaska 2005, McPeek & Keller 2002, tiple potential reaching actions have been re-
Platt & Glimcher 1997, Powell & Goldberg ported in the dorsal premotor cortex (PMd)
2000, Schall & Bichot 1998, Scherberger & even when the choices are distinct and mutually
Andersen 2007). For example, behavioral data exclusive (Cisek & Kalaska 2005). As shown in
(McPeek et al. 2000) and neurophysiological Figure 2, when a monkey was presented with
data (McPeek & Keller 2002) suggest that the two opposite potential reaching actions, only
preparation of multiple sequential saccades can one of which would later be indicated as the
overlap in time. When two or more potential correct choice by a nonspatial cue, neural ac-
saccade targets are presented simultaneously, tivity in the premotor cortex specified both di-
neural correlates for each are observed in rections simultaneously. When information for
area LIP (Platt & Glimcher 1997, Powell & selecting one action over the other became
Goldberg 2000) and even in the superior col- available, the representation of the chosen di-
liculus, where they are modulated by selection rection was strengthened while that of the un-
probability (Basso & Wurtz 1998, Kim & wanted direction was suppressed. The monkey
Basso 2008). used a strategy of preparing both movements si-
Likewise, behavioral studies of reaching multaneously during the initial period of uncer-
have suggested that the brain simultaneously tainty despite the fact that the task design per-
processes information about multiple poten- mitted the use of an alternative strategy (more
tial actions. For example, the trajectory of a consistent with traditional models of process-
reaching movement to a target is influenced by ing) in which target locations are stored in a
the presence of distracters (Song & Nakayama general-purpose working memory buffer that
2008, Tipper et al. 2000, Welsh et al. 1999) is distinct from motor representations and only

Go signal
Color cue
Memory
period
e
Spatial cues Tim
–10 0 +10
Ce s
lls 0m Activity
50 with respect to
baseline
Figure 2
Population activity in the dorsal premotor cortex during a reach-selection task. The 3D colored surface
depicts neural activity with respect to baseline, with cells sorted by their preferred direction along the
bottom edge. Diagrams on the left show the stimuli presented to the monkey at different points during the
trial (cross indicates the cursor). Note that during the period of ambiguity, even after stimuli vanished, the
population encodes two potential directions. Data from Cisek & Kalaska (2005).
converted to a motor plan after the decision time-locked to stimulus appearance, does not
is made. In contrast, we propose that multiple encode the stimuli themselves but rather the set
movement options are specified within the same of potential actions that are most strongly asso-
system that is used to prepare and guide the ex- ciated with those stimuli (Wise et al. 1996), such
ecution of the movement that is ultimately se- as actions with high stimulus-response com-
lected. The simultaneous specification of mul- patibility (Crammond & Kalaska 1994). This
tiple actions can even occur when only a single would imply that the functional role of this ac-
object is viewed. For example, the multiple af- tivity does not change in time from sensory to
fordances offered by a single object can evoke motor encoding but simply reflects the arrival
neural activity in the grasp-related area AIP that of selection influences from slower but more
can represent several potential grasps until one sophisticated mechanisms for deciding which
is instructed (Baumann et al. 2009), in agree- action is most appropriate.
ment with the predictions of theoretical models Recent computational models have pro-
(Fagg & Arbib 1998). posed that whenever multiple potential targets
Evidence that the nervous system can si- are available, representations of potential ac-
multaneously represent multiple potential actions emerge within several frontoparietal neu-
tions suggests a straightforward interpretation ral populations, each composed of a continuum
of the finding, described above, that early re- of cells with different preferences for the po-
sponses in many premotor and parietal re- tential parameters of movement (Cisek 2006,
gions first appear to encode information about Erlhagen & Schöner 2002, Tipper et al. 2000).
relevant stimuli and later change to encode In each population, cells with similar prefer-
motor variables. Perhaps the early activity, ences mutually excite each other (even if they

are not physically adjacent), which leads to the sensory, motor, and cognitive in nature (Wise
activation of groups of cells with similar tun- et al. 1996). The case of area LIP is particularly
ing. At the same time, cells with different pref- instructive. If LIP is involved in the specifica-
erences inhibit each other, thus implementing tion of potential saccades, then its activity must
a competition between representations of ac- correlate with the location of possible saccade
tions that are mutually exclusive. Unlike classi- targets (Mazzoni et al. 1996, Snyder et al. 2000),
cal models of decisions, in which the different even when multiple potential saccades are pro-
choices are abstract and clearly distinct (e.g., cessed simultaneously (Platt & Glimcher 1997,
choosing between gambles A or B), models Powell & Goldberg 2000). At the same time,
in which decisions emerge within tuned pop- however, the ongoing selection of potential ac-
ulations suggest that the same mechanism— tions will modulate the strength of activities in
lateral inhibition—is responsible for defining LIP. Such modulation is influenced by target
the choices as well as for implementing the salience (Colby & Goldberg 1999, Kusunoki
competition between them. Importantly, they et al. 2000), reward size and selection proba-
suggest that decisions about actions emerge bility (Platt & Glimcher 1999, Yang & Shadlen
within the same populations of cells that de- 2007), and other decision variables (Dorris &
fine the physical properties of those actions and Glimcher 2004, Sugrue et al. 2004), as well as
guide their execution. prior information on the type of action to be
This proposal can account for phenomena performed (Calton et al. 2002, Cui & Andersen
that cannot be explained using models in which 2007). The progressive elimination of potential
the decision process occurs in an abstract space saccade targets along the dorsal stream also ex-
that is separate from a representation of the plains why the representation of visual space in
metrics of motor options. For example, al- LIP is so sparse (Gottlieb et al. 1998): Only the
though it is well-known that reaction time (RT) most promising and salient targets make it to
generally increases with the number of choices LIP.
presented to a subject, it is less widely recog- If the presence of salient targets can engage
nized that RT is also dependent upon the spa- the simultaneous specification of several poten-
tial separation of the response options (Bock & tial actions in a variety of frontoparietal systems,
Eversheim 2000). As another example, Ghez then this process is closely related to the con-
et al. (1997) showed that forced rapid choices cept of a salience map (Kusunoki et al. 2000,
between precued options are dependent on tar- Powell & Goldberg 2000). In particular, the
get separation. If cues are close together, sub- front end of a system for action selection should
jects initially move in between them (continu- enhance the most behaviorally salient informa-
ous mode) before deviating toward one or the tion in the environment to bias sensorimotor
other in mid-reach. If the cues are far apart, they systems toward the most behaviorally relevant
choose one at random and move to it directly potential actions. Thus, it will be action depen-
(discrete mode). To explain such results, mod- dent (Snyder et al. 2000) but still influenced by
els of decisions must capture how the choices the salience of stimuli, even while actions are
themselves are defined in physical space and instructed elsewhere (Kusunoki et al. 2000). In
how the similarity of potential actions influ- short, attention and intention may be different
ences their interactions (Cisek 2006, Erlhagen aspects of a common process that progressively
& Schöner 2002). This is straightforward if the narrows the set of potential actions that will
representations of choices exist within neural be processed further downstream. This agrees
populations that encode the physical parame- with the proposal (Allport 1987, Neumann
ters of the movements used to report the choice. 1990) that attention is a mechanism for early
From this perspective, it is not surprising action selection and not a solution to the
that neural activity in the frontal and parietal purely internal problem of a computational
cortex encodes information that appears to be bottleneck for processing sensory information

(Broadbent 1958), as is often assumed. Indeed, neural correlates of cognitive processes can be
models of action selection compatible with the seen throughout the brain during sensorimotor
kind of framework shown on Figure 1 (Cisek and decision tasks. When we look for the neural
2006, Erlhagen & Schöner 2002) are func- correlates of cognition, it does not appear as an
tionally equivalent to the biased competition independent module (Fodor 1983) that receives
model used to explain data on visual attention input from perceptual modules and sends goal
(Boynton 2005, Desimone & Duncan 1995, signals to motor centers. Instead, it appears as a
Treue 2001). In both cases, parallel representa- process that is closely integrated with action se-
tions of targets/actions compete through lateral lection, evaluation, and motor execution (Cisek
inhibition; and in both cases, they are biased by 2007, Glimcher 2003, Gold & Shadlen 2007,
top-down and or bottom-up influences. Heekeren et al. 2008, Hoshi & Tanji 2007,
The idea that decisions emerge within a Pesaran et al. 2008, Rizzolatti & Luppino 2001,
continuum of tuned cells can even go be- Shadlen et al. 2008).

yond strictly action-related decisions (Song & The recent evolution of primates is distin-
Nakayama 2009) and may provide insights into guished by advances in the ability to select ac-
mechanisms that have traditionally been stud- tions based on increasingly abstract and arbi-
ied using tasks in which subjects report choices trary criteria. This kind of selection may have
by pressing distinct keys on a keyboard. For been made possible by the dramatic elabora-
example, when subjects are asked to report on tion of the prefrontal cortex (Hauser 1999), es-
seemingly binary yes/no questions (such as, is pecially the granular frontal cortex, which does
the sky ever green?) by moving a cursor instead not appear to have a homolog in rodents (Wise
of just pressing one key or another, their move- 2008). These frontal regions are strongly im-
ment trajectories reveal a wealth of phenomena plicated in decision making and action selec-
that suggest the decision is represented in a con- tion (Fuster et al. 2000, Kim & Shadlen 1999,
tinuous space that spills into overt movement Miller 2000, Romo et al. 2004, Rowe et al. 2000,
(McKinstry et al. 2008). For example, features Tanji & Hoshi 2001). For example, neurons
of the trajectory such as its endpoint and peak in the dorsolateral prefrontal cortex (DLPFC)
velocity correlate with the subject’s confidence are sensitive to various combinations of stim-
about their choice. Even linguistic decisions, ulus features, and this sensitivity is always re-
such as those made while parsing an ambigu- lated to the particular demands of the task at
ous sentence, appear to occur in a continuous hand (Barraclough et al. 2004, Hoshi et al. 1998,
parameter space that can influence movement Kim & Shadlen 1999, Quintana & Fuster 1999,
trajectories (Farmer et al. 2007). These find- Rainer et al. 1998). For example, an experiment
ings are difficult to reconcile with the idea that on reach target selection (Hoshi et al. 2000)
cognition is separate from sensorimotor control found that when arbitrary iconic stimuli were
(Fodor 1983) but make good sense if the con- presented, activity in the DLPFC was sensitive
tinuous nature of the representations that un- to potentially relevant stimulus features, such
derlie the selection of actions has been retained as shape and location. After the presentation of
as selection systems evolved to implement in- a signal that indicated the correct selection rule
creasingly abstract decisions. (shape-match or location-match), rule-sensitive
neurons briefly became active, selecting out the
relevant memorized stimulus features needed
Action Selection Signals from to make the response choice. After this process
Prefrontal and Subcortical Sources was complete, the remaining activity reflected
From the perspective of the parallel architec- the intended movement choice. Prefrontal de-
ture reviewed here, there may be no single cen- cisions appear to evolve through the collection
tral executive module that guides decisions. As of votes for categorically selecting one choice
shown by many of the studies reviewed above, over others. For example, when monkeys were

trained to report perceptual discriminations us- estimate of the behavioral relevance of potential
ing saccades (Kim & Shadlen 1999), DLPFC actions in the ventrolateral prefrontal cortex
activity initially reflected the quality of evi- (VLPFC), which then projects to the DLPFC
Key stimulus: a
particular feature of dence in favor of a given target and later simply and premotor regions to influence action
the environment reflected the monkey’s choice. Similar effects selection.
which, when detected, have been reported in PMv and prefrontal cor- Because action selection is a fundamental
elicits a specific action tex during tactile vibration frequency discrimi- problem faced by even the most primitive ver-
nation tasks (Romo et al. 2004). tebrates, it likely involves structures that were
The information for visually-based action prominently developed long ago and have been
selection can also come from the ventral vi- conserved in evolution. The basal ganglia are
sual stream (Cisek 2007, Kalaska et al. 1998, promising candidates (Kalivas & Nakamura
Passingham & Toni 2001, Sakagami & Pan 1999, Mink 1996, Redgrave et al. 1999). The
2007), where cells are sensitive to object identity basal ganglia may form a central locus in which
(Pasupathy & Connor 2002, Sugase et al. 1999, excitation that arrives from different motor sys-
Tanaka et al. 1991) and modulated by attention tems competes, and a winning behavior is se-
(Treue 2001). The detection of stimulus fea- lected while others are inhibited through pro-
tures relevant for action selection is reminiscent jections back to the motor systems (Brown et al.
of what ethologists referred to as the detection 2004, Leblois et al. 2006, Mink 1996, Redgrave
of a key stimulus that releases specific behaviors et al. 1999). Afferents to the input nuclei of
(Ewert 1997, Tinbergen 1950). The detection the basal ganglia (the striatum and subthalamic
of key stimuli need not require full-fledged ob- nucleus) arrive from nearly the entire cerebral
ject recognition (and indeed may be its precur- cortex and from the limbic system, converge
sor) because often a fragment or specific feature onto the output nuclei (substantia nigra and
that has consistent meaning within an animal’s globus pallidus), and project through the tha-
niche is all that is necessary to elicit behavior. lamus back to the cerebral cortex. This cortico-
Therefore, the properties of ventral stream pro- striatal-pallido-thalamo-cortical loop is orga-
cessing may not have originally evolved for a nized into multiple parallel channels, which
role in pure perception, but may instead reveal run through specific motor regions as well as
its earlier and more fundamental role in collect- through regions implicated in higher cogni-
ing information useful for action selection. In- tive functions (Alexander & Crutcher 1990a,
deed, the distinction between pure vision-for- Middleton & Strick 2000). In agreement with
perception versus vision-for-action systems is the hypothesis of basal ganglia selection, cell ac-
difficult to make at the neuroanatomical level tivity in the input nuclei is related to movement
(Lebedev & Wise 2002). parameters (Alexander & Crutcher 1990a) but
Wallis (2007) reviews evidence that the is also influenced by the expectation of reward
motivational value of potential actions is com- (Schultz et al. 2000, Takikawa et al. 2002). Dur-
puted by the orbitofrontal cortex (OFC), which ing learning of arbitrary visuomotor mappings,
integrates sensory and affective information to striatal activity evolves in concert with PMd ac-
estimate the value of a reward outcome. Single- tivity to indicate the selected movement (Buch
neuron studies have shown that the OFC repre- et al. 2006). The inactivation of cells in out-
sents the value of goods in a manner that is not put nuclei disrupts movement speed in a man-
dependent on their relative value with respect ner consistent with the proposal that the inhibi-
to other available choices (Padoa-Schioppa & tion of competing motor programs is disrupted
Assad 2008) but scales with the range of values (Wenger et al. 1999). Furthermore, the finding
in a given context (Padoa-Schioppa 2009). that the basal ganglia connect with prefrontal
Sakagami & Pan (2007) suggest that this infor- regions, in a manner similar to their connec-
mation is further integrated with sensory signals tions with premotor cortex, suggests that basal
from the ventral visual stream to provide an ganglia innervation of prefrontal regions also

mediates selection but on a more abstract level in a given trial usually does not determine the
(Hazy et al. 2007). This is also consistent with stimulus in the next trial. Of course, animals
motor and cognitive aspects of basal ganglia dis- are capable of dealing with this artificial sce-
eases (Mink 1996, Sawamoto et al. 2002). nario and able to learn which responses yield
the best rewards. The question addressed here
is the following: What would a parallel architec-
Parallel Operation ture such as that of Figure 1 predict about the
Continuous interactive behavior often does not time course of processing in such a situation?
allow one to stop to think or to collect infor- When the stimulus is first presented, we
mation to build a complete knowledge of one’s should expect an initial fast feedforward sweep
surroundings. The demands of survival in an of activity along the dorsal stream, crudely
ever-changing environment drove evolution to representing the potential actions that are
endow animals with an architecture that allows most directly specified by the stimulus. Indeed,
them to partially prepare several courses of ac- Schmolesky et al. (1998) showed that neural re-
tion simultaneously, so that alternatives can be sponses to simple visual flashes appear quickly
ready for release at short notice. Such an eco- throughout the dorsal visual system and engage
logical view of behavior suggests that the pro- putatively motor-related areas such as FEF in as
cesses of action selection and specification nor- little as 50 ms. This is significantly earlier than
mally occur simultaneously and continue even some visual areas such as V2 and V4. In general,
during the overt performance of movements, even within the visual system neural activation
which allows animals to switch to another op- does not appear to follow a serial sequence from
tion if they change their mind (Resulaj et al. early to late areas (Paradiso 2002). In a reaching
2009). That is, sensory information that ar- task, population activity in PMd responds to a
rives from the world is continuously used to learned visual cue within 50 ms of its appearance
specify several currently available potential ac- (Cisek & Kalaska 2005). Such fast responses are
tions, in parallel, while other kinds of informa- not purely visual because they reflect the con-
tion are collected to select the one that will text within which the stimulus is presented. For
be released into execution at a given moment example, they reflect whether the monkey ex-
(Cisek & Kalaska 2005, Glimcher 2003, Gold pects to see one or two stimuli (Cisek & Kalaska
& Shadlen 2007, Kalaska et al. 1998, Kim & 2005), reflect anticipatory biases or priors (Coe
Shadlen 1999, Shadlen et al. 2008). From this et al. 2002, Takikawa et al. 2002), and can be
perspective, behavior is viewed as a constant entirely absent if the monkey already knows
competition between internal representations what action to take and can ignore the stim-
of conflicting demands and opportunities. ulus altogether (Crammond & Kalaska 2000)
Suppose that an animal is endowed with this (Figure 3). In short, these phenomena are com-
kind of parallel architecture, adapted for con- patible with the notion of a fast dorsal specifi-
tinuous real-time interaction with a natural en- cation system that quickly uses novel visual in-
vironment. What would happen if we remove formation to specify the potential actions most
that animal from its natural environment and consistently associated with a given stimulus
place it in a neurophysiological laboratory? In (Gibson 1979, Milner & Goodale 1995).
this highly controlled and impoverished setting, After the initial options are quickly spec-
time is broken into discrete trials, each starting ified, slower selection processes should begin
with the presentation of a stimulus and ending to sculpt the neural activity patterns by intro-
with the production of a response (and if the re- ducing a variety of task-relevant biasing factors.
sponse is correct, a reward). Furthermore, un- Indeed, extrastriate visual areas MT and 7a re-
like in natural behavior, most features of the spond to a stimulus in approximately 50 ms but
sensory input are deliberately made indepen- begin to reflect the influence of attention in
dent from the animal’s actions—the response 100–120 ms (Constantinidis & Steinmetz 2001,

Activity during Activity during 50 ms and discriminate targets from distracters

new trial repeated trial
in 138 ms (Thomas & Pare 2007). Neurons in
Cue Cue
dorsal premotor cortex respond to the locations
a of cues instructing two potential movements in
70 ms but begin to predict the monkey’s choice
in 110–130 ms (Cisek & Kalaska 2005).
A recent study by Ledberg et al. (2007)
provides an overall picture of the time
course observed in all of the experiments de-
scribed above. These authors simultaneously
recorded local field potentials (LFPs) from up
to 15 cerebral cortical regions of monkeys
that performed a conditional Go/NoGo task

(Figure 4a). Through an elegant experimen-

b tal design, Ledberg and colleagues identified
the first neural events that responded to the
appearance of a stimulus, those which discrimi-
nated its identity, as well as those that predicted
the monkey’s chosen response (Figure 4b). In
agreement with earlier studies (c.f. Schmolesky
et al. 1998), they observed a fast feedforward
sweep of stimulus onset-related activity appear-
ing within 50–70 ms in striate and extrastri-
ate cortex and 55–80 ms in FEF and premo-
20 Hz
tor cortex. Discrimination of different stimulus
categories occurred later, within approximately
500 ms
Time 100 ms of onset in prestriate areas and 200 ms
in prefrontal sites. The Go/NoGo decision
Figure 3
appeared approximately 150 ms after stimu-
(a) Pooled activity of three phasic PMd neurons during trials in their preferred
direction, aligned on cue onset. The left panel shows activity during trials in lus onset, nearly simultaneously within a di-
which a novel and unpredictable cue is presented, instructing the monkey about verse mosaic of cortical sites including pres-
the required reaching movement. The right panel shows activity from trials triate, inferotemporal, parietal, premotor, and
that follow errors and repeat the same cue at the same location. Because the prefrontal areas.
monkey has learned from experience that the same target will be presented in
In summary, when behavior is experimen-
trials following an error, the presentation of the target stimulus provides it with
no new salient information about movement and does not evoke a neural tally isolated in the lab, the continuous and par-
response. (b) Pooled activity of three tonic PMd neurons, same format. Note allel processes critical for interaction appear as
that in trials following errors, the directionally tuned activity is already present two waves of activation: an early wave crudely
before the cue appears. This reflects the retention of prior knowledge about the specifying a menu of options and a second wave
imminent presentation of the same target after an error. Reprinted with
that selects among them approximately 120–
permission from Crammond & Kalaska (2000).
150 ms after stimulus onset (Ledberg et al.
2007). It appears that the brain can quickly
Treue 2001). FEF neurons respond to the onset specify multiple potential actions within its fast
of a stimulus in 50 ms (Schmolesky et al. 1998), frontoparietal sensorimotor control system, but
but detect the singleton of a visual-search ar- it takes approximately 150 ms (in the case of
ray with a median of approximately 100 ms and simple tasks) to integrate sufficient informa-
discriminate pro- versus antisaccades in approx- tion to make a decision between them. How-
imately 120 ms (Sato & Schall 2003). LIP neu- ever, the apparent serial order of these events is
rons respond to stimulus onset in approximately largely a result of the experimental strategy of

Figure 4
a (a) Anatomical
K
location of electrodes
N L measuring local-field
J A
potentials. (b) Time
O B
M D course of average
I
normalized event-
E
C related potentials from
the electrodes shown
F
in (a). (Solid line)
H Potentials during Go
G
Monkey brain trials. (Dashed line)
electrode locations Potentials during
Go trials NoGo trials. Gray-
b
NoGo trials shaded regions

highlight epochs of
A Striate B Striate C Striate time during which

0.5 0.5 1.0
these differ. Note that
0.5 the earliest responses
0.0 0.0
0.0 to stimuli appear
– 0.5 –0.5 throughout the
–0.5 cerebral cortex after
−1.0 −1.0 −1.0 approximately 50 ms,
D Prestriate E Prestriate F Inferior-temporal and activity begins to
1.0 0.5 0.5 predict the monkey’s
choice after
0.5
0.0 0.0 approximately 150 ms
0.0 in a distributed
–0.5 –0.5 network that includes
– 0.5
parietal and frontal
−1.0 −1.0 −1.0 regions. Reprinted
G Inferior-temporal H Superior-temporal I Superior-temporal with permission from
Normalized voltage
0.5 1.0 1.0 Ledberg et al. (2007).

0.5 0.5
0.0
0.0 0.0
– 0.5
–0.5 –0.5
−1.0 −1.0 −1.0

J Motor K Dorsal-premotor L FEF
1.0 1.0 1.0
0.5 0.5 0.5
0.0 0.0 0.0
– 0.5 –0.5 –0.5
−1.0 −1.0 −1.0

M Prefrontal N Prefrontal O Prefrontal
2 1.0 2
0.5
1 1
0.0
0 0
–0.5
−1 −1.0 −1
Stim 100 200 300 Stim 100 200 300 Stim 100 200 300
Time (ms)

dividing behavior into a sequence of discrete experiments must be conducted in a careful and
and independent trials. During natural activity, quantitative manner to allow the interpretation
all of these events presumably occur continu- of resulting data. But how can one quantita-
ously. tively study natural behavior, which is inher-
These results are consistent with the hy- ently variable and unconstrained?
pothesis that decisions emerge through a dis- One approach is to continue as before.
tributed consensus achieved among recipro- There is no reason why data obtained in a
cally connected frontoparietal regions, each of classical laboratory setting cannot still be
which may contain representations of several interpreted in the broader context of natural
potential actions. This makes a further predic- behavior. For example, the study of Ledberg
tion: The precise order in which decisions ap- et al. (2007) on the timing of cortical processes
pear across the cerebral cortex will be highly was done with head-fixed animals that were
task dependent (Cisek 2006). For example, if observing an impoverished stimulus and mak-
the factors that lead to a decision are bottom- ing a single Go/Nogo response, but its results
up visual features such as stimulus salience, then can still be used to gain valuable insights into
neural correlates of that decision should appear the organization of a flexible parallel system for
first in the parietal cortex and then in frontal interactive behavior. In fact, all of the studies
regions. In contrast, if the biasing factors re- discussed above are still relevant and amenable
quire the kinds of complex stimulus-rule con- to interpretation in terms of interactive be-
junctions that engage neurons in the prefrontal havior. However, we should not mistake our
cortex, then the decision should emerge first in experimental method for the outline of a theory.
frontal regions before propagating back to the Controlled laboratory experiments can also
parietal cortex. Recent studies have supported be designed to be inspired by natural behavior.
that prediction. For example, when monkeys For example, visual-search tasks capture many
perform pop-out visual search tasks, neural ac- aspects of natural foraging activity, which re-
tivity in LIP reflects the choice before FEF, but quires animals to discriminate food (targets)
if the task involves conjunction search then FEF within a cluttered environment (distracters).
reflects the choice before LIP (Buschman & Recent studies by Michael Dorris and col-
Miller 2007). Interestingly, during a Go/NoGo leagues take the analogy further, by presenting
task in which monkeys made decisions on the monkeys with a visual foraging task in which
basis of cognitive rules, activity that predicted they can explore their environment through un-
the response appeared in PMd even before the constrained saccades, making tradeoffs between
prefrontal cortex (PFC) (Wallis & Miller 2003). harvesting rewards by looking at one stimulus
It is as if, at least in that kind of task, a decision versus searching for better payoffs among other
may be influenced by noisy neural votes arriving stimuli (Kan & Dorris 2009).
from the PFC but is determined by a consensus Finally, technical and mathematical ad-
that is reached in a frontoparietal network that vances are starting to make it possible to study
includes the PMd (Pesaran et al. 2008). truly unconstrained behavior while still yield-
ing solid and interpretable data. For exam-
ple, d’Avella & Bizzi (2005) studied motor
FUTURE DIRECTIONS control in frogs by allowing them to freely
We conclude this article by returning to sev- move around their environment—swimming,
eral practical issues. If we hypothesize that jumping, and walking without constraints—
the functional architecture of the brain con- while EMG activity was chronically recorded
sists of simultaneous competing sensorimo- from 13 hindlimb muscles. Through a care-
tor control systems and distributed selection ful analysis of muscular patterns, they extracted
mechanisms—how should we proceed to study the motor synergies that appear to underlie
these processes? Clearly, neurophysiological these natural behaviors. A still more ambitious

approach is to implant wireless multielectrode of the contributors to these views. Our purpose
arrays in the brains of rats (Sodagar et al. 2007) here is to draw attention to how recent neuro-
or monkeys (Chestek et al. 2009) and record physiological experiments lend strong support
ensemble activity during free behavior. How- to these alternative ways of thinking about the
ever, unconstrained behavior requires novel brain.
ways of analyzing and thinking about our data. The proposals we review above address in-
In particular, we need methods that move away teractive sensorimotor behavior and are not
from the standard approach of averaging over meant to constitute a general theory of all
similar trials and toward analyzing behavior brain function. Clearly, tasks such as writing
and the neural firing patterns of many neu- an email or playing chess involve processes
rons during each individual action (Yu et al. that are far removed from simple sensorimo-
2009). tor control. Nevertheless, theories of embod-
ied cognition have suggested, following Piaget,

that cognitive abilities may have evolved within

CONCLUSIONS
the context of ancestral abilities for interact-
One of the major goals of neuroscience research ing with the world (Hendriks-Jansen 1996,
is to reveal the brain’s functional architecture Pezzulo & Castelfranchi 2009, Powers 1973,
to build a theoretical framework that bridges Thelen et al. 2001, Toates 1998). For example,
the brain and behavior (Schall 2004). In re- Toates (1998) proposed that primitive switch-
cent decades, an influential framework has been ing mechanisms, which mediate between dif-
based on concepts developed in cognitive psy- ferent stimulus-response associations, have be-
chology, which were originally intended to ex- come elaborated through evolution into the
plain human abstract problem solving behavior. cognitive systems that now allow us to make
These led to a functional architecture of infor- complex and sophisticated decisions. Pezzulo
mation processing stages that can be roughly & Castelfranchi (2009) suggest that our ability
categorized as perceptual, cognitive, and mo- to think about the world results from the in-
tor control modules. However, as we review ternalization of the processes of predicting the
above, a growing number of neurophysiolog- consequences of actions. Their cognitive lever-
ical studies in nonhuman primates appear dif- age hypothesis proposes that as the sensorimo-
ficult to interpret from this perspective. This tor control system gradually evolved, it began
motivates us to consider alternative theoretical to predict increasingly abstract consequences of
frameworks. Above, we consider proposals de- behavior. This eventually allowed the mental
veloped for many decades in fields such as ethol- rehearsal of entire sequences of acts and evalua-
ogy, ecological psychology, and autonomous tion of their potential outcomes, without overt
robotics research, which were designed to ex- motor activity. Their hypothesis is consistent
plain the original and still primary purpose of with the close relationship between mental im-
the brain—to endow organisms with the abil- agery and the systems for motor preparation
ity to adaptively interact with their environ- (Cisek & Kalaska 2004, di Pellegrino et al. 1992,
ment. We discuss diverse neurophysiological Rizzolatti & Craighero 2004, Umilta et al.
data on the control of voluntary behavior that, 2001) and potentially explains how an organ-
in our opinion, appear to be more compatible ism may go beyond merely reacting to proper-
with these alternative frameworks. This leads ties of the immediate environment and act in a
to the claim that perhaps they provide a better goal-directed manner. In conclusion, the neural
foundation for interpreting neural data and de- systems that mediate the sensorimotor behavior
signing future experiments. We acknowledge of our ancient ancestors may have provided the
that similar claims have been made many times foundations for modern cognitive abilities, and
in the past, but limitations of space and our their consideration may shed light on the neural
own knowledge make it impossible to list all mechanisms that underlie human thought.

SUMMARY POINTS
1. Brains evolved for sensorimotor control and retained much of that architecture—even
the neocortex is still part of that old circuit.
2. Natural interactive behavior requires sensorimotor control and selection systems to op-
erate continuously and in parallel.
3. Distinctions between perceptual, cognitive, and motor processes, although descriptively
useful, might not reflect the natural categories of the brain’s functional organization.
4. Decisions appear to be made through a distributed consensus that emerges in competitive
populations.
5. Neurophysiological data may be more readily interpreted from the perspective of inter-
active behavior than from the perspective of serial information processing.
FUTURE ISSUES
1. What are possible experimental approaches for studying behavior without constraining
its interactive nature? Although some behavioral abilities are already studied in a relatively
natural setting (e.g., locomotion), other systems demand new technologies for wireless
multi-unit recording and new methods for analyzing data.
2. For a deeper understanding of the evolution of modern behavioral abilities, we would
like to reconstruct the sequence of phylogenetic elaborations of a given system along a
particular branch of the evolutionary tree. This calls for comparative neurophysiological
studies of a diverse set of related species. However, this poses a significant challenge,
not only because homologies are difficult to establish but also because practical matters
motivate scientists to study particular species whose brains are already well mapped (e.g.,
rats, cats, and macaques).
3. From a theoretical standpoint, we need models that explain how sensorimotor control
systems could have become elaborated to implement more sophisticated behavior. For
example, if action selection takes place within a space defined by movement parameters
(e.g., reach direction), what are the parameter spaces in which high-level decisions are
made? Do these high-level spaces maintain topology, similar to the somatotopy and
spatial maps useful for action selection?
4. How can an advanced agent discover the high-level opportunities afforded within its
behavioral niche and link them with long-term goals?
DISCLOSURE STATEMENT
The authors are not aware of any affiliations, memberships, funding, or financial holdings that
might be perceived as affecting the objectivity of this review.

ACKNOWLEDGMENTS
This article is dedicated to Steve Wise, a friend and colleague whose work has had a major
influence on our thinking about the brain. We thank Ignasi Cos, Trevor Drew, Andrea Green,
and Christopher Pack for their comments and their contributions to the development of the
ideas described in this paper. We apologize in advance to all of the investigators whose relevant
research could not be cited owing to space limitations. Our research is supported by grants from
the Canadian Institutes of Health Research, the Fonds de la Recherche en Santé du Québec, the
Natural Sciences and Engineering Research Council of Canada, the National Institutes of Health,
the Canadian Foundation for Innovation, and the EJLB Foundation.
LITERATURE CITED
Albright TD, Kandel ER, Posner MI. 2000. Cognitive neuroscience. Curr. Opin. Neurobiol. 10(5):612–24
Alexander GE, Crutcher MD. 1990a. Functional architecture of basal ganglia circuits: neural substrates of
parallel processing. TINS 13(7):266–71
Alexander GE, Crutcher MD. 1990b. Neural representations of the target (goal) of visually guided arm
movements in three motor areas of the monkey. J. Neurophysiol. 64(1):164–78
Allport DA. 1987. Selection for action: some behavioral and neurophysiological considerations of attention
and action. In Perspectives on Perception and Action, ed. H Heuer, AF Sanders, pp. 395–419. Hillsdale, NJ:
Erlbaum
Andersen RA, Buneo CA. 2003. Sensorimotor integration in posterior parietal cortex. Adv. Neurol. 93:159–77
Andersen RA, Snyder LH, Bradley DC, Xing J. 1997. Multimodal representation of space in the posterior
parietal cortex and its use in planning movements. Annu. Rev. Neurosci. 20:303–30
Ashby WR. 1965. Design for a Brain: The Origin of Adaptive Behavior. London: Chapman and Hall
Barraclough DJ, Conroy ML, Lee D. 2004. Prefrontal cortex and decision making in a mixed-strategy game.
Nat. Neurosci. 7(4):404–10
Basso MA, Wurtz RH. 1998. Modulation of neuronal activity in superior colliculus by changes in target
probability. J. Neurosci. 18(18):7519–34
Bastian A, Riehle A, Erlhagen W, Schöner G. 1998. Prior information preshapes the population representation
of movement direction in motor cortex. Neuroreport 9:315–19
Baumann MA, Fluet MC, Scherberger H. 2009. Context-specific grasp movement representation in the
macaque anterior intraparietal area. J. Neurosci. 29(20):6436–48
Bock O, Eversheim U. 2000. The mechanisms of movement preparation: a precuing study. Behav. Brain Res.
108(1):85–90
Boynton GM. 2005. Attention and visual perception. Curr. Opin. Neurobiol. 15(4):465–69
Broadbent DE. 1958. Perception and Communication. New York: Pergamon
Brooks R. 1991. Intelligence without representation. Artif. Intell. 47:139–59
Brown JW, Bullock D, Grossberg S. 2004. How laminar frontal cortex and basal ganglia circuits interact to
control planned and reactive saccades. Neural Networks 17(4):471–510
Buch ER, Brasted PJ, Wise SP. 2006. Comparison of population activity in the dorsal premotor cortex and
putamen during the learning of arbitrary visuomotor mappings. Exp. Brain Res. 169(1):69–84
Buneo CA, Jarvis MR, Batista AP, Andersen RA. 2002. Direct visuomotor transformations for reaching. Nature
416(6881):632–36
Buschman TJ, Miller EK. 2007. Top-down versus bottom-up control of attention in the prefrontal and
posterior parietal cortices. Science 315(5820):1860–62
Bushnell MC, Goldberg ME, Robinson DL. 1981. Behavioral enhancement of visual responses in monkey
cerebral cortex. I. Modulation in posterior parietal cortex related to selective visual attention. J. Neuro-
physiol. 46(4):755–72
Butler AB, Hodos W. 2005. Comparative Vertebrate Neuroanatomy: Evolution and Adaptation. New York:
Wiley-Liss

Calton JL, Dickinson AR, Snyder LH. 2002. Non-spatial, motor-specific activation in posterior parietal cortex.
Nat. Neurosci. 5(6):580–88
Carello CD, Krauzlis RJ. 2004. Manipulating intent: evidence for a causal role of the superior colliculus in
target selection. Neuron 43(4):575–83
Castiello U. 1999. Mechanisms of selection for the control of hand action. Trends. Cogn. Sci. 3(7):264–71
Chestek CA, Gilja V, Nuyujukian P, Kier RJ, Solzbacher F et al. 2009. HermesC: low-power wireless neural
recording system for freely moving primates. IEEE Trans. Neural Syst. Rehabil. Eng. 17(4):330–38
Cisek P. 2007. Cortical mechanisms of action selection: the affordance competition hypothesis. Philos. Trans.
R. Soc. Lond. B Biol. Sci. 362(1485):1585–99
Cisek P. 2006. Integrated neural processes for defining potential actions and deciding between them: a com-
putational model. J. Neurosci. 26(38):9761–70
Cisek P, Crammond DJ, Kalaska JF. 2003. Neural activity in primary motor and dorsal premotor cortex in
reaching tasks with the contralateral versus ipsilateral arm. J. Neurophysiol. 89(2):922–42
Cisek P, Kalaska JF. 2004. Neural correlates of mental rehearsal in dorsal premotor cortex. Nature
431(7011):993–96
Cisek P, Kalaska JF. 2005. Neural correlates of reaching decisions in dorsal premotor cortex: Specification of
multiple direction choices and final selection of action. Neuron 45(5):801–14
Clark A. 1997. Being There: Putting Brain, Body, and World Together Again. Cambridge, MA: MIT Press
Coe B, Tomihara K, Matsuzawa M, Hikosaka O. 2002. Visual and anticipatory bias in three cortical eye fields
of the monkey during an adaptive decision-making task. J. Neurosci. 22(12):5081–90
Colby CL, Duhamel JR. 1996. Spatial representations for action in parietal cortex. Brain Res. Cogn. Brain Res.
5(1–2):105–15
Colby CL, Goldberg ME. 1999. Space and attention in parietal cortex. Annu. Rev. Neurosci. 22:319–49
Coles MG, Gratton G, Bashore TR, Eriksen CW, Donchin E. 1985. A psychophysiological investigation
of the continuous flow model of human information processing. J. Exp. Psychol. Hum. Percept. Perform.
11(5):529–53
Constantinidis C, Steinmetz MA. 2001. Neuronal responses in area 7a to multiple-stimulus displays. I. Neurons
encode the location of the salient stimulus. Cereb. Cortex. 11(7):581–91
Crammond DJ, Kalaska JF. 2000. Prior information in motor and premotor cortex: activity during the delay
period and effect on premovement activity. J. Neurophysiol. 84(2):986–1005
Crammond DJ, Kalaska JF. 1994. Modulation of preparatory neuronal activity in dorsal premotor cortex due
to stimulus-response compatibility. J. Neurophysiol. 71(3):1281–84
Cui H, Andersen RA. 2007. Posterior parietal cortex encodes autonomously selected motor plans. Neuron
56(3):552–59
Culham JC, Kanwisher NG. 2001. Neuroimaging of cognitive functions in human parietal cortex. Curr. Opin.
Neurobiol. 11(2):157–63
d’Avella A, Bizzi E. 2005. Shared and specific muscle synergies in natural motor behaviors. Proc. Natl. Acad.
Sci. USA 102(8):3076–81
Desimone R, Duncan J. 1995. Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18:193–222
Dewey J. 1896. The reflex arc concept in psychology. Psychol. Rev. 3(4):357–70
di Pellegrino G, Fadiga L, Fogassi L, Gallese V, Rizzolatti G. 1992. Understanding motor events: a neuro-
physiological study. Exp. Brain Res. 91(1):176–80
Dorris MC, Glimcher PW. 2004. Activity in posterior parietal cortex is correlated with the relative subjective
desirability of action. Neuron 44(2):365–78
Engel AK, Fries P, Singer W. 2001. Dynamic predictions: oscillations and synchrony in top-down processing.
Nat. Rev. Neurosci. 2(10):704–16
Erlhagen W, Schöner G. 2002. Dynamic field theory of movement preparation. Psychol. Rev. 109(3):545–72
Ewert J-P. 1997. Neural correlates of key stimulus and releasing mechanism: a case study and two concepts.
TINS 20(8):332–39
Fadiga L, Fogassi L, Gallese V, Rizzolatti G. 2000. Visuomotor neurons: ambiguity of the discharge or ‘motor’
perception? Int. J. Psychophysiol. 35(2–3):165–77
Fagg AH, Arbib MA. 1998. Modeling parietal-premotor interactions in primate control of grasping. Neural
Netw. 11(7–8):1277–303

Farmer TA, Cargill SA, Spivey MJ. 2007. Gradiency and visual context in syntactic garden-paths. J. Mem.
Lang. 57(4):570–95
Felleman DJ, Van Essen DC. 1991. Distributed hierarchical processing in the primate cerebral cortex. Cereb.
Cortex 1(1):1–47
Fitts PM. 1954. The information capacity of the human motor system in controlling the amplitude of move-
ment. J. Exp. Psychol. 47(6):381–91
Fodor JA. 1983. Modularity of Mind. Cambridge, MA: MIT Press
Furman M, Wang XJ. 2008. Similarity effect and optimal control of multiple-choice decision making. Neuron
60(6):1153–68
Fuster JM, Bodner M, Kroger JK. 2000. Cross-modal and cross-temporal association in neurons of frontal
cortex. Nature 405(6784):347–51
Gail A, Klaes C, Westendorff S. 2009. Implementation of spatial transformation rules for goal-directed reach-
ing via gain modulation in monkey parietal and premotor cortex. J. Neurosci. 29(30):9490–99
Gallivan JP, Cavina-Pratesi C, Culham JC. 2009. Is that within reach? fMRI reveals that the human superior
parieto-occipital cortex encodes objects reachable by the hand. J. Neurosci. 29(14):4381–91

The definitive work on
Gazzaniga MS. 2000. The New Cognitive Neurosciences. Cambridge, MA: MIT Press
ecological psychology
Ghez C, Favilla M, Ghilardi MF, Gordon J, Bermejo R, Pullman S. 1997. Discrete and continuous planning and the concept of
of hand movements and isometric force trajectories. Exp. Brain Res. 115(2):217–33 affordances.
Gibson JJ. 1979. The Ecological Approach to Visual Perception. Boston: Houghton Mifflin
Glimcher PW. 2003. The neurobiology of visual-saccadic decision making. Annu. Rev. Neurosci. 26:133–
79 An excellent review of
Gold JI, Shadlen MN. 2007. The neural basis of decision making. Annu. Rev. Neurosci. 30:535– research on the neural
74 mechanisms of
Goodale MA, Milner AD. 1992. Separate visual pathways for perception and action. TINS 15(1):20–25 decisions, particularly in
Gottlieb JP, Kusunoki M, Goldberg ME. 1998. The representation of visual salience in monkey parietal cortex. the saccadic system.
Nature 391(6666):481–84
Graziano MS, Aflalo TN. 2007. Mapping behavioral repertoire onto the cortex. Neuron 56(2):239–51
Hauser MD. 1999. Perseveration, inhibition and the prefrontal cortex: a new look. Curr. Opin. Neurobiol.
9(2):214–22
Hazy TE, Frank MJ, O’Reilly RC. 2007. Towards an executive without a homunculus: computational models
of the prefrontal cortex/basal ganglia system. Philos. Trans. R. Soc. Lond. B Biol. Sci. 362(1485):1601–13
Heekeren HR, Marrett S, Ungerleider LG. 2008. The neural systems that mediate human perceptual decision
making. Nat. Rev. Neurosci. 9(6):467–79
Held R, Hein A. 1963. Movement-produced stimulation in the development of visually guided behavior.
J. Comp. Physiol. Psychol. 56(5):872–76 An outstanding
Hendriks-Jansen H. 1996. Catching Ourselves in the Act: Situated Activity, Interactive Emergence, Evo- synthesis of ethological,
lution, and Human Thought. Cambridge, MA: MIT Press evolutionary, and
Hernandez A, Zainos A, Romo R. 2002. Temporal evolution of a decision-making process in medial premotor developmental
cortex. Neuron 33(6):959–72 literature and
Hinde RA. 1966. Animal Behavior: A Synthesis of Ethology and Comparative Psychology. New York: McGraw-Hill implications for
Holland LZ, Holland ND. 1999. Chordate origins of the vertebrate central nervous system. Curr. Opin. large-scale brain
theories.
Neurobiol. 9(5):596–602
Horwitz GD, Batista AP, Newsome WT. 2004. Representation of an abstract perceptual decision in macaque
superior colliculus. J. Neurophysiol. 91(5):2281–96
Hoshi E, Tanji J. 2007. Distinctions between dorsal and ventral premotor areas: anatomical connectivity and
functional properties. Curr. Opin. Neurobiol. 17(2):234–42
Hoshi E, Shima K, Tanji J. 1998. Task-dependent selectivity of movement-related neuronal activity in the
primate prefrontal cortex. J. Neurophysiol. 80(6):3392–97
Hoshi E, Shima K, Tanji J. 2000. Neuronal activity in the primate prefrontal cortex in the process of motor
selection based on two behavioral rules. J. Neurophysiol. 83(4):2355–73
Hughlings Jackson J. 1884. Evolution and dissolution of the nervous system. In Selected Writings of John
Hughlings Jackson, ed. J Taylor, pp. 45–75. London: Staples

Humphreys GW, Riddoch JM. 2000. One more cup of coffee for the road: object-action assemblies, response
blocking and response capture after frontal lobe damage. Exp. Brain Res. 133:81–93
Iriki A, Tanaka M, Iwamura Y. 1996. Coding of modified body schema during tool use by macaque postcentral
neurones. Neuroreport 7(14):2325–30
Johnson PB, Ferraina S, Bianchi L, Caminiti R. 1996. Cortical networks for visual reaching: physiological and
anatomical organization of frontal and parietal arm regions. Cereb. Cortex 6(2):102–19
Johnson-Laird PN. 1988. The Computer and the Mind: An Introduction to Cognitive Science. Cambridge, MA:
Harvard Univ. Press
Kalaska JF. 1996. Parietal cortex area 5 and visuomotor behavior. Can. J. Physiol. Pharmacol. 74:483–98
Kalaska JF, Crammond DJ. 1995. Deciding not to GO: neuronal correlates of response selection in a
GO/NOGO task in primate premotor and parietal cortex. Cereb. Cortex 5:410–28
Kalaska JF, Scott SH, Cisek P, Sergio LE. 1997. Cortical control of reaching movements. Curr. Opin. Neurobiol.
7:849–59
Kalaska JF, Sergio LE, Cisek P. 1998. Cortical control of whole-arm motor tasks. In Sensory Guidance of
Movement, Novartis Foundation Symposium #218, ed. M Glickstein, pp. 176–201. Chichester, UK: Wiley
Kalivas PW, Nakamura M. 1999. Neural systems for behavioral activation and reward. Curr. Opin. Neurobiol.
9(2):223–27
Kan JY, Dorris MC. 2009. Superior colliculus activity reflects saccade profitability during visual foraging task.
Progr. No. 285.12 2009 Neuroscience Meeting Planner. Chicago IL: Soc. Neurosci. Online
Karten HJ. 1969. The organization of the avian telencephalon and some speculations on the phylogeny of the
amniote telencephalon. Ann. N. Y. Acad. Sci. 167(1):164–79
Katz PS, Harris-Warrick RM. 1999. The evolution of neuronal circuits underlying species-specific behavior.
Curr. Opin. Neurobiol. 9(5):628–33
Keele SW. 1968. Movement control in skilled motor performance. Psychol. Bull. 70:387–403
Kim B, Basso MA. 2008. Saccade target selection in the superior colliculus: a signal detection theory approach.
J. Neurosci. 28(12):2991–3007
Kim J-N, Shadlen MN. 1999. Neural correlates of a decision in the dorsolateral prefrontal cortex of the
macaque. Nat. Neurosci. 2(2):176–85
Klatzky R, Behrmann M, MacWhinney B. 2008. Embodiment, Ego-Space and Action. New York: Psychology
Press
Kornblum S, Hasbroucq T, Osman A. 1990. Dimensional overlap: cognitive basis for stimulus-response
compatibility—a model and taxonomy. Psychol. Rev. 97(2):253–70
Kurata K. 1993. Premotor cortex of monkeys: set- and movement-related activity reflecting amplitude and
direction of wrist movements. J. Neurophysiol. 69(1):187–200
Kusunoki M, Gottlieb J, Goldberg ME. 2000. The lateral intraparietal area as a salience map: the representation
of abrupt onset, stimulus motion, and task relevance. Vision Res. 40(10–12):1459–68
Lebedev MA, Wise SP. 2002. Insights into seeing and grasping: distinguishing the neural correlates of per-
ception and action. Behav. Cogn. Neurosci. Rev. 1:108–29
Leblois A, Boraud T, Meissner W, Bergman H, Hansel D. 2006. Competition between feedback loops un-
An important derlies normal and pathological dynamics in the basal ganglia. J. Neurosci. 26(13):3567–83
electrophysiological Ledberg A, Bressler SL, Ding M, Coppola R, Nakamura R. 2007. Large-scale visuomotor integration
study showing the in the cerebral cortex. Cereb. Cortex 17(1):44–62
time-course of neural Marr DC. 1982. Vision. San Francisco: W. H. Freeman
processing across Mazzoni P, Bracewell RM, Barash S, Andersen RA. 1996. Motor intention activity in the macaque’s lateral
diverse regions of the intraparietal area. I. Dissociation of motor plan from sensory memory. J. Neurophysiol. 76:1439–57
primate cerebral cortex. McKinstry C, Dale R, Spivey MJ. 2008. Action dynamics reveal parallel competition in decision making.
Psychol. Sci. 19(1):22–24
McPeek RM, Keller EL. 2002. Superior colliculus activity related to concurrent processing of saccade goals
in a visual search task. J. Neurophysiol. 87(4):1805–15
McPeek RM, Skavenski AA, Nakayama K. 2000. Concurrent processing of saccades in visual search. Vision
Res. 40(18):2499–516
Medina L, Reiner A. 2000. Do birds possess homologues of mammalian primary visual, somatosensory and
motor cortices? TINS 23(1):1–12

Merleau-Ponty M. 1945. Phénoménologie de la Perception. Paris: Gallimard

Meyer J-A. 1995. The animat approach to cognitive science. In Comparative Approaches to Cognitive Science, ed.
HL Roitblat, J-A Meyer, pp. 27–44. Cambridge, MA: MIT Press A detailed review of the
Middleton FA, Strick PL. 2000. Basal ganglia and cerebellar loops: motor and cognitive circuits. Brain anatomical circuits
Res. Rev. 31(2–3):236–50 between the cerebral
Miller EK. 2000. The prefrontal cortex and cognitive control. Nat. Rev. Neurosci. 1(1):59–65 cortex, cerebellum, and
Miller GA. 1956. The magical number seven, plus or minus two: some limits on our capacity for processing the basal ganglia.
information. Psychol. Rev. 63:81–97
Miller GA, Galanter E, Pribram KH. 1960. Plans and the Structure of Behavior. New York: Holt, Rinehart and
Winston An influential synthesis
Milner AD, Goodale MA. 1995. The Visual Brain in Action. Oxford, UK: Oxford Univ. Press of research on the
Mink JW. 1996. The basal ganglia: focused selection and inhibition of competing motor programs. Prog. dorsal and ventral visual
Neurobiol. 50(4):381–425 pathways and their
Moran J, Desimone R. 1985. Selective attention gates visual processing in the extrastriate cortex. Science functional
229:782–84 interpretation.
Mountcastle VB, Andersen RA, Motter BC. 1981. The influence of attentive fixation upon the excitability of
the light-sensitive neurons of the posterior parietal cortex. J. Neurosci. 1(11):1218–25
Mountcastle VB, Lynch JC, Georgopoulos AP, Sakata H, Acuna C. 1975. Posterior parietal association cortex
of the monkey: command functions for operations within extrapersonal space. J. Neurophysiol. 38(4):871–
908
Nakamura H, Kuroda T, Wakita M, Kusunoki M, Kato A et al. 2001. From three-dimensional space vision to
prehensile hand movements: the lateral intraparietal area links the area V3A and the anterior intraparietal
area in macaques. J. Neurosci. 21(20):8174–87
Neumann O. 1990. Visual attention and action. In Relationships Between Perception and Action: Current Ap-
proaches, ed. O Neumann, W Prinz, pp. 227–267. Berlin: Springer-Verlag
Newell A, Simon HA. 1972. Human Problem Solving. Englewood Cliffs, NJ: Prentice-Hall
Núñez R, Freeman WJ. 2000. Reclaiming Cognition: The Primacy of Action, Intention and Emotion. Thorverton:
Imprint Academic
Padoa-Schioppa C. 2009. Range-adapting representation of economic value in the orbitofrontal cortex.
J. Neurosci. 29(44):14004–14
Padoa-Schioppa C, Assad JA. 2008. The representation of economic value in the orbitofrontal cortex is
invariant for changes of menu. Nat. Neurosci. 11(1):95–102
Paradiso MA. 2002. Perceptual and neuronal correspondence in primary visual cortex. Curr. Opin. Neurobiol.
12(2):155–61
Paré M, Wurtz RH. 2001. Progression in neuronal processing for saccadic eye movements from parietal cortex
area LIP to superior colliculus. J. Neurophysiol. 85(6):2545–62
Passingham RE, Toni I. 2001. Contrasting the dorsal and ventral visual systems: guidance of movement versus
decision making. Neuroimage 14(1 Pt. 2):S125–31
Pasupathy A, Connor CE. 2002. Population coding of shape in area V4. Nat. Neurosci. 5(12):1332–38
Pesaran B, Nelson MJ, Andersen RA. 2008. Free choice activates a decision circuit between frontal and parietal
cortex. Nature 453(7193):406–9 A control-theoretic
Pezzulo G, Castelfranchi C. 2009. Thinking as the control of imagination: a conceptual framework perspective on how
for goal-directed systems. Psychol. Res. 73(4):559–77 sensorimotor control
Piaget J. 1954. The Construction of Reality in the Child. New York: Basic Books systems may have
Platt ML, Glimcher PW. 1999. Neural correlates of decision variables in parietal cortex. Nature evolved toward more
400(6741):233–38 sophisticated planning.
Platt ML, Glimcher PW. 1997. Responses of intraparietal neurons to saccadic targets and visual distractors.
J. Neurophysiol. 78(3):1574–89
Powell KD, Goldberg ME. 2000. Response of neurons in the lateral intraparietal area to a distractor flashed
during the delay period of a memory-guided saccade. J. Neurophysiol. 84(1):301–10
Powers WT. 1973. Behavior: The Control of Perception. New York: Aldine Publ.
Pylyshyn ZW. 1984. Computation and Cognition: Toward a Foundation for Cognitive Science. Cambridge, MA:
MIT Press

Quintana J, Fuster JM. 1999. From perceptions to actions: temporal integrative functions of prefrontal and
parietal neurons. Cereb. Cortex 9(3):213–21
Rainer G, Asaad WF, Miller EK. 1998. Selective representation of relevant information by neurons in the
primate prefrontal cortex. Nature 363(6885):577–79
Ratcliff R, Hasegawa YT, Hasegawa RP, Smith PL, Segraves MA. 2007. Dual diffusion model for single-cell
recording data from the superior colliculus in a brightness-discrimination task. J. Neurophysiol. 97(2):1756–
74
Redgrave P, Prescott TJ, Gurney K. 1999. The basal ganglia: a vertebrate solution to the selection problem?
Neuroscience 89(4):1009–23
Resulaj A, Kiani R, Wolpert DM, Shadlen MN. 2009. Changes of mind in decision-making. Nature
461(7261):263–66
Riehle A, Requin J. 1989. Monkey primary motor and premotor cortex: single-cell activity related to prior
information about direction and extent of an intended movement. J. Neurophysiol. 61(3):534–49
Riesenhuber M, Poggio T. 1999. Are cortical models really bound by the “binding problem”? Neuron 24(1):87–
93
Riesenhuber M, Poggio T. 2002. Neural mechanisms of object recognition. Curr. Opin. Neurobiol. 12(2):162–
68
Rizzolatti G, Craighero L. 2004. The mirror-neuron system. Annu. Rev. Neurosci. 27(1):169–92
Rizzolatti G, Luppino G. 2001. The cortical motor system. Neuron 31(6):889–901
Roitman JD, Shadlen MN. 2002. Response of neurons in the lateral intraparietal area during a combined
visual discrimination reaction time task. J. Neurosci. 22(21):9475–89
Romo R, Hernandez A, Zainos A. 2004. Neuronal correlates of a perceptual decision in ventral premotor
cortex. Neuron 41(1):165–73
Romo R, Hernandez A, Zainos A, Lemus L, Brody CD. 2002. Neuronal correlates of decision-making in
secondary somatosensory cortex. Nat. Neurosci. 5(11):1217–25
Rowe JB, Toni I, Josephs O, Frackowiak RS, Passingham RE. 2000. The prefrontal cortex: response selection
or maintenance within working memory? Science 288(5471):1656–60
Sahin E, Çakmak M, Dogar MR, Ugur E, Üçoluk G. 2007. To afford or not to afford: a new formalization of
affordances toward affordance-based robot control. Adaptive Behav. 15(4):447–71
Sakagami M, Pan X. 2007. Functional role of the ventrolateral prefrontal cortex in decision making. Curr.
Opin. Neurobiol. 17(2):228–33
Sanger TD. 2003. Neural population codes. Curr. Opin. Neurobiol. 13(2):238–49
Sato TR, Schall JD. 2003. Effects of stimulus-response compatibility on neural selection in frontal eye field.
Neuron 38(4):637–48
Sawamoto N, Honda M, Hanakawa T, Fukuyama H, Shibasaki H. 2002. Cognitive slowing in Parkinson’s
disease: a behavioral evaluation independent of motor slowing. J. Neurosci. 22(12):5198–203
Schall JD. 2004. On building a bridge between brain and behavior. Annu. Rev. Psychol. 55:23–50
Schall JD, Bichot NP. 1998. Neural correlates of visual and motor decision processes. Curr. Opin. Neurobiol.
8:211–17
Scherberger H, Andersen RA. 2007. Target selection signals for arm reaching in the posterior parietal cortex.
J. Neurosci. 27(8):2001–12
Schmolesky MT, Wang Y, Hanes DP, Thompson KG, Leutgeb S et al. 1998. Signal timing across the macaque
visual system. J. Neurophysiol. 79(6):3272–78
Schultz W. 2004. Neural coding of basic reward terms of animal learning theory, game theory, microeconomics
and behavioural ecology. Curr. Opin. Neurobiol. 14(2):139–47
A broad discussion of a Schultz W, Tremblay L, Hollerman JR. 2000. Reward processing in primate orbitofrontal cortex and basal
general theoretical ganglia. Cereb. Cortex 10(3):272–84
framework for Shadlen MN, Kiani R, Hanks TD, Churchland AK. 2008. Neurobiology of decision making: An inten-
decision-making, and tional framework. In Better than Conscious? Decision Making, the Human Mind, and Implications
implications for human for Institutions, ed. C Engel, W Singer, pp. 71–101. Cambridge, MA: MIT Press
cognitive function. Shadlen MN, Movshon JA. 1999. Synchrony unbound: a critical evaluation of the temporal binding hypothesis.
Neuron 24(1):67–25

Shafir E, Tversky A. 1995. Decision making. In Thinking: An Invitation to Cognitive Science, ed. EE Smith, DN
Osherson, pp. 77–100. Cambridge, MA: MIT Press
Shen L, Alexander GE. 1997. Neural correlates of a spatial sensory-to-motor transformation in primary motor
cortex. J. Neurophysiol. 77:1171–94
Singer W. 2001. Consciousness and the binding problem. Ann. N. Y. Acad. Sci. 929:123–46
Smith EE, Osherson DN. 1995. Thinking: An Invitation to Cognitive Science. Cambridge, MA: MIT Press
Snyder LH, Batista AP, Andersen RA. 1998. Change in motor plan, without a change in the spatial locus of
attention, modulates activity in posterior parietal cortex. J. Neurophysiol. 79(5):2814–19
Snyder LH, Batista AP, Andersen RA. 2000. Intention-related activity in the posterior parietal cortex: a review.
Vision Res. 40(10–12):1433–41
Sodagar AM, Wise KD, Najafi K. 2007. A fully integrated mixed-signal neural processor for implantable
multichannel cortical recording. IEEE Trans. Biomed. Eng. 54(6):1075–88
Song JH, Nakayama K. 2008. Target selection in visual search as revealed by movement trajectories. Vision
Res. 48:853–61
Song JH, Nakayama K. 2009. Hidden cognitive states revealed in choice reaching tasks. Trends Cogn Sci.
13(8):360–66
Stein JF. 1992. The representation of egocentric space in the posterior parietal cortex. Behav. Brain Sci.
15:691–700
Sugase Y, Yamane S, Ueno S, Kawano K. 1999. Global and fine information coded by single neurons in the
temporal visual cortex. Nature 400(6747):869–73
Sugrue LP, Corrado GS, Newsome WT. 2004. Matching behavior and the representation of value in the
parietal cortex. Science 304(5678):1782–87
Takikawa Y, Kawagoe R, Hikosaka O. 2002. Reward-dependent spatial selectivity of anticipatory activity in
monkey caudate neurons. J. Neurophysiol. 87(1):508–15
Tanaka K, Saito H-A, Fukada Y, Moriya M. 1991. Coding visual images of objects in the inferotemporal
cortex of the macaque monkey. J. Neurophysiol. 66(1):170–89
Tanji J, Hoshi E. 2001. Behavioral planning in the prefrontal cortex. Curr. Opin. Neurobiol. 11(2):164–70
Thelen E, Schöner G, Scheier C, Smith LB. 2001. The dynamics of embodiment: a field theory of infant
perseverative reaching. Behav. Brain Sci. 24(1):1–34
Thevarajah D, Mikulic A, Dorris MC. 2009. Role of the superior colliculus in choosing mixed-strategy sac-
cades. J. Neurosci. 29(7):1998–2008
Thomas NW, Pare M. 2007. Temporal processing of saccade targets in parietal cortex area LIP during visual
search. J. Neurophysiol. 97(1):942–47
Tinbergen N. 1950. The hierarchical organisation of nervous mechanisms underlying instinctive behavior.
Symp. Soc. Exp. Biol. 4:305–12
Tipper SP, Howard LA, Houghton G. 2000. Behavioural consequences of selection from neural population
codes. In Control of Cognitive Processes: Attention and Performance XVIII, ed. S Monsell, J Driver, pp. 223–45.
Cambridge, MA: MIT Press
Toates F. 1998. The interaction of cognitive and stimulus-response processes in the control of behavior.
Neurosci. Biobehav. Rev. 22(1):59–83
Treue S. 2001. Neural correlates of attention in primate visual cortex. Trends Neurosci. 24(5):295–300
Trommershauser J, Landy MS, Maloney LT. 2006. Humans rapidly estimate expected gain in movement
planning. Psychol. Sci. 17(11):981–88
Tversky A, Kahneman D. 1981. The framing of decisions and the psychology of choice. Science 211:453–58
Umilta MA, Kohler E, Gallese V, Fogassi L, Fadiga L et al. 2001. I know what you are doing. a neurophysi-
ological study. Neuron 31(1):155–65
Ungerleider LG, Mishkin M. 1982. Two cortical visual systems. In Analysis of Visual Behavior, ed. DJ Ingle,
MA Goodale, RJW Mansfield, 18:549–586. Cambridge, MA: MIT Press
Wallis JD. 2007. Orbitofrontal cortex and its contribution to decision-making. Annu. Rev. Neurosci. 30:31–56
Wallis JD, Miller EK. 2003. From rule to response: neuronal processes in the premotor and prefrontal cortex.
J. Neurophysiol. 90(3):1790–806
Welsh TN, Elliott D, Weeks DJ. 1999. Hand deviations toward distractors. Evidence for response competition.
Exp. Brain Res. 127(2):207–12

Wenger KK, Musch KL, Mink JW. 1999. Impaired reaching and grasping after focal inactivation of Globus
Pallidus pars interna in the monkey. J. Neurophysiol. 82(5):2049–60
Wise SP. 2008. Forward frontal fields: phylogeny and fundamental function. Trends Neurosci. 31(12):599–608
Wise SP, Boussaoud D, Johnson PB, Caminiti R. 1997. Premotor and parietal cortex: corticocortical connec-
tivity and combinatorial computations. Annu. Rev. Neurosci. 20:25–42
Wise SP, di Pellegrino G, Boussaoud D. 1996. The premotor cortex and nonstandard sensorimotor mapping.
Can. J. Physiol. Pharmacol. 74:469–82
Yang T, Shadlen MN. 2007. Probabilistic reasoning by neurons. Nature 447(7148):1075–80
Yu BM, Cunningham JP, Santhanam G, Ryu SI, Shenoy KV, Sahani M. 2009. Gaussian-process factor analysis
for low-dimensional single-trial analysis of neural population activity. J. Neurophysiol. 102(1):614–35

NE33-FM ARI 14 May 2010 19:47
Annual Review of
Neuroscience
Contents Volume 33, 2010

Attention, Intention, and Priority in the Parietal Lobe

James W. Bisley and Michael E. Goldberg p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 1

The Subplate and Early Cortical Circuits
Patrick O. Kanold and Heiko J. Luhmann p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p23
Fly Motion Vision
Alexander Borst, Juergen Haag, and Dierk F. Reiff p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p49
Molecular Pathways of Frontotemporal Lobar Degeneration
Kristel Sleegers, Marc Cruts, and Christine Van Broeckhoven p p p p p p p p p p p p p p p p p p p p p p p p p p p p p71
Error Correction, Sensory Prediction, and Adaptation
in Motor Control
Reza Shadmehr, Maurice A. Smith, and John W. Krakauer p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p89
How Does Neuroscience Affect Our Conception of Volition?
Adina L. Roskies p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 109
Watching Synaptogenesis in the Adult Brain
Wolfgang Kelsch, Shuyin Sim, and Carlos Lois p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 131
Neurological Channelopathies
Dimitri M. Kullmann p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 151
Emotion, Cognition, and Mental State Representation in Amygdala
and Prefrontal Cortex
C. Daniel Salzman and Stefano Fusi p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 173
Category Learning in the Brain
Carol A. Seger and Earl K. Miller p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 203
Molecular and Cellular Mechanisms of Learning Disabilities:
A Focus on NF1
C. Shilyansky, Y.S. Lee, and A.J. Silva p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 221
Wallerian Degeneration, WldS , and Nmnat
Michael P. Coleman and Marc R. Freeman p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 245
v
NE33-FM ARI 14 May 2010 19:47
Neural Mechanisms for Interacting with a World Full

of Action Choices
Paul Cisek and John F. Kalaska p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 269
The Role of the Human Prefrontal Cortex in Social Cognition
and Moral Judgment
Chad E. Forbes and Jordan Grafman p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 299
Sodium Channels in Normal and Pathological Pain
Sulayman D. Dib-Hajj, Theodore R. Cummins, Joel A. Black,
and Stephen G. Waxman p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 325
Mechanisms of Synapse and Dendrite Maintenance and Their

Disruption in Psychiatric and Neurodegenerative Disorders

Yu-Chih Lin and Anthony J. Koleske p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 349
Connecting Vascular and Nervous System Development: Angiogenesis
and the Blood-Brain Barrier
Stephen J. Tam and Ryan J. Watts p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 379
Motor Neuron Diversity in Development and Disease
Kevin C. Kanning, Artem Kaplan, and Christopher E. Henderson p p p p p p p p p p p p p p p p p p p p p p 409
The Genomic, Biochemical, and Cellular Responses of the Retina in
Inherited Photoreceptor Degenerations and Prospects for the
Treatment of These Disorders
Alexa N. Bramall, Alan F. Wright, Samuel G. Jacobson, and Roderick R. McInnes p p p 441
Genetics and Cell Biology of Building Specific Synaptic Connectivity
Kang Shen and Peter Scheiffele p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 473
Indexes
Cumulative Index of Contributing Authors, Volumes 24–33 p p p p p p p p p p p p p p p p p p p p p p p p p p p 509

Cumulative Index of Chapter Titles, Volumes 24–33 p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p p 513
Errata
An online log of corrections to Annual Review of Neuroscience articles may be found at

http://neuro.annualreviews.org/
vi Contents

Cisek and Kalaska 2010 - Neural Mechanisms For Interacting With A World Full of Action Choices

Uploaded by

Copyright:

Available Formats

Cisek and Kalaska 2010 - Neural Mechanisms For Interacting With A World Full of Action Choices

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cisek and Kalaska 2010 - Neural Mechanisms For Interacting With A World Full of Action Choices

Uploaded by

Copyright:

Available Formats

NE33CH13-Cisek ARI 22 May 2010 19:1

• Top downloaded articles

• Our comprehensive search

Annu. Rev. Neurosci. 2010. 33:269–98 Key Words

movement. Neural correlates of putative

general questions of what these and other

recent data imply for large-scale theories of the

270 Cisek · Kalaska

COGNITIVE NEUROSCIENCE system is formalized in a theoretical framework

tive systems bring salient context-dependent

www.annualreviews.org • Neural Mechanisms for Interactive Behavior 271

272 Cisek · Kalaska

www.annualreviews.org • Neural Mechanisms for Interactive Behavior 273

or release a lever in response to a sequence as input to the cognitive system. However,

274 Cisek · Kalaska

the distinctions among perceptual, cognitive, in many psychological or neurophysiological

www.annualreviews.org • Neural Mechanisms for Interactive Behavior 275

276 Cisek · Kalaska

environment for central knowledge acquisition, affordance competition hypothesis (Cisek

concerned with descriptive representations of before speciﬁcation (movement planning), we

www.annualreviews.org • Neural Mechanisms for Interactive Behavior 277

278 Cisek · Kalaska

large family of computational models of deci- Frontoparietal Specification

(Cisek 2006, Erlhagen & Schöner 2002, Fur-

man & Wang 2008, Tipper et al. 2000), at least

www.annualreviews.org • Neural Mechanisms for Interactive Behavior 279

280 Cisek · Kalaska

www.annualreviews.org • Neural Mechanisms for Interactive Behavior 281

282 Cisek · Kalaska

continuum of tuned cells can even go be- Shadlen et al. 2008).

www.annualreviews.org • Neural Mechanisms for Interactive Behavior 283

284 Cisek · Kalaska

www.annualreviews.org • Neural Mechanisms for Interactive Behavior 285

Activity during Activity during 50 ms and discriminate targets from distracters

that performed a conditional Go/NoGo task

(Figure 4a). Through an elegant experimen-

286 Cisek · Kalaska

NoGo trials shaded regions

A Striate B Striate C Striate time during which

0.5 1.0 1.0 Ledberg et al. (2007).

−1.0 −1.0 −1.0

0.5 0.5 0.5

0.0 0.0 0.0

– 0.5 –0.5 –0.5

−1.0 −1.0 −1.0

www.annualreviews.org • Neural Mechanisms for Interactive Behavior 287

288 Cisek · Kalaska

ied cognition have suggested, following Piaget,

that cognitive abilities may have evolved within

www.annualreviews.org • Neural Mechanisms for Interactive Behavior 289

290 Cisek · Kalaska

www.annualreviews.org • Neural Mechanisms for Interactive Behavior 291

292 Cisek · Kalaska

parieto-occipital cortex encodes objects reachable by the hand. J. Neurosci. 29(14):4381–91

www.annualreviews.org • Neural Mechanisms for Interactive Behavior 293

294 Cisek · Kalaska

Merleau-Ponty M. 1945. Phénoménologie de la Perception. Paris: Gallimard

www.annualreviews.org • Neural Mechanisms for Interactive Behavior 295

296 Cisek · Kalaska