36
The Open Behavioral Science Journal, 2008, 2, 36-52
Open Access
Aftereffects in the Perception of Emotion Following Brief, Masked Adaptor
Faces
Nicola van Rijsbergen*,1,2, Ali Jannati2 and Alessandro Treves2
1
Department of Psychology, Glasgow University, 58 Hillhead St, G12 8QB, UK
2
SISSA, Cognitive Neuroscience Sector, via Beirut 2-4, Trieste, 34014, Italy
Abstract: Adaptation aftereffects are the tendency to perceive an ambiguous target stimulus, which follows an adaptor
stimulus, as different from the adaptor. A duration dependence of face adaptation aftereffects has been demonstrated for
durations of at least 500ms, for identity related judgments. Here we describe the duration dependence of the adaptation aftereffects of very brief (11.7ms-500ms) backwardly masked faces, on both expression and identity category judgments of
ambiguous target faces. We find significant aftereffects at minimum duration 23.5ms for emotional expression, and 47ms
for identity, but these are abolished by backward masking with an inverted face, although these same adaptors can be correctly categorized above chance.
The presence of a short duration adaptation effect in expression might be mediated by rapid transfer of low spatial frequency (LSF) information. We tested this possibility by comparing aftereffects in low pass and high pass filtered ambiguous targets, and found no evidence of independent adaptation of a LSF specific channel.
Keywords: Face aftereffects, emotion, backward masking, adaptation.
INTRODUCTION
People use the visual properties of faces to make vital
social judgments about the identity, emotional state, gender,
and even the trustworthiness of other people. These judgments make use of all levels of immediate visual information
in the face, and also incorporate prior knowledge about both
face categories, and individual faces. Recent experimental
work with faces [1, 2] has found that category judgments can
be biased by recent experience, if it includes prolonged exposure (adaptation) to an example face of a related category.
This bias has been described as the ‘face category adaptation aftereffect’.
How much recent visual experience is needed to produce
significant bias? In the experiments we report here, we examine how the face category adaptation aftereffect depends
on how much exposure to the adaptor there is in immediate
perceptual history, in two different ways. First, we manipulate the exposure to the adaptor face by changing its duration
[3]; previous work has shown a duration dependence only
for adaptation exposures longer than 1s, while here we consider short durations, between 11.7ms and 500ms. Second,
we manipulate the amount of ‘neural exposure’, or the
amount of processing time allowed for the adaptor by using
backward masking, as described below. We also ask whether
judgments about different aspects of faces depend in the
same way on recent experience by comparing the effect of
masking in identity and emotion judgments.
Finally, we consider the question of whether face information at low spatial frequencies (LSF; below about 8 cycles
*Address correspondence to this author at the Department of Psychology,
Glasgow University, 58 Hillhead St, G12 8QB, UK;
E-mail: nicola@psy.gla.ac.uk
1874-2300/08
per image [4]) is more involved in mediating the aftereffect
to emotional expression, than information at higher spatial
frequencies (HSF; above 23 cycles per image), as it has been
hypothesized that LSF information plays an important role in
rapid emotion detection.
PREVIOUS STUDIES OF ADAPTATION AFTEREFFECTS WITH FACES
Visual adaptation aftereffects [5-7] have historically been
demonstrated for simple, local visual features, such as tilt,
and colour [8]. Adaptation to one stimulus (the adaptor) can
change the perceived content of a following target stimulus.
The induced change, i.e. the aftereffect, is consistently away
from the adaptor, whereby subjects are more likely to classify an ambiguous stimulus as belonging to a different category from that of the adaptor. One explanation for these opponent aftereffects is in terms of fatigue of the neural populations encoding the adapted feature, where the exact form of
the adaptation depends on the nature of the neural encoding
[9, 10]. Recently, aftereffects in the perception of natural
face categories, such as identity and emotion, have been reported [1, 2, 11, 12], as part of an emerging body of work
describing aftereffects in global, high level visual features.
Aftereffects have been observed along the dimension of
emotion [2], but only for relatively long exposure to the
adaptor (at least one second [13]). Exposure to a fearful face
(as in Fig. (1), task II) leads to a change in categorization
behaviour of a following, previously ambiguous stimulus
(for example, the morph target in Fig. (1)), which tends to be
classified as belonging to a different category to the adaptor,
i.e. neutral. Webster et al. [2] showed the adaptor before the
target on every trial. Their protocol also included a phase of
passive viewing of the adaptor in advance of classification
2008 Bentham Open
Aftereffects in the Perception of Emotion Following Brief, Masked Adaptor Faces
trials. In experiment 1 we examine the contribution of a
phase of pre-exposure to the magnitude of the aftereffect.
The face expression aftereffect is similar to, and may
share common mechanisms with, the ‘face distortion’ aftereffect in the configural processing of facial identity, where
altering the geometric structure of the face in the adaptor
creates a misperception of the normal face structure of the
target [11]. These face aftereffects have been shown to contain a component that is invariant under transformations in
size [13-15] and 2D orientation [14, 16], and position [17]
and therefore probably involve relatively high level, non
retinopically organized cortical areas. The identity aftereffects obtained in the same fashion using ‘face-antiface’ pairs,
defined relative to a face space norm [1], show similar size
invariance and partial 3D rotation invariance [18, 19].
The strength of the adaptation aftereffect on the identity
dimension has been shown to be duration dependent [3, 20]
for durations longer than 1s. Almost nothing is known about
the duration dependence of the expression aftereffect [20].
The experiments described in this paper show duration dependence when the durations of adaptors are short, i.e. less
than 500ms. Aftereffects from short duration adaptors are of
particular interest because it has been suggested that they are
more associated with adaptation of higher level areas [2123]. In particular, a recent paper [24] found that it is the position invariant component (at least on the gender dimension)
that is induced with short (500ms) duration adaptors, rather
than the position specific component that can be seen with
longer adaptors.
Rather than focusing on which visual areas mediate the
adaptation aftereffect, in our central experiment 2 we employ
backward masking to try to examine the aftereffects of different temporal stages of processing of the adaptor. We refer
to these short duration, masked adaptors as (negative)
primes.
BACKWARDLY MASKED FACES
The perception of a visual stimulus, such as a face, can
be impaired or interfered with by the presentation of a second following stimulus, such as another face. This form of
interference is known as backward masking, and can result
in the perceiver being unable to accurately detect, and report,
the content of the first stimulus. The effectiveness of visual
masking is known to depend on the Stimulus Onset Asynchrony (SOA) [25-28], i.e. the interval between the first (in
this paper, the adaptor) and second stimulus (in this paper,
the mask). In general, the shorter the interval between the
stimulus onset, and the mask onset, the more effective is the
masking. However, the mask may interfere with more than
one stage of processing. At very short (less than 10ms) intervals, masking appears to block processing in the visual system completely [25]. At longer intervals (20-40ms), imaging
and electrophysiological recording studies in humans suggests that early feedforward (retinotopic) stages of visual
processing are largely preserved [29-31]. Comparably, in
rhesus monkeys, the first few spikes (20-30ms) of neural
response in inferior temporal cortex are reported not to be
significantly affected by the mask at an SOA of 20ms [26,
27, 32] while the later sustained response is suppressed. This
may be due to the mask interrupting feedforward processing
[29] or possibly disrupting a late phase of neural activity
The Open Behavioral Science Journal, 2008, Volume 2
37
driven by re-entrant connections [30]. Face processing continues to be affected by masking up to SOAs of at least
130ms [33].
The SOA is not, however, the only factor that influences
masking efficacy. The content of both the mask and first
stimulus is also influential. We exploit the results of [60] to
adjust the content of the masking stimuli to create two masks
of different strengths. Behaviourally salient stimuli, such as
fearful or angry faces [34, 35] or negatively conditioned
faces [36, 37] are still processed even when participants report no subjective awareness of the masked stimulus. We
therefore chose to measure the efficacy of the two masking
stimuli in reducing processing of the adaptors by our subjects’ objective performance (i.e. not whether they think they
saw the adaptor, but whether they can guess its content) on a
forced choice task.
A number of recent studies [38-40] have reported that an
early, event related potential component of face processing,
between 90ms post stimulus [39] to 130ms post stimulus
[41], is sensitive to modulation by the emotional content of
face stimuli although the source of this modulation is still
unknown. Face processing therefore perhaps starts relatively
early, and continues to be vulnerable to interruption by a
masking stimulus for a prolonged period of time, and it can
be hypothesized [32] that the information build up about the
stimulus is cumulative. Using the backward masking technique, we investigate in experiment 2 how quickly information about the emotional category of a face can be extracted,
and how much time is needed to produce a measurable category aftereffect, on the following (third) target stimulus.
SPEED OF PROCESSING OF LOW AND HIGH SPATIAL FREQUENCY INFORMATION
The evidence that the emotional content of faces can be
processed very rapidly [39, 42], and in the absence of awareness, has led to the speculation that emotional face processing is mediated by an independent fast pathway (driven by
magnocellular inputs). In addition to a fast cortical route [40,
43, 44], several authors [35, 45, 46] have proposed the existence of a sub-cortical fast forward pathway sensitive to
emotional content relayed via the colliculus and pulvinar
directly to the amygdala. Evidence from evoked potentials
suggests that rapid emotional processing via either route may
be preferentially driven by low spatial frequency information
[40, 43].
If a faster pathway exists for processing emotional expression information in faces, information about the emotional content of the face might be available to the subject
earlier in time than information about identity, because it
‘arrived faster’ through a separate channel. Alternatively,
making decisions about emotion might simply require less
information and a shorter processing time than making decisions about identity, just because emotion discrimination is
easier. These two possibilities are difficult to distinguish in
behavioral experiments. In experiment 3 we examine
whether the information about emotion is carried through a
low spatial frequency channel.
OVERALL DESIGN
In order to measure aftereffects, we created a set of ambiguous morph stimuli (described more fully in the Appen-
38 The Open Behavioral Science Journal, 2008, Volume 2
dix) from the Ekman and Matsumoto faces [47]. For the
emotional expression aftereffects, we created 29 morphs
from a neutral and an emotional example of the same face,
whereas for the identity morphs we created 29 morphs from
two neutral identities of the same gender. We took the first
and last morphs as representing 100% emotion and 0% emotion, and used them as the adaptors. The 27 morphs lying
between the two adaptors were used as target stimuli. For
the emotion dimension, the magnitude of the aftereffect was
estimated from the difference between the amount of emotion perceived in target faces after ‘emotional’ adaptation
and the amount of emotion perceived in target faces after
‘neutral’ adaptation. The ‘amount of emotion perceived’ is
measured as the proportion of ‘emotional’ responses at each
point of a morphing continuum, spanning the 27 morph
points, each point 3.44% apart, between the ‘neutral’ and an
‘emotional’ expression with the same identity. To illustrate
the aftereffect we plot, for each experiment, the average proportion of emotion reported in the target against the percentage of emotion in the target along the morphing dimension.
For the identity dimension, the aftereffect was measured
as the difference between the proportion of responses of
identity A after adaptation with identity B, and the proportion of responses of identity A after adaptation with identity
A.
Our first experiment was designed to test whether adaptation aftereffects for emotional expression can be obtained
with a very brief presentation of the adaptor face, subsequently masked by a blurred face of different identity (which
acts as a mask of weak intensity). As we found that the aftereffect persists after a 23.5ms prime, we then expanded the
design to include duration as a varying parameter. We also
included a stronger masking condition – when the mask is an
inverted face. Experiment 2 then measures the aftereffects on
the emotional continuum due to short duration adaptors,
ranging from 11.7ms to 300ms. A parallel condition measures aftereffects on identity continua in exactly the same
conditions, and the same range. These parallel experiments
enable us to compare the duration dependence of identity and
emotional expression aftereffects.
As we find that the aftereffect to the 23.5ms prime is not
present on the identity dimension, one possible explanation
is a faster extraction of emotional information from the
masked adaptor. In experiment 3 we aimed to test the hypothesis that if there is a fast pathway for emotional processing, it is more sensitive to information carried in low spatial
frequency [46]. If low spatial frequency information is critical for rapid emotion detection, the channel, or representation of LSF information may also adapt earlier than the representation of high spatial frequency (HSF) information.
Our method was to use a broad spatial frequency (BSF), i.e.
intact face as the adaptor stimulus, for a short duration, and
then measure the aftereffects induced in the LSF and HSF
’components’ of the ambiguous target, within the same
subject.
van Rijsbergen et al.
ware, and data analyzed in MATLAB (7.3). Subjects were
seated approximately 50 cm from the screen, although their
head position was not constrained. All stimuli were presented 12x12cm wide, of which a 2cm surround was a dark
border superimposed over the hair contour (Fig. 1) and thus
the images extended about 14° of their visual fields. Of this,
the actual face features occupied 7°.
Stimulus Duration. Exact timing of stimulus presentation was obtained with a photodiode, connected to an oscilloscope. Timing logs were also recorded during the experiment, and any trials with unusual timing errors were eliminated (about 1 or 2 per subject).
Morphed
Stimuli.
Gray-scale
equi-luminescent
256x241-pixel stimuli were created in X-morph, from 12
face identities from the JACFEE/JACNEU Eckman series
[47]. Xmorph interpolates smoothly between a grid of points,
which can be fitted to the features of the face. We used a grid
of approximately 100 points per face. 29 morphs were created between the neutral and emotional examples of each
face. Stimuli at points 1 and 29 of the 30-point scale were
used as the adaptors (the neutral adaptor and the emotional
adaptor), while the remaining 27 served as target stimuli.
Note that in all graphs points 1 (not 0) and 30 were conventionally set as containing 0% and 100% emotion, respectively, therefore the 27 target stimuli are positioned on the
morphing continuum as conveying from 3.4% to 93% of the
fully expressed emotion. We also performed a pilot experiment examining the emotional intensity of the morphing
continua for each stimuli set (two adaptors, one neutral and
one emotional, and the morphing continuum between them).
This experiment is described in more detail in the supplementary material.
For experiment 2, we created a set of identity A to identity B morphs in a similar fashion, using examples of neutral
expressions.
Masks. The weak mask was a quantized image made in
Matlab (by setting blocks of pixels (14x14) to their average
value) from a face that did not otherwise appear in the experiment, which had the same average luminance. The strong
mask was made from the same face image as the weak mask
in full detail, but inverted.
The effect of the masks at different durations was examined in a pilot experiment, which asked subjects to classify
the emotion on the face of a prime of variable duration, followed either by the quantized, or the inverted mask. Our
quantized mask is similar to a quantized masking condition
employed by Bachmann [48], which is shown to have a
strong masking effect at short durations, similar to noise
masking, and a weaker masking effect at long SOAs. The
inverted face mask was found in the pilot to reduce adaptor
discrimination to chance at short (less than 40ms) durations,
a more striking effect than previous reports have suggested
[49], and to produce a stronger effect than noise at intermediate SOAS.
We included a reference table showing all experiments
and conditions (Table 1).
ADAPTATION BY PRIOR EXPOSURE, AND BY
SHORT AND LONG PRIMES
Experimental Setup. All experiments were run on a
Pentium 4 computer running Windows 2000, with a CRT
monitor, set to a refresh rate of 85 Hz using E-prime soft-
Experiment 1
In this first experiment, we tested whether a reliable aftereffect could be obtained with a short duration adaptor. We
Aftereffects in the Perception of Emotion Following Brief, Masked Adaptor Faces
call this adaptation by prime. Secondly, we investigated
whether prior exposure to the same stimulus in another context would lead to measurably enhanced aftereffects. We call
this adaptation by exposure. As an auxiliary goal, we wished
to determine the best place on the emotion-neutral continuum to reliably measure aftereffects for our stimulus set.
Experimental Design and Methods
The experiment contained five different conditions.
These five conditions are represented in Table 1, but in
summary are (a) exposure only, (b) short prime and exposure, (c) short prime only, (d) long prime and exposure, and
(e) long prime only. Our dependent measure was the magnitude of the aftereffect obtained under each condition. Statistical significance throughout the paper was considered to be
p = 0.05, with non-sphericity corrections where appropriate.
Tasks. There were two tasks, a repetition detection task
(inducing adaptation by exposure), and a classification task
(with and without adaptation by prime), probing the aftereffect. These tasks were combined to create the five experimental conditions.
During adaptation by exposure subjects were simultaneously adapted to two randomly sequenced adaptors, one fearful (e.g. S1, “fearful Mary”), one neutral (e.g. S2, “neutral
John”) over 3 minutes, during task (I), represented in the top
half of Fig. (1). The subject had to press the mouse every
time the same identity was repeated. The stimuli were on
screen for 800 ms each, and were separated by a mask, a
Table 1.
1
2
3
Mask
Type
Target
(Filter)
Prior
Exposure
a
None
W
None
Yes (one)
b
23.5
W
c
23.5
W
No
d
500
W
Yes (one)
e
500
W
a
11.7
W/S
b
23.5
W/S
fixation point, and a variable delay (150 or 650ms). The total
viewing time for each stimulus within the exposure period
was 36s. These two stimuli then served as the primes during
the classification task.
In the classification task, inducing adaptation by prime,
represented as task (II) in the bottom half of Fig. (1), subjects signaled (L/R mouse click) whether each target face
they saw on the screen was neutral or emotional. The task
was divided into blocks, one containing examples of identity
S1, the second containing identity S2. For the prime conditions, an adaptor preceded each target face. This adaptor was
the emotional extreme for S1 and the neutral extreme for S2.
One was shown in a short prime condition, of 23.5ms duration, followed by a mask of 50ms duration, and one was
shown in the long adaptor condition (500ms), followed by
the mask (50 ms), then after 250 ms the target appeared for
800ms, then a response screen (1000ms). Subjects were instructed to ignore the first ‘distracter’ face stimulus (the
adaptor) and the mask, and respond to the third ‘target’ face.
For the exposure only condition the task was identical except
that the adaptors were omitted, and only the mask was
shown before the target.
The two tasks described above were combined to form 5
experimental conditions in the following manner. Two Exposure & Primes conditions, (b) and (d), included the initial
repetition detection task, with two adaptor stimuli, S1 and
S2, followed by categorization trials with a block of long
(S1) primes, and a block of short primes (S2); Exposure
Yes (one)
Exposure Task
Exposure
Measure
Aftereffect Task
Aftereffect Measure
Repetition detection
None
Categorization of
morph target as emotional face or neutral
face
Difference between mean
emotion responses, after
emotion adaptor and
neutral adaptor
Categorization of
masked adaptor as
emotional face or
neutral face
% Correct
Categorization of
morph target as emotional face or neutral
face.
Difference between mean
emotion responses, after
emotion adaptor and after
neutral adaptor
No
None
To both
adaptors
c
47
W/S
d
106
W/S
OR:
OR:
OR:
e
306
W/S
Categorization of
masked adaptor as
identity A face or
identity B face
Categorization of
morph target as identity
A face or identity B
face
Difference between mean
identity A responses, after
identity A adaptor and
after identity B adaptor
a
1000
(BSF)
W
Categorization of
HSF and LSF
Morphs
Categorization of
morph target as emotional face or neutral
face
Difference between mean
emotion responses, after
emotion adaptor and
neutral adaptor
Categorization of
morph target as emotional face or neutral
face
Difference in HSF faces
and difference in LSF
faces after emotion adaptor and neutral
b
a
b
W
W
c
3
39
Summary of Experimental Conditions
Adaptor
(Msec)
Exp.
The Open Behavioral Science Journal, 2008, Volume 2
23.5
(BSF)
11.7
(BSF)
W
W
None
(BSF)
HSF
Yes
LSF
HSf/
LSF
HSF/
LSF
Yes
Categorization of
HSF and LSF
morphs and
masked BSF
adaptors
as neutral or emotional
Adaptor: The duration of the adaptor is shown. Mask type: W denotes the weaker mask, the upright quantized face, and S the inverted face that acts as a stronger mask. Both masks
were derived from faces that did not appear in the experiment. W/S indicates that the experiment included trials with both mask types. Target: The types of filtering of the target in
each experiment. Exposure indicates whether subjects were pre-exposed to one or both adaptors prior to the trials measuring the aftereffect. Remaining columns describe the tasks
and measures used. Parameters that are not included were constant for versions of the same experiment.
40 The Open Behavioral Science Journal, 2008, Volume 2
van Rijsbergen et al.
Fig. (1). Experiment 1. During the preliminary stimulus repetition task, (I), top, subjects were simultaneously adapted, by exposure, to two
randomly alternating adaptors, one emotional (e.g. S1,”fearful Mary”) and one neutral (e.g. S2, “neutral John”). In the classification task,
(II), bottom, subjects signaled (L/R mouse click) whether each target face they saw on the screen was neutral or emotional. In the priming
conditions, the target was preceded by a masked prime of duration, in conditions 1b and 1d, 23.5ms (short), or 1c and 1e, 500ms (long), In
the Exposure only condition (1a), the target was preceded only by the mask.
only, (a), included the repetition detection task but the categorization task contained no primes; and Prime only, (c) and
(e), included separate blocks of long and of short primes,
within the categorization task but omitted the repetition task.
The order in which conditions were presented was counterbalanced across subjects. S1 and S2 were fully counterbalanced across subjects. Table 1 also contains a summary of
the same information.
Procedure. Twenty-seven naïve subjects contributed to
two experimental sessions, where stimuli (identities and
emotional dimensions) were pseudo randomly assigned to
the two 25min sessions so that each subject contributed
equally to the emotional adapted and the neutral adapted
data, and categorized each identity (N=12) once only. Each
subject performed three blocks of two categorization sets per
session, and was required to rest for 2-3 minutes between
blocks. Each categorization set consisted of 40 morphs from
a particular image spectrum (morphs 3.4%-24%, and 72%93% were shown once, and the intermediate (27-69%) ambiguous morphs twice). The valence (i.e. neutral or emotional) of the adaptor and exposure stimulus was counterbalanced across subjects, and a neutral adapted set always
followed an emotional-adapted set within each block.
For each subject, we measured the magnitude of the aftereffect at the central morphs (41%-62%), by subtracting the
mean proportion of emotion responses after the neutral adap-
Aftereffects in the Perception of Emotion Following Brief, Masked Adaptor Faces
tor, from the mean proportion of emotional responses after
the emotional adaptor. We also fitted cumulative normal
functions to each subject’s data, and found the threshold,
which is the point of inflection of the decision function, in
the cumulative normal. The threshold represents the hypothetical point where a subject is equally likely to classify the
target as emotional or neutral. The parametric method did
not fit all subjects’ data equally well, so we also employed
non-parametric methods, and defined a nonparametric
threshold (described in the supplementary material).
The Open Behavioral Science Journal, 2008, Volume 2
41
We also analyzed reaction times, as described in the supplementary material.
RESULTS
The main result of these experiments is that a reliable
aftereffect could be elicited with a 23.5ms adaptor, with and
without prior exposure to the same stimulus.
Analysis of Aftereffects. Fig. (2) shows the change in
the average of all the subjects’ proportion of ‘emotion’ classification responses, for each possible emotion level of the
Fig. (2). Size of the adaptation aftereffects. Average proportion of ‘emotion’ classification responses across subjects, for each emotion level
of the target (3.4% is the most neutral target, and 93% is the most emotional), shown after emotional adaptation (red), and neutral adaptation
(blue). Each condition is shown separately, with the exposure conditions on the right, and the prime only conditions on the left. Individual
Conditions 1a-1e: The 5 conditions tested differ by the absence (left column) or presence (right column) of a prior exposure adaptation
phase; the absence of a priming face (top row); the presence of a 23.5 ms prime (middle row) or of a 500 ms prime (bottom row). Error bars
+/- are S.E.M. Inset in each graph is the histogram of average threshold changes for the 12 stimuli sets, estimated with the parametric
method.
42 The Open Behavioral Science Journal, 2008, Volume 2
van Rijsbergen et al.
target, after emotional adaptation (red) and neutral adaptation (blue). Each condition is shown separately, with the
exposure conditions on the right, and the prime only conditions on the left.
the exposure session and the main task (including all the
intervening stimuli). However, it does not add to the short
term effect of the prime, even if the effect of the prime is
small.
A 2-way ANOVA (5 conditions, and adapting end-point)
of the frequency of responding emotional to the seven most
ambiguous morphs (the central morph 41% - 62%) found an
effect of adaptation direction, F{1,200}=123, p<0.0001,
MSE=4.60, and a significant interaction between adaptation
direction and condition, F{3,200}=7.41, p=0.0001,
MSE=0.28. Significant aftereffects appeared in all five conditions. Neutral adapted – Emotional adapted contrasts were
significant for Long Prime, F{1,34}=34.1, p<0.01,
MSE=1.52; Short Prime, F{1,34}=7.12, p<0.05, MSE=0.26;
Exposure & Long Prime, F{1,30}=44, p<0.0001, MSE=1.71;
Exposure & Short Prime, F{1,34}=11.15, p=0.002,
MSE=0.39; Exposure only, F{1,37}=4.7, p<0.04, MSE=0.12.
Subject debriefing afterwards suggested that subjects
paid little attention to the masked stimuli, and were unaware
of the content of the short prime. However, the subjects in an
independent pilot experiment performed significantly above
chance across trials (mean 69%+ STD +/-11%), when single
stimuli were presented for 23.5 ms and masked by the quantized (weak) mask (see supplementary material).
The effect of condition was essentially a difference between the two long adaptor conditions (d, e), and the other
three. There were no statistical differences between the two
500ms prime conditions, nor between the 23.5 ms prime
conditions, or between the latter and exposure only. This
suggests that the exposure period had no additional effect
above that of the prime. Long and short primes produced
different magnitudes of aftereffect, determined by comparison of the difference between emotional adapted and neutral
adapted in each condition. We used the t-test to examine the
difference in the magnitude of the adaptation aftereffects
between pairs of conditions. The differences between long
and short primes were significant: Exposure & Long Prime
versus Exposure & Short Prime: t{32}=2.92, p<0.03; Long
Prime versus Short Prime: t{34}=2.63, p=0.05, Bonferroni
corrected for 5 tests.
Distributions of parametric threshold values are shown in
the insets of Fig. (2). Each inset shows a histogram of the
average (across subjects) of the threshold shifts for the 12
stimuli. The threshold shifts of individual stimuli were not
significantly correlated across conditions.
DISCUSSION
We established that a measurable aftereffect can be obtained with the presentation of a 23.5 ms prime, masked with
a weak mask, and that the magnitude of this aftereffect appears to depend on the duration of the preceding prime, and
not on the presence or absence of pre-exposure. This is different from results reported by Hsu and Young [13] who did
not obtain a reliable aftereffect in emotional expression with
500ms adaptors, but who used a very long (1 second) interval between adaptor and target.
The large aftereffects obtained with long primes presumably reflect a variety of conscious processes, involving
the careful examination of the adaptor and the retention of
relevant information in short term memory. Our data also
suggest that in the absence of an immediately preceding
adaptor of sufficient duration to allow conscious comparison
with the target face, the aftereffects induced by prior exposure and by short adaptors are all of similar magnitude. In
particular, adaptation by prior exposure does induce a measurable aftereffect, which appears to be stimulus-specific and
to survive the time interval (approximately 2min) between
STRONGER MASKING, AND IDENTITY VS EMOTION
Experiment 2
A pilot experiment had shown that masking with an inverted face, i.e. a stronger mask, at the same adaptor duration
(23.5ms), reduced discrimination of the adaptor content to
chance level. With this result in mind, we designed the next
experiment to examine the effects of stronger backward
masking’ on short duration stimuli.
The motivation for experiment 2 was two-fold. Firstly,
we wanted to describe the dependence of the aftereffect on
the duration of the adaptor on both identity and emotion adaptation. Secondly, we wanted to examine the effect of
masking of the adaptor on the aftereffect, by using a mask
that reduced subject’s ability to explicitly categorize the
adaptor content.
We hypothesized that masking may disrupt identity adaptation at durations where emotional adaptation is still preserved, as information about the emotional content of the
adaptor may be processed more rapidly than identity information. Thus, if the mask stops processing at the right point,
we should be able to observe adaptation to the emotional
category but not to identity. To test the latter hypothesis, we
chose to compare adaptation obtained with the mask used in
experiment 1 (our weak masking condition) with a mask
consisting of an inverted face of the same average luminance
(a strong masking condition). We also measured objective
categorization of the masked adaptor at each duration in a
separate block.
In order to describe the duration dependence of identity
adaptation and emotional adaptation, we conducted two parallel experiments with the same procedure, an emotion condition inducing emotional adaptation, and an identity condition, inducing identity adaptation. The experimental procedure used the same set of durations, and two different masks
for both conditions. To do this we created a second set of
morph stimuli spanning four continua between four pairs of
adaptor identities (A and B), where one of the identities was
the neutral adaptor from the emotion experiment. Obviously
identity A / identity B morphing continua do not form a
‘natural’ spectrum, and subjects require some prior experience or training to be able to successfully identify unknown
faces. We therefore introduced a training phase into the experimental design, for both the emotion/neutral and the identity/identity experiments. During this phase, we measured
the subjects’ accuracy at classifying examples of the adaptors at different durations, and under different masking conditions. This enabled us to examine the relationship between
Aftereffects in the Perception of Emotion Following Brief, Masked Adaptor Faces
The Open Behavioral Science Journal, 2008, Volume 2
43
each individual subject’s ability to objectively classify adaptors, and their bias during the aftereffect trials.
scores and adaptation magnitudes for each duration/mask
combination, both on average, and across the population.
Experimental Design and Methods
RESULTS
The two experimental factors, adaptor duration, and mask
- strong (inverted) versus weak (quantized) - were varied in
exactly the same manner in the two experiments. The two
shortest presentation times were 11.7ms (one screen refresh)
and 23.5ms (2 screen refreshes). Additional durations of
47ms, 106ms, and 305ms (4, 9 and 26 refreshes) were also
used.
The left panels of Fig. (3) show how accuracy of the objective classification of the adaptor increases with duration in
both identity and emotion experiments. The right hand panels show how the mean magnitude of aftereffects in the two
conditions also increases. There is a clear difference between
the weakly masked (dashed lines) and the strongly masked
conditions in both tasks of both conditions. The main differences between the two experiments are summarized in Fig.
(4), where there is a clear aftereffect at 23.5ms in the emotion experiment for weak masking that is not present in the
identity experiment and an absence of an aftereffect in the
strong masking condition for identity, at 305ms.
Subjects. 19 students from the University of Trieste participated in the emotion/neutral adaptation condition, and 20
students participated in the identity/identity condition.
Procedure. Each subject performed 4 blocks of trials,
order counterbalanced across subjects, where each block
included a different pair of adaptors, and a set of morphs
taken from the morphing continuum. In the identity A to
identity B continuum, the two adaptor faces were of the
same gender and neutral expression. In the emotional to neutral continua, the two faces were of the same identity, one
was always neutral, and one was fearful or angry.
Each block comprised a training phase and an adaptation
phase. In the identity experiment the training phase consisted
of 20 trials as follows: the subject was presented with a training screen for 3s containing a pair of identities, with associated names e.g., ‘Anna’ on the left of the screen, and ‘Lucy’
on the right side of the screen. The screen also assigned Lucy
or Anna to two letters on the keyboard, a right (n) or left key
press (x), which corresponded to their screen position, counterbalanced across subjects. Two seconds afterwards they
were presented with a masked stimulus at one of the five
durations, and asked to guess the identity of the stimulus
before the mask, and to respond with the corresponding x/n
key choice. The training screen with the key assignments
preceded every trial until they had seen two examples of
each mask/duration combination for that identity pair.
The subject then performed 80 adaptation trials in random order, 2 adaptors (e.g. Lucy or Anna) at five durations,
and with 2 masks. Here they were asked to classify the target
following the masked stimulus. Targets were shown for
300ms, and were morphs from the central section of the continuum, levels (41%, 48%, 54% or 62%).
The emotion condition was designed and run in exactly
the same manner, except that the training screen in each
block contained examples of a neutral expression, and a fearful or angry expression of the same identity, presented side
by side, and labeled ‘fear’ and ‘neutral’, or ‘angry’ and ‘neutral’ with associated key responses.
For the training data, we calculated the percent correct at
each mask/duration combination. We also measured each
subject’s average reaction time for each mask and duration
combination. For the aftereffects, each subject’s average
proportion of response ‘emotion’ and average response
’identity A’ was computed for each adaptor, mask and duration combination. We took the magnitude of the difference
between the two adaptation directions, at each mask and duration, for each subject, and then averaged across subjects.
We also computed correlations between percent correct
Identity versus Emotion. Subjects were significantly
more accurate on the emotion classification task, overall,
than on the identity task. Separate ANOVAs, for each mask
type, with two factors, experiment (identity or emotion) and
duration, showed an effect of experiment on the classification task, in both the quantized mask F{1,37}=17.38,
p<0.001, MSE=0.48 and in the inverted mask F{1,37}=5.02,
p<0.03, MSE=0.18, as well as strong effects of duration,
F{4,34}=19.44, p<0.005 and F{4,34}=24.12, p<0.005.
However, comparison of the magnitudes of aftereffects between experiments showed no overall difference between
identity and emotion, although there was still a strong effect
of duration, F{4,34}=11.8, p<0.01 and F{4,34}=4.13,
p<0.05. Comparison of aftereffect magnitudes in the two
experiments is hard to interpret, due to potential scaling issues. To capture the effect of duration, therefore, we divided
the data into the two experimental conditions and analyzed
them separately with within-subject ANOVAs.
Emotion. Within subjects, however, there was a striking
difference between the two mask types in both tasks. In the
emotion experiment, a within subjects ANOVA, on the classification task, found effects of mask type, F{1,18}=23.02,
p<0.0005, MSE=0.76 and duration F{4,72}=17.17, p<0.005,
MSE=0.57. The same analysis on the adaptation task found
an effect of duration, F{4,72}=12.75, p<0.005, MSE=0.34,
and mask type F{1,18}=10.81, p<0.005, MSE=0.44.
In the classification task (Fig. 3, bottom left) subjects
were on average more accurate for weak than strong masking. Post hoc paired t-tests found a reliable difference between the two masking conditions at 23.5ms, p<0.01, and
otherwise the difference was approximately constant, but not
significant at individual durations.
The aftereffect (Fig. 3, bottom right), was also greater for
the weak masking condition than the strong masking condition, but the effect of duration much less on the strong masking condition. In order to understand the effect of durations
on the aftereffect in each masking condition, we conducted
four paired t-tests comparing successive points for each
masking condition, looking for the largest transition between
successive durations. We also compared the largest increases
with chance (zero bias). Adaptation effects for the weak
mask show the largest increase when we compare 11.7ms
with 23.5ms (p<0.05, Bonferroni-corrected for 5 tests), and
the aftereffect at 23.5ms is also different from chance (one
44 The Open Behavioral Science Journal, 2008, Volume 2
van Rijsbergen et al.
Fig. (3). Duration dependence of prime identification, and of the resulting aftereffect. The results of the emotion condition are shown in the
lower graphs, and the results of the identity condition in the upper graphs. Average percent correct adaptor classification (left) and average
aftereffect magnitude (right) plotted against adaptor duration on a log scale. In the identity condition (top) the aftereffect magnitude is the
difference between (the fraction of) identity A responses, after the A adaptor, and identity A responses after the B adaptor; while for the emotion condition (bottom) it is the average of the difference, for each subject, between emotion responses after the emotional adaptor, and after
the neutral adaptor. In each experiment, the weak masking condition is shown as crosses and the strong masking condition as circles. Error
bars are confidence intervals.
tailed t-test, p=0.05, Bonferroni-corrected) and there is a trend
towards a change between 106ms and 305ms, that does not
meet the correction level (p=0.02, uncorrected). For the strong
face mask, the adaptation aftereffect becomes significantly
different from chance at 305ms (p<0.002, Bonferronicorrected for 5 tests), but none of the transitions are significant. In particular, the largest transition from 106ms to 305ms
is not significant (p=0.026, uncorrected), suggesting that the
aftereffect is already there at 100ms in some subjects.
Identity. In the identity condition, the effect of duration
on accuracy in the classification task was larger than for the
same task in the emotion experiment, F{4,76}=21, p<0.001,
MSE=0.64 and the effect of mask type smaller F{1,19}=9,
p<0.003, MSE=0.27, with a trend towards an interaction,
F{4,76}=1.91, p=0.1. In the adaptation task, the effects were
similar to emotion for mask type F{1,19}=8.6, p<0.001,
MSE=0.45, and smaller for duration F{4,76}=5, p<0.004,
MSE=0.26.
The classification data (Fig. 3, top left), shows that subjects are more accurate in the weak masking condition than
the strong masking condition, but at 305ms the difference
disappears. Pair-wise comparisons between the two masking
conditions at matched durations, established that, on the
classification task, subject accuracy in the two mask conditions differs reliably only at 23.5ms, where it is above
chance with the weak mask, t=2.81, p=0.01 and at chance
with the strong mask. Percent correct data in the strong mask
condition is above chance, t=2.17, p=0.03 at 47ms.
Aftereffects in the Perception of Emotion Following Brief, Masked Adaptor Faces
The Open Behavioral Science Journal, 2008, Volume 2
45
Fig. (4). Summary of experiment 2. The average percent correct on the classification task (left) is shown in both masking conditions at
23.5ms, and at 305ms, for both experiments. The strong and weak masking conditions are significantly different from each other in both the
identity and emotion experiments at 23.5ms (top left) but not at 305ms (bottom left). On the top right, the average aftereffect is significantly
different from chance at the bonferroni corrected level 0.05, at 23.5 ms only in the weak masking condition in the emotion experiment. On
the bottom right, at 305ms, in the emotion experiment, both masking conditions show aftereffects different from chance, and are not different
from each other. In the identity experiment, only the aftereffect in the weak masking condition is significantly different from chance. It is
also different from the strong masking condition. * indicates p<0.05, ** indicates p<0.005, corrected for five tests.
In the adaptation task (Fig. 3, top right), we found no
significant pairwise transitions. A reliable aftereffect appears
at 47ms, for the weak mask, and reaches the Bonferronicorrected level of significance, p=0.05 (compared to chance,
one-tailed); whereas the aftereffect, in the strong masking
condition at 305ms, only reaches a significance level of
p=0.05, uncorrected (also compared to chance, one-tailed).
This pattern of results again suggested that it was a subgroup
of subjects who were driving the aftereffect in the strong
masking condition, and that this group might already show
aftereffects at 106ms (even though the group is still at
chance, as can be seen in Fig. (3)). This proved to be the
case, as aftereffects were strongly correlated within subjects,
correlation coefficient r=0.48, p<0.03 at these durations.
Reaction Times. We analyzed the reaction times of subjects’ responses on both tasks. Only the classification task
showed any effects of duration, mask, or experiment. Subjects were faster on the emotion classification task, than
identity: rank sum test, p=0.03. This was due to a difference
between the two full masking conditions (p=0.007), as the
median RT’s for the weak masking conditions were not significantly different, p=0.43. Again, we also analyzed the two
experiments separately. Overall, subjects were slower to
respond to the strongly masked adaptors, in both experiments.
Further analysis, i.e. an ANOVA on reaction times for
the emotion experiment for random effects, with duration
and mask as factors (thus neglecting the non-normality),
showed that there was an effect of duration F{1,18}=11.5,
p<0.03, and mask F{4,72}=12.65, p<<0.005. Identical
analysis of the Identity experiment found larger effects of
masking F{1,19}=68.7, p<<0.0005, and a smaller effect of
46 The Open Behavioral Science Journal, 2008, Volume 2
duration F{4,76}=9.58, p<0.005. In both experiments, subjects took longer to respond to the shortest stimuli, 11.7 and
23.5ms, in both conditions. Two Friedman’s ANOVAs confirmed these effects. Overall the slowest reaction times were
for strongly masked identity trials.
Comparison of the Two Tasks. Finally, we were interested in whether performance on the two tasks was closely
related, both at an individual, and at a group level.
Our hypothesis was that aftereffect would depend on
duration. Using ANCOVA, we found that the aftereffect
magnitude depended on duration F{1,12}=53, p<.001 once
group means F{3,12}=6.17 were taken into account, with a
borderline interaction term F{3,12}=3.01, p=0.07. The accuracy results also depended on duration, F{1,17}=25,
p=0.002, best fit by the same relationship for all the groups
F{3,13}=3.35, p<0.05. Performance on both tasks clearly
depends on a common variable, duration. We were also interested in the relationship between performance on the classification task and the aftereffect. An ANCOVA using accuracy as the covariate, found that accuracy was a predictor of
aftereffect magnitude F{1,18}=17, p=0.006, and group differences were no longer significant.
The relationship between accuracy on the classification
task and aftereffect magnitude only held at the level of the
group means. Analysis of individual subject data found significant correlations in individual performance between these
two tasks only in one of the four groups, the identity experiment, with strong masking condition, at the duration of 106
ms, r=-0.45, p=0.06, and the duration of 305 ms, r=-0.42,
p=0.04. The same sub group of subjects that showed high
accuracy on the classification task, also showed evidence of
aftereffects (i.e., negative bias) at both durations, even
though the group on average showed no aftereffects, as
shown in the bottom right of Fig. (4).
DISCUSSION
In summary, the data suggest that the magnitude of the
adaptation effect for both identity and emotion depends on
duration in much the same manner. However, information
about emotion category is available at shorter durations.
Emotional category decisions might not, however, be made
faster, as reaction times are not significantly different across
the identity and emotion experiments for the partial masking
condition. In the strong masking condition of the identity
experiment, decisions are made slower, and are less accurate
than in the strongly masked condition of the emotion task.
Subjects also took longer to respond to the short duration
stimuli, and longer to process the fully masked stimuli. This
is consistent with the idea that subjects are using longer
processing times to attempt to compensate for reduced visibility of stimuli, whether reduced by short duration times, or
severe masking [25].
In addition, strong masking interferes with the aftereffect
in both identity and emotion, almost completely abolishing
the aftereffect in identity. The aftereffect in the emotion task
appears at a shorter duration (23.5ms), in the partially
masked condition. In the identity condition, at that duration,
there is not even a trend. The observed relationship between
accuracy and aftereffect suggests that ‘decoding’ the neural
response plays a substantial role in determining aftereffect
van Rijsbergen et al.
magnitude. It might have been the case that aftereffect
strength was most closely related to the duration of the adaptor, indicating that the aftereffect was driven by adaptation to
an early, arguably feedforward phase, of stimulus processing. Instead, we find that full masking interferes with the
aftereffect, abolishing it completely in the identity experiment, even up to a duration time of 300ms, as illustrated by
Fig. (4, bottom right) even though the subjects’ ability to
categorize the adaptor is equivalent to the partial masking
case, (shown in Fig. (4), bottom left).
Overall, our results on the classification support the idea
that information about emotion is in part extracted prior to
information about individual identity. We also find that
strong masking disrupts the emotional expression aftereffect.
We therefore suggest that emotional classification is perhaps
possible on the basis of a stage of stimulus processing that is
preserved in backward masking (perhaps the stages < 110ms
after stimulus onset [30, 31]). In contrast, the aftereffect may
result from adaptation of a later neural process, which is thus
vulnerable to interference from a masking stimulus. The content of the mask may also have played an important role; the
choice of another face, albeit inverted, may have promoted
competition for neural representation in the same region
[50].
Our results establish that there is rapid adaptation to emotional content in faces, but there is still an open question
about the nature of the channel through which emotional
information is extracted. It has been argued elsewhere that
there is a fast processing pathway for emotions, via the pulvinar to the amygdala. Information derived from this pathway
could be sufficient to give category information about emotional expression, and could be driving aftereffect induction
at short durations. One putative characteristic of this pathway is that it is more sensitive to information in the low spatial frequency range. This motivated us to investigate
whether the aftereffects were expressed independently in
High Spatial Frequency (HSF) and Low Spatial Frequency
(LSF) representations of the same face.
ADAPTATION OF LOW AND HIGH SPATIAL FREQUENCY CHANNELS
Experiment 3
Experiment 3 was designed to test the possibility that a
fast emotional pathway preferring LSF information was driving the aftereffect at the shortest duration (23.5ms) at which
we reported significant effects for emotional expression. We
also included a shorter duration (11.7ms), in case 23.5ms
was insufficiently brief for channel specific effects. We
chose to use a Broad Spatial Frequency (BSF, i.e. unfiltered)
adaptor, and to look for effects in HSF and LSF faces derived from that adaptor. If LSF information was being used
to drive the category decision, we expected to find more adaptation in the LSF face target, and for the magnitude of the
aftereffect to be uncorrelated with the magnitude of the aftereffect expressed in HSF targets. Previous work, for example [12], has concentrated on whether adaptation can be induced independently by HSF and LSF adaptors. They found
that configural face adaptation in HSF and LSF channels is
highly specific, and that aftereffects do not survive changes
in spatial frequency.
Aftereffects in the Perception of Emotion Following Brief, Masked Adaptor Faces
HSF and LSF information is thought to be processed
through largely independent channels, at early stages in the
visual system, and presumably integrated into a single percept before reaching higher cortical areas. It is hypothesized
that the amygdala and connected sub-cortical systems preferentially process low frequency information [51-53]. In
Vuilleumier, Armony et al. (2003) [53], the responses of sub
cortical and cortical regions to a HSF or LSF face image
were measured after ‘priming’ with HSF and LSF versions
of the same image. Repetition suppression in the BOLD signal in the fusiform gyrus was reported for LSF faces with
HSF primes, but not vice versa. This asymmetrical transfer
may suggest that cortical processes establish repetition suppression (a form of adaptation), subsequently affecting also
sub-cortical processes. It thus remains unclear how independent the processing of HSF and LSF information about
high-level properties might be. Among face selective neuronal populations tuned to different spatial frequency components, some may be distinct cortical populations [54],
while others may reflect the cortical / sub-cortical divide [52,
54]. Notably [40, 43] found LSF fearful faces modulated the
P100, an early component of visual processing, while HSF
faces did not.
Our approach differs from both [12] and [51] as we examine the results of adaptation by an intact (BSF stimulus),
on its HSF and LSF components. We thus consider the question of whether, in the perception of natural face images,
LSF information and HSF information adapt independently
and whether more rapid extraction of LSF information,
means that at short adaptor durations, adaptation is expressed
more strongly in LSF components.
If our weakly masked short duration adaptors only permitted LSF information to be extracted, we might expect
uncorrelated adaptation to expression in the HSF and LSF
representations, and significantly less adaptation of HSF face
representations. An alternative possibility is that the aftereffect is induced by adaptation that occurs at a stage after integration of HSF and LSF information. In this case, adaptation
should either be seen in both, or neither.
Experimental Design and Methods
In order to first examine whether BSF images adapted
their HSF and LSF counterparts, we conducted a preliminary
experiment, using a long duration adaptor (1s), and examined the sizes of the aftereffects induced, and the shifts in the
decision function for each morph set. Then we conducted
experiments with short duration adaptors, one 23.5ms, and
one 11.7ms, and measured aftereffect sizes. Both experiments followed the same procedures, except for a change in
baseline measurement, and used the same stimuli set. For the
23.5ms and 11.7ms adaptation conditions, we also included
20 trials asking subjects to classify the content of the masked
adaptors, as part of a pre-training block (similar to the classification tasks in experiment 2).
Stimuli. Spatially filtered stimuli were derived from four
morph sets. The images were filtered in Matlab with a 2dimensional Butterworth filter, at a range of cutoff frequencies, and the HSF images at the endpoints of the continua
(not the morphs) were also weakly smoothed with a gaussian
filter to simulate the loss of high frequency components observed to result from the morphing process. The lumines-
The Open Behavioral Science Journal, 2008, Volume 2
47
cence of the images was then renormalized. Psychophysical
testing, aimed at balancing the difficulty of discrimination
within the high and low pass filtered morphs, led us to select
cut off frequencies of 12 cycles per image (LSF), and 40
cycles per image (HSF; see Fig. (6)) consistent with the
ranges of cutoff frequencies that support identity judgments
in other experiments [55]. Subjects were 50cm from the
screen and the stimuli were 12x12cm wide (256x241 pixels),
with the face image occupying the central 7cm, resulting in a
LSF cut off of approximately 0.8 cycles per visual degree,
about 6-7 cycles per face. HSF stimuli had a cutoff about 23
cycles per face. Pilot study subjects (N=20) asked to categorize the end point targets achieved accuracy levels of 73%
(LSF) and at 82% (HSF), which were not significantly different.
Procedure (1s Adaptor). Subjects first classified a baseline block as containing all of the target stimuli, both HSF
and LSF, as either emotional or neutral expressions, with no
adaptation.
Subjects then classified targets in four separate blocks
(order counterbalanced between subjects). Each block contained both high frequency and low frequency versions of
faces sampled from a particular continuum, preceded by the
BSF version of the face (the adaptor). Blocks were constructed as follows. The adaptor was shown for ten presentations; then four target trials in random order, as Adaptor,
followed by HSF, Adaptor followed by LSF, Adaptor followed by LSF and then HSF, or Adaptor followed by HSF,
then LSF. The sequence of trials then repeated. There were
60 trials per stimulus block comprising HSF and LSF
morphs from positions 10-20 (34%-68%) inclusive, and 16
HSF & LSF versions of the endpoints (3.4% and 93%).
Adaptors and targets were shown for one second. Target
morphs and endpoints were randomly sorted, and each subject saw each morph at least twice, once directly after the
adaptor, and once following one intervening target of the
other spatial frequency.
Procedure (23.5ms, and 11.7ms). The procedure for the
conditions with brief adaptors (23.5ms, and 11.7ms), was
essentially the same, except that the stimulus blocks were
lengthened to 80 adaptor trials, with each target preceded by
an adaptor, and the trial order fully randomized. 20 baseline
(no adaptor) trials were included within the blocks. Subjects
still classified a pre-training block containing all the stimuli,
but these responses were discarded.
Subjects. Fourteen subjects saw 1s adaptors, Fourteen
subjects saw 23.5ms adaptors, and seventeen subjects saw
11.7ms adaptors. Six subjects also performed the same procedure with the same proportion of prime to test stimuli, on
full-spectrum faces, thus enabling us to compare HSF and
LSF adaptation expression to the BSF adaptation expression
between subjects.
Analysis. Aftereffects were measured by the average
difference in proportion of the response ‘emotional’ following the emotional adaptor, minus the proportion of response
‘emotional’, following the neutral adaptor. In the 1s adaptor
condition we initially analyzed the responses preceded by the
adaptor separately from responses where there was an intervening stimulus, but found that there was no difference between them. For further analyses we collapsed across these
two sorts of trials. We also fitted nonparametric thresholds to
48 The Open Behavioral Science Journal, 2008, Volume 2
van Rijsbergen et al.
the responses to spatially filtered, HSF and LSF targets, and
unfiltered BSF targets. The final part of the analysis examined the correlation between the expression of the aftereffect
in HSF and LSF targets, within individual subjects.
subjects who saw the BSF targets, and found significant aftereffects, BSF (F{1,5}=30, p<0.05, MSE=3.9) which were
of intermediate size and not significantly different from the
effects in the LSF targets.
Results: 1s Adaptor
The non parametric threshold difference was largest for
HSF morphs (21% units), smaller for BSF (14% units) and
much smaller for LSF stimuli (3.4%). It is noticeable that the
thresholds for both emotional and neutral-adapted LSF stimuli slightly shifted towards the emotional, relative to baseline. The accuracy of classification of the endpoints also increased relative to baseline in both emotional and neutral
adaptor conditions. Both changes probably reflect increasing
subject familiarity with the face set.
The graphs of Fig. (5) show both the baseline (a) responses to HSF and LSF targets, and then the decision functions to HSF (a), LSF (b), and BSF (c), after emotional and
neutral adaptation. In each case, there is a clear difference
between the two decision functions. We found significant
aftereffects in three-way ANOVAs (morph level, and adaptor type, subject) for each morph set: HSF (F{1,13}=79,
p<0.0001, MSE=10.8) and LSF (F{1,13}=18.26, p=0.02,
MSE=2.5). Average aftereffect (Neutral adaptation – Emotional adaptation), within subjects, was greater than zero for
HSF (mean difference +/-standard deviation, 0.27+/-0.21),
t{13}=4.9, p<0.02); and LSF (0.12+/-0.17, t{13}=2.4,
p<0.05). The average difference between the aftereffects in
HSF and LSF sets was also significant (0.15+/-0.19,
t{26}=2.11, p<0.05). We also analyzed the data from the six
We also examined the extent to which adaptation of low
and high pass filtered stimuli was correlated within the responses to each stimulus for each of the fourteen subjects.
We compared the difference between an individual subject’s
mean proportion of classification as ‘emotional’ before and
after adaptation, for each morph set. Neutral adaptation expressed in the HSF morph set was weakly but significantly
Fig. (5). Aftereffects with spatially filtered targets. To enable comparison with intact images (BSF), and the decision functions to the shorter
duration targets in Fig. (6), we show all the decision functions to each different kind of target. Graph (a) shows the baseline condition without adaptation, with the average proportion of emotion reported with HSF targets (triangles, and dashed linear fit) and LSF targets (circles,
dot-dashed fit), plotted against percent emotion in the target. Graphs (b-d) show the proportion of emotional responses after neutral (solid
line) and emotional (dashed line) adaptors, plotted against the percent emotion of high (b), low (c) and broad (d) spatial frequency stimuli.
The pre-adaptation baseline data (dotted line) in the BSF condition is also plotted in (d). Error bars are +/- S.E.M. In the centre is an example
spatially filtered face shown in the three conditions. The graphs show a larger adaptation aftereffect with HSF targets, than with LSF targets.
Aftereffects in the Perception of Emotion Following Brief, Masked Adaptor Faces
correlated with that expressed in the corresponding LSF set
(r{27}=0.35, p<0.05, for single tailed t-test). Emotional adaptation in the HSF set was not correlated with that in the
LSF set.
Results: 23.5ms Adaptor
The results of adaptation with a 23.5ms adaptor are
summarized in the right-most panels of Fig. (6). We find that
significant aftereffects in HSF and LSF stimuli are correlated
after a 23.5ms adaptor, but that after the 11.7ms, only a weak
residual aftereffect remains in the HSF targets. Subjects were
73%+/-16% (STD) accurate on the individual adaptors, significantly above chance (50%) (t-test p<.0001). The upper
panels of Fig. (6) show a difference in the two adaptation
conditions for both HSF and LSF targets.
A 3-way ANOVA with factors adaptor type (neutral,
emotional and none) and morph level, and with subjects as a
random effect, found a significant effect of adaptor type in
both LSF (F{2,26}=8.42, p<0.02, MSE=0.14) and HSF targets (F{2,26}=3.24, p=0.05, MSE=0.108). One-tailed paired
t-tests found the average aftereffect was less than zero for
both HSF, p<0.05, and LSF faces, p<0.02. The effect was
slightly, but not significantly larger in the LSF targets p=0.6.
The average within subject difference between emotional
The Open Behavioral Science Journal, 2008, Volume 2
49
and baseline was also significantly less than zero, in both
HSF p<0.04, and LSF p<0.002. The differences are represented on the bar chart in Fig. (6).
To examine the persistence of the aftereffects, we separated the baseline trials into those trials following an emotional adaptor trial, and those following a neutral trial, and
found there was no significant difference between them. Accuracy on the endpoint images of the baseline (with no adaptation) was 82%+/-29% for HSF targets and 84%+/-26%, for
LSF.
The size of the adaptation aftereffect (i.e., the magnitude
of the difference in responses between emotional and neutral
adaptation in each subject) was strongly correlated between
the HSF and LSF stimuli, r{13}=0.92, p<0.0001. This relationship is represented in the scatter graph in Fig. (6).
The extent to which subjects were able to categorize the
adaptors (percent correct on the discrimination task) was also
weakly correlated with the average adaptation effect in the
HSF stimuli, r=0.47, p=0.09 and significantly correlated with
the effect in LSF stimuli, r=0.53 p<0.05. This correlation
was largely driven by two outlying subjects, that showed the
largest aftereffect, and near perfect adaptor discrimination.
The correlation between the HSF and LSF effects, however,
Fig. (6). Brief adaptors and spatially filtered targets.. The probability of emotional responses after a neutral adaptor (solid line) and an emotional adaptor (dashed line) plotted against percentage emotion in the target, shown for 23.5ms adaptor durations (top), and for 11.7ms durations (bottom) separated into HSF (triangles) and LSF (circles) targets. Errors are not shown, for the sake of clarity. The bar chart (top right)
shows the mean aftereffects, as mean differences +/- S.E.M. A very weak mean residual aftereffect is still present in the HSF targets, with
adaptor duration 11.7ms, smaller than the aftereffects for 23.5ms. The correlation between average adaptation of HSF and LSF targets at
both durations (dark 23,5ms, light 11.7ms) for each subject is shown with the regression line for the fourteen subjects that saw adaptors of
23.5s duration.
50 The Open Behavioral Science Journal, 2008, Volume 2
van Rijsbergen et al.
survived the removal of those two points (r{12}=66,
p<0.03).
Results: 11.7ms Adaptor
The results for the 11.7ms adaptor condition are
on the bottom panels of Fig. (6). In the HSF targets,
tereffect is now negligible, and no longer distributed
across the central morph levels. In the LSF targets, it
non-existent.
shown
the afevenly
is now
Subjects were 71%+/-17% accurate on the adaptor discrimination trials, and therefore still statistically above
chance (p<0.001).
A 3-way ANOVA, fixed factors morph and adaptor category (neutral, emotional, and none), with subject as a random effect, found a no overall effect for HSF targets,
F{2,32}=2.59, p=0.09, MSE=0.13, or LSF, F{2,32}=1.95,
p=0.16, MSE=0.016. However, there was a residual effect
for HSF faces, in the difference between neutral and emotional adapted responses, which was significantly less than
zero in a one tailed t-test, p<0.03. We also checked to see
whether some individual subjects might show aftereffects,
looking for a correlation between LSF and HSF adaptation
effects, but this was now close to zero, r=-0.08.
DISCUSSION
The main purpose of these experiments was to test the
possibility that rapid emotional processing through a LSF
sensitive channel was driving the aftereffect at very short
adaptor durations. The shortest duration that we expected to
see a reliable aftereffect was 23.5ms, based on the results of
experiment 2 but we included a shorter 11.7ms duration as a
control, to check for a residual, smaller effect in LSF targets.
We found no evidence of a channel specific effect in LSF
at short durations, as the reliable aftereffects in LSF and HSF
targets at 23.5ms were strongly correlated within subjects. If
anything, we found stronger effects in HSF, at both 11.7ms,
and long (1s) durations. There are several explanations for
this finding. One possibility is that adaptation of non face
specific features, e.g. lines and curves, contributes to adaptation of HSF targets. Channel specific attention may also play
a role in determining the level of adaptation to adaptors
when there is a long period of exposure.
Although subjects were able to discriminate the content
of both the 23ms, and the 11.7ms adaptors, the effects on the
following filtered targets were markedly different for the two
durations. This is summarized in the last panel of Fig. (6).
A ‘coarse to fine’ hypothesis e.g., [56] suggests that LSF,
or ‘global’ information is processed first, with HSF information, about the fine detail of image features, gradually added.
Our results do not immediately support such a hypothesis.
The shortest duration induces aftereffects on the categorization of HSF targets, rather than in LSF targets. A second
possibility [55, 57-59] is that HSF information can also be
accessed rapidly, but whether this occurs depends on the
demands of the task. This explanation fits our data better, as
one possibility is that the 11.7ms presentation time, while
below the threshold required to fully integrate information
about the adaptor derived through different channels (if any),
perhaps allows subjects to perform the discrimination task on
the basis of some individual feature. A single feature – perhaps a high contrast feature such as the mouth, or the whites
of the eyes, might be able to induce a rapid, very weak, “featural” aftereffect, that influences category decision [60],
without necessarily inducing the category based (or, better,
“whole image-based”) aftereffect.
Further experiments are needed to investigate whether a
single feature alone, such as eye shape or mouth shape, can
induce an aftereffect at the category level, in the BSF image.
We did not find evidence of an aftereffect at 11.7ms in experiment 2, but experiment 3 was a very much more sensitive design than the preceding experiments with multiple
durations and conditions.
GENERAL DISCUSSION
The main findings of this paper are as follows. Experiment 1 found that the adaptation aftereffect magnitude depends on the duration of the adaptor, rather than on preceding exposure, and that significant adaptation could be induced with a 23.5ms adaptor, very much shorter than anything previously reported. This is consistent with the short
adaptation times used in complex shape adaptation, for example, by Suzuki and Cavanagh (1998).
In experiment 2 we also found that the aftereffect is sensitive to backward masking. We hypothesize that this sensitivity is best understood as an effect of subjects’ accuracy at
extracting category information from the adaptor. More
speculatively, the findings are compatible with the possibility that part of the aftereffect is mediated by adaptation of
relatively late neural firing, in high-level areas [61], consistent with translation invariance of the aftereffect to relatively
short adaptors.
Our finding that adaptation to emotional content occurs
at shorter durations than adaptation to identity content supports the idea that information about emotional content is
available earlier and hence to some extent independently, of
identity information. However, experiment 1 showed that at
least part of the emotion aftereffect is identity specific.
Moreover, the results of experiment 3 in the spatial frequency domain do not suggest that adaptation is mediated
through independent spatial frequency specific pathways, as
we did not observe a bias towards the adaptation of low spatial frequency information. We did notice a bias towards
adaptation of HSF in experiment 3, where we used relatively
long, 1 second, adaptor durations.
One possible interpretation of the bias towards HSF is
that high spatial frequency information becomes more important in relatively late occurring processing of faces. In
particular, perhaps the integration of multiple features in
HSF information requires more processing time. We note
that one of the differences between the inverted face mask,
and the quantized face mask, is the presence of more HSF
information in the inverted face. It is possible that it is this
late processing, or integration of HSF information that is
interrupted by the strong masking in experiment 2.
It is interesting that it appears to be masking the adaptor
with a different face that interferes with the aftereffect, perhaps by competing for neural representational resources.
Further work will establish whether non face salient, meaningful images can also produce this effect.
The results of experiment 2 also raised questions about
the relationship between the explicit extraction of category
Aftereffects in the Perception of Emotion Following Brief, Masked Adaptor Faces
information measured in the classification task, and the aftereffect. Our data are too sparse to allow us to say whether
the two tasks depend differently on duration. The relationship between accuracy on the classification task, and aftereffect magnitude, does suggest that the two tasks use some of
the same information. In further work we will consider the
relationship at additional intermediate durations.
The final issue, which we deliberately do not raise in this
paper, is the issue of the contribution of subjective awareness
of the adaptor to aftereffect magnitude. We presume that
awareness is correlated on average, with duration. Further
investigation will uncover whether awareness of the adaptor
on individual trials plays a role in aftereffect induction.
[18]
[19]
[20]
[21]
[22]
[23]
[24]
ACKNOWLEDGEMENTS
This work was supported by a HSFP grant
RGP0047/2004-c to A.T. and we thank Ray Dolan for ideas
about spatial frequency and Nick Furl for comments on an
early version of the manuscript.
[25]
Supplementary Material
[27]
For experiment 1 we also include some additional discussion of methods in supplementary sections. It can be viewed
at www.bentham.org/open/tobsj
[26]
[28]
[29]
REFERENCES
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
Leopold DA, O'Toole AJ, Vetter T, et al. Prototype-referenced
shape encoding revealed by high-level aftereffects. Nat Neurosci
2001; 4: 89-94.
Webster MA, Kapling D, Mizokami Y, et al. Adaptation to natural
facial categories. Nature 2004; 428: 557-61.
Leopold DA, Rhodes G, Muller KM, et al. The dynamics of visual
adaptation to faces. Proc Biol Sci 2005; 272: 897-904.
Morrison DJ, Schyns PG. Usage of spatial scales for the categorization of faces, objects, and scenes. Psychon Bull Rev 2001; 8: 45469.
Harris CS, Gibson AR. Is orientation-specific color adaptation in
human vision due to edge detectors, afterimages, or "dipoles"?
Science 1968; 162: 1506-7.
Paradiso MA, Shimojo S, Nakayama K. Subjective contours, tilt
aftereffects, and visual cortical organization. Vision Res 1989; 29:
1205-13.
Gibson JJ, Radner M. Adaptation, aftereffect and contrast in the
perception of tilted lines I. Quantitative studies. J Exp Psychol
1937; 20: 453-67.
Vul E, MacLeod DI. Contingent aftereffects distinguish conscious
and preconscious color processing. Nat Neurosci 2006; 9: 873-4.
Menghini F, van Rijsbergen N, Treves A. Modelling adaptation
aftereffects in associative memory. Neurocomputing 2007; 70:
2000-4.
Clifford CW, Webster MA, Stanley GB, et al. Visual adaptation:
Neural, psychological and computational aspects. Vision Res 2007;
47: 3125-31.
Webster MA, MacLin O. Figural aftereffects in the perception of
faces. Psychon Bull Rev 1999; 6: 647-53.
Yamashita JA, Hardy JL, De Valois K, et al. Stimulus selectivity of
figural aftereffects for faces. J Exp Psychol Hum Percept Perform
2005; 31: 420-37.
Hsu SM, Young A. Adaptation effects in facial expression recognition. Vis Cogn 2004; 11: 871-99.
Watson TL, Clifford CW. Pulling faces: an investigation of the
face-distortion aftereffect. Perception 2003; 32: 1109-16.
Zhao L, Chubb C. The size-tuning of the face-distortion aftereffect. Vision Res 2001; 41: 2979-94.
Rhodes G, Jeffrery L, Watson T, et al. Orientation-contingent face
aftereffects and implications for face-coding mechanisms. Curr
Biol 2004; 14: 2119-23.
Kovacs G, Zimmer M, Harza I, et al. Position-specificity of facial
adaptation. Neuroreport 2005; 16: 1945-9.
[30]
[31]
[32]
[33]
[34]
[35]
[36]
[37]
[38]
[39]
[40]
[41]
[42]
[43]
[44]
The Open Behavioral Science Journal, 2008, Volume 2
51
Jiang F, Blanz V, O'Toole AJ. Probing the visual representation of
faces with adaptation: A view from the other side of the mean.
Psychol Sci 2006; 17: 493-500.
Jiang F, Blanz V, O'Toole AJ. The role of familiarity in threedimensional view-transferability of face identity adaptation. Vision
Res 2007; 47: 525-31.
Rhodes G, Jeffrey L, Clifford DA, et al. The timecourse of higherlevel face aftereffects. Vision Res 2007; 47: 2291-6.
Suzuki S, Cavanagh P. A shape-contrast effect for briefly presented
stimuli. J Exp Psychol Hum Percept Perform 1998; 24: 1315-41.
Fang F, Murray FO, Kersten D, et al. Orientation-tuned FMRI
adaptation in human visual cortex. J Neurophysiol 2005; 94: 418895.
Suzuki S. Attention-dependent brief adaptation to contour orientation: a high-level aftereffect for convexity? Vision Res 2001; 41:
3883-902.
Kovacs G, Zimmer M, Harza I, et al. Adaptation duration affects
the spatial selectivity of facial aftereffects. Vision Res 2007; 47:
3141-3149.
Bacon-Mace N, Mace MJM, Fabre-Thorpe M, et al. The time
course of visual processing: backward masking and natural scene
categorisation. Vision Res 2005; 4: 1459-69.
Rolls ET, Tovee MJ, Purcell EG, et al. The responses of neurons in
the temporal cortex of primates, and face identification and detection. Exp Brain Res 1994; 101: 473-84.
Kovacs G, Vogels R, Orban GA. Cortical correlate of pattern
backward masking. Proc Natl Acad Sci USA 1995; 92: 5587-91.
Humphreys GW, Bruce V. Visual cognition: computational, experimental, and neuropsychological perspectives. Hove Lawrence
Erlbaum Associates 1989.
Green MF, Glahn D, Engel SA, et al. Regional brain activity associated with visual backward masking. J Cogn Neurosci 2005; 17:
13-23.
Fahrenfort JJ, Scholte HS, Lamme VA. Masking disrupts reentrant
processing in human visual cortex. J Cogn Neurosci 2007; 19:
1488-97.
Del Cul A, Baillet S, Dehaene S. Brain dynamics underlying the
nonlinear threshold for access to consciousness. PLoS Biol 2007; 5:
e260.
Rolls ET, Tovee MJ, Panzeri S. The neurophysiology of backward
visual masking: information analysis. J Cogn Neurosci 1999; 11:
300-11.
Loffler G, Gordon GE, Wilkinson F, et al. Configural masking of
faces: evidence for high-level interactions in face perception.
Vision Res 2005; 45: 2287-97.
Whalen PJ, Rauch SL, Etcoff EL, et al. Masked presentations of
emotional facial expressions modulate amygdala activity without
explicit knowledge. J Neurosci 1998; 18: 411-8.
de Gelder B, Vroomen J, Portois G, et al. Non-conscious recognition of affect in the absence of striate cortex. Neuroreport 1999; 10:
3759-63.
Morris JS, Ohman A, Dolan RJ. Conscious and unconscious emotional learning in the human amygdala. Nature 1998; 393: 467-70.
Morris JS, Friston KJ, Buchel C, et al. A neuromodulatory role for
the human amygdala in processing emotional facial expressions.
Brain 1998; 121: 47-57.
Pizzagalli D, Regard M, Lehmann D. Rapid emotional face processing in the human right and left brain hemispheres: an ERP study.
Neuroreport 1999; 10: 2691-8.
Batty M, Taylor MJ. Early processing of the six basic facial emotional expressions. Brain Res Cogn Brain Res 2003; 17: 613-20.
Pourtois G, Dan ES, Granjean D, et al. Enhanced extrastriate visual
response to bandpass spatial frequency filtered fearful faces: time
course and topographic evoked-potentials mapping. Hum Brain
Mapp 2005; 26: 65-79.
Eimer M, Holmes A. Event-related brain potential correlates of
emotional face processing. Neuropsychologia 2007; 45: 15-31.
Vuilleumier P, Armony JL, Clarke K, et al. Neural response to
emotional faces with and without awareness: event-related fMRI in
a parietal patient with visual extinction and spatial neglect.
Neuropsychologia 2002; 40: 2156-66.
Morris JS, deBonis M, Dolan RJ. Human amygdala responses to
fearful eyes. Neuroimage 2002; 17: 214-22.
Bar M. A cortical mechanism for triggering top-down facilitation in
visual object recognition. J Cogn Neurosci 2003; 15: 600-9.
52 The Open Behavioral Science Journal, 2008, Volume 2
[45]
[46]
[47]
[48]
[49]
[50]
[51]
[52]
[53]
van Rijsbergen et al.
Dolan RJ, Vuilleumier P. Amygdala automaticity in emotional
processing. Ann N Y Acad Sci 2003; 985: 348-55.
Morris JS, Ohman A, Dolan RJ. A subcortical pathway to the right
amygdala mediating "unseen" fear. Proc Natl Acad Sci USA 1999;
96: 1680-5.
Matsumoto D, Ekman P. Japanese and Caucasian facial expressions
of emotion (JACFEE) and neutral faces(JACNEUF). 1988.
Bachmann T, Luiga I, Poder E. Variations in backward masking
with different masking stimuli: I. Local interaction versus attentional switch. Perception 2005; 34: 131-7.
Costen NP, Shepard J, Ellis HD, et al. Masking of faces by facial
and non facial stimuli. Vis Cogn 1994; 1: 227-251.
Keysers C, Perrett D. Visual masking and RSVP reveal neural
competition. Trend Cogn Sci 2002; 6: 120-25.
Johnson M. Subcortical Face Processing. Nat Neurosci Rev 2005;
6: 766-74.
Winston JS, Vuilleumier P, Dolan RJ. Effects of low-spatial frequency components of fearful faces on fusiform cortex activity.
Curr Biol 2003; 13: 1824-9.
Vuilleumier P, Armony JL, Driver J, Dolan RJ. Distinct spatial
frequency sensitivities for processing faces and emotional expressions. Nat Neurosci 2003; 6: 624-31.
Received: July 25, 2008
[54]
[55]
[56]
[57]
[58]
[59]
[60]
[61]
Gauthier I, Kurby K, Skudlasrski P, et al. Individual differences in
FFA activity suggest independent processing at different spatial
scales. Cogn Affect Behav Neurosci 2005; 5: 222-34.
Schyns PG, Oliva A. Dr. Angry and Mr. Smile: when categorization flexibly modifies the perception of faces in rapid visual presentations. Cognition 1999; 69: 243-65.
Goffaux V, Rossion B. Faces are "spatial"--holistic face perception
is supported by low spatial frequencies. J Exp Psychol Hum Percept Perform 2006; 32: 1023-39.
Ozgen E, Payne H, Sowden P, et al. Retinotopic sensitisation to
spatial scale: evidence for flexible spatial frequency processing in
scene perception. Vision Res 2006; 46: 1108-19.
Oliva A, Schyns PG. Coarse blobs or fine edges? Evidence that
information diagnosticity changes the perception of complex visual
stimuli. Cognit Psychol 1997; 34: 72-107.
Joyce CA, Schyns PG, Gosselin F, et al. Early selection of diagnostic facial information in the human visual cortex. Vision Res 2006;
46: 800-13.
Williams MA, Moss SA, Bradshaw JL. A unique look at face processing: the impact of masked faces on the processing of facial features. Cognition 2004; 91: 155-72.
Furl N, van Rijsbergen N, Treves A, et al. Experience-dependent
coding of facial expression in superior temporal sulcus. Proc Natl
Acad Sci USA 2007; 104: 13485-9.
Revised: September 5, 2008
Accepted: September 17, 2008
© van Rijsbergen et al.; Licensee Bentham Open.
This is an open access article licensed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/bync/3.0/) which permits unrestricted, non-commercial use, distribution and reproduction in any medium, provided the work is properly cited.