Arch Sex Behav (2016) 45:829–841

DOI 10.1007/s10508-016-0725-3

ORIGINAL PAPER

Measuring Implicit Sexual Response Biases to Nude Male and

Female Pictures in Androphilic and Gynephilic Men
Liadh Timmins1 • Dermot Barnes-Holmes2 • Claire Cullen3

Received: 27 June 2011 / Revised: 18 February 2016 / Accepted: 24 February 2016 / Published online: 14 March 2016
Ó The Author(s) 2016. This article is published with open access at Springerlink.com

Abstract Snowden, Wichter, and Gray (2008) demonstrated orientationwereconsistentlyhigh.Thefindingssupport theIRAP

that an Implicit AssociationTest anda PrimingTask both predicted as a potentially valuable tool in the study of sexual preferences.
thesexualorientationofgynephilicandandrophilicmenintermsof
their attraction biases towards pictures of nude males and females. Keywords Implicit measurement
Sexual orientation

For both measures, relative bias scores were obtained, with no Erotic preference
information on the separate response biases to each target
gender. The present study sought to extend this research by
assessing both relative and individual implicit biases using the Introduction
Implicit Relational Assessment Procedure (IRAP). An explicit
measure screened for men with androphilic (n = 16) or gyne- Researchers studying sexual orientation and sexual preference1
philic (n = 16) orientations on the dimensions of‘‘sexual attrac- have begun to explore methods designed to measure so-called
tion,’’‘‘sexual behavior,’’‘‘sexual fantasies,’’‘‘hetero/gay lifestyle,’’ implicit attitudes. Such attitudes typically involve immediate,
and‘‘self identification.’’The IRAP involved responding‘‘True’’or automatic (possibly unconscious), and non-declarative evalua-
‘‘False’’topicturesofnudemalesandfemalesaseitherattractiveor tions(DeHouwer,2006;DeHouwer,Teige-Mocigemba,Spruyt,
unattractive. Participants were required to respond in a manner & Moors, 2009; Fazio & Olson, 2003; Gawronski, 2009), which
consistent with their reported sexual orientation for half of the are contrasted with the deliberate and controlled evaluative judg-
IRAP’s test blocks and inconsistent for the other half. Response ments (i.e., explicit attitudes) captured by self-report measures
latencies were recorded and analyzed. The IRAP revealed a non- (Greenwald & Banaji, 1995; Nosek, 2007). There is an on-going
orthogonal pattern of biases across the two groups and had an debate concerning the nature of these two types of attitudes and
excellent ability to predict sexual orientation with areas under the how they operate and influence behavior (Gawronski & Boden-
curves of 1.0 for the relative bias score and .94 and .95 for the hausen, 2006; Rydell & McConnell, 2006; Wilson, Lindsey, &
bias scores for the male and female pictures, respectively. Schooler, 2000; for a recent review, see Hughes, Barnes-Holmes,
Correlations between the IRAP and explicit measures of sexual & De Houwer, 2011). However, there exists a general consensus
that measures of implicit and explicit attitudes are sensitive to
‘‘related but distinct constructs’’ (Nosek, 2005; but see Arkes &
Tetlock, 2004). Critically, a growing body of evidence indicates
& Liadh Timmins that the two types of measure predict different types of behavior.
liadh.timmins@kcl.ac.uk Specifically, traditional self-report methodologies appear to
1
Department of Psychology, Institute of Psychiatry, King’s
predict intentional and controlled behaviors (Dovidio, Kawa-
College London, 5th Floor, Bermondsey Wing, Guy’s Hospital kami, Johnson, Johnson, & Howard, 1997; Dovidio, Kawakami,
Campus, London SE1 9RT, UK Smoak, & Gaertner, 2009), whereas scores obtained on implicit
2
Department of Experimental, Clinical and Health Psychology,
Ghent University, Ghent, Belgium 1
We use‘‘preference’’as a broad term for a number of relativistic responses
3
School of Nursing & Human Sciences, Dublin City University, including those related to sexual orientation. This should not be taken to
Dublin 9, Ireland imply that such responses are volitional.

123
830 Arch Sex Behav (2016) 45:829–841

measures typically track spontaneous, immediate and, perhaps, with a range of explicit measures of sexual preference (ranging
more automatic responses and judgments (Friese, Hofmann, & from r = .72 to .80).
Wanke, 2008; Galdi, Arcuri, & Gawronski, 2008; McConnell & The other measure of implicit preference, the PT, also
Leibold, 2001). predicted self-reported sexual orientation, but with a slight
This distinction may be of key importance for research into drop in accuracy relative to the IAT (i.e., AUC = 0.86) and,
sexual preferences. While sexual orientation is often conceptual- once more, a range of correlations were obtained between the
ized as unidimensional in nature, there are likely multiple under- implicit and explicit measures (ranging from r = .49 to .56),
lying constructs that determine human sexual behavior. Indeed, it although again these were weaker compared to the IAT. Finally,
is probable that explicit and implicit measures can tap into dif- the two implicit measures correlated with each other (r = .59).
ferent classes of associated processes. For example, implicit mea- Based on these findings, Snowden et al. (2008) concluded that
sures may reflect fleeting thoughts and fantasies, visual interest in ‘‘male sexual orientation to men or women can be indexed by
bodies of a particular sex, and/or arousal to those bodies, whereas implicit measures’’(p. 563).
explicit measures of sexual orientation may reflect desires to act A limitation to the research reported by Snowden et al.
on one’s arousal, strong sustained attractions to specific individu- (2008) is that one of their measures, the IAT, has a widely rec-
als, and/or other complex social information. Thus, implicit mea- ognized weakness. Specifically, it provides only one relative bias
sures of sexual preference may tap into a unique aspect of sexual score, which creates a lack of precision in determining the nature
orientation that self-report methodologies cannot, which could of the attitudes under study (see De Houwer, 2003). If, for exam-
present distinct patterns of responses within certain groups. For ple, participants responded more quickly on male-attractive and
groups that display these divergent response patterns, either type female-unattractive trials than on the reversed counterparts (i.e.,
of measure could prove to be a more accurate predictor of certain male-unattractive and female-attractive), a number of interpre-
types of sexual behavior, sexual behavior within certain contexts, tations are possible. For instance, participants may (1) have found
and/or sexual behavior altogether. males attractive and females aversive or (2) found both males and
The first published study that sought to determine if sexual femalesattractive,but theformermoreso,or(3)foundbothmales
preference could be indexed with implicit measures (Snowden, and females aversive, but the latter more so, or (4) found males
Wichter, & Gray, 2008) employed two of the most well estab- attractive and females neither aversive nor attractive or (5) found
lished methodologies, the Implicit Association Test (IAT) and a females aversive and males neither aversive nor attractive.
PrimingTask(PT).Maleparticipantswhoreportedthattheywere This is particularly relevant for two reasons. Firstly, this
either primarily androphilic or gynephilic completed both mea- severely restricts the IAT’s utility with bisexual individuals,
sures. given that bisexual individuals who experience strong, but
The critical parts of the IAT involved two types of com- not equal, sexual attraction to both males and females could
puter-based tasks. In one task, participants were required to be erroneously miscategorized as gynephilic or androphilic.
press the same button as quickly as possible if a picture of a Secondly, the IAT’s potential for exploring sexual aversion to
nude male or a word indicating sexually attractive was pre- the non-preferred gender in gynephilic and androphilic indi-
sented (e.g.,‘‘arousing,’’‘‘erotic,’’etc.); pressing a different button viduals is also limited. One might expect that such sexual
(as quickly as possible) was required if the computer presented a aversion can be assumed, however self-report data suggests
pictureofanudefemaleorawordindicatingsexuallyunattractive that while gynephilic males and females display aversion to
(e.g.,‘‘repulsive,’’‘‘repelling,’’etc.). In the other task, the catego- sex with those of their non-preferred gender, androphilic females
rization responses were reversed; pressing one button for male do not, and results for androphilic males are mixed (Freund,
pictures and unattractive and pressing the other button for female Langevin, Chamberlayne, Deosoran, & Zajac, 1974a; Freund,
pictures and attractive. As predicted, the androphilic participants Langevin, Cibiri, & Zajac, 1973; Freund, Langevin, & Zajac,
responded significantly more quickly when they were asked to 1974b; Israel & Strassberg, 2009; Rullo, Strassberg, & Israel,
categorize the male pictures with sexually attractive words and 2010). Additionally, phallometric testing suggest that aversion
the female pictures with sexually unattractive words then vice does not exist at the level of genital arousal in androphilic or
versa (male with unattractive and female with attractive). Also as gynephilic men (Freund et al., 1973, 1974a, 1974b), whereas
predicted, the gynephilic participants produced the opposite viewing time research suggests that it does exist in gynephilic
pattern to the androphilic participants; male pictures were cate- men, but not in androphilic or gynephilic women (Israel &
gorized more rapidly with unattractive and females with attrac- Strassberg, 2009; Rullo et al., 2010).
tive then vice versa. The relative difference in response latency To measure implicit attitudes to individual types of stim-
between the two types of task was thus consistent with the par- uli, an alternative non-relative measure is thus required. In
ticipants’ self-reported sexual preferences. Furthermore, the IAT fact, a number of researchers have attempted to develop such
data successfully predicted self-reported sexual orientation with non-relative tests, including, for instance, the Extrinsic Affective
an area under the curve (AUC) of 0.97 and correlated strongly Simon Test (De Houwer, 2003), the Go/No-Go Association Task

123
Arch Sex Behav (2016) 45:829–841 831

(Nosek & Banaji, 2001), and the Implicit Relational Assessment gynephilic group?). Third, would both groups show significant
Procedure (IRAP) (Barnes-Holmes et al., 2006). As an aside, the IRAP effects consistent with aversion to their self-reported non-
PT employed by Snowden et al. (2008) could have yielded sep- preferred gender? Fourth, would the two IRAP bias scores pro-
arate bias scores for male and female pictures but these were not duce similar or different levels of predictive validity in terms of
reported in the article, presumably because they could not be identifying the sexual orientation of the participants? Fifth, would
compared meaningfully with the single relative IAT scores. the IRAP bias scores correlate with the explicit measures
The present study sought to replicate and extend the research employed in the study?
conducted by Snowden et al. (2008) by assessing both relative and
individual implicit biases for male and female pictures using the
IRAP. Research has shown that the IRAP (1) compares well with Method
the IAT as a measure of individual differences (Barnes-Holmes,
Murtagh, Barnes-Holmes, & Stewart, 2010c; Barnes-Holmes, Participants
Waldron,Barnes-Holmes,&Stewart,2009),(2)isnoteasilyfaked
(McKenna, Barnes-Holmes, Barnes-Holmes, & Stewart, 2007), Given that the current study was a ‘‘first test’’ of the validity and
(3) may be used as a measure of implicit self-esteem (Timko, utility of the IRAP as a measure of sexual orientation, participants
England, Herbert, & Forman, 2010; Vahey, Barnes-Holmes, were 16 gynephilic men (M age = 23.8 years; range, 18–54) and
Barnes-Holmes, & Stewart, 2009), and (4) produces effects that 16 androphilic men (M age = 22.8; range, 18–39). Gynephilic and
indicate levels of bias not recorded with explicit measures androphilic men tend to display category-specific sexual respon-
(Barnes-Holmes, Murphy, Barnes-Holmes, & Stewart, 2010b; ses at both a subjective and genital arousal level, whereas this is
Dawson, Barnes-Holmes, Gresswell, Hart, & Gore, 2009; Power, less so the case in gynephilic women (Chivers, Rieger, Latty, &
Barnes-Holmes, Barnes-Holmes, & Stewart, 2009; Roddy, Ste- Bailey, 2004; Chivers, Seto, & Blanchard, 2007; Chivers, Seto,
wart, & Barnes-Holmes, 2010). Lalumiére, Laan, & Grimbos, 2010). This makes gynephilic and
One feature of the IRAP that was particularly important for androphilic men ideal to test the discriminability of the IRAP at
the current study is that it consists of multiple trial-types, this early stage.
which, in principle, permits the assessment of more than one Gynephilic participants were students of Maynooth Univer-
response bias (see Barnes-Holmes et al., 2010b). In the pre- sity. Androphilic participants were recruited through the Lesbian,
sent research, each IRAP trial presented either a picture of a Gay, Bisexual and Transgender society at Maynooth University
nude male or female as a label stimulus with either a positive and via snowball sampling through those participants. Consistent
(e.g.,‘‘arousing’’) or negative (e.g.,‘‘repulsive’’) target word. with Snowden et al. (2008), gynephilic men were operationally
The IRAP thus allowed us to determine separate responses defined as men with a relatively stable preference for sexual part-
biases for the male and female pictures for gynephilic and ners of the opposite gender and androphilic men were defined as
androphilic participants, as well as an overall relative IRAP men with a relatively stable preference for sexual partners of the
effect, similar to that reported by Snowden et al. (2008) for the same gender. Such preference was confirmed by a modified ver-
IAT and PT. sion of the Klein Sexual Orientation Grid (KSOG) (Klein, 1993;
The first aim of the current study was to replicate the find- Klein, Sepekoff, & Wolf, 1985), which showed all participants to
ings reported by Snowden et al. (2008) with the IRAP. That is, we be either primarily gynephilic or androphilic (see next section for
predicted that the overall relative IRAP effects would differ sig- details).Volunteersreceivedachocolatebrowniefortheirpartici-
nificantly between men who reported being primarily gynephilic pation, but no other rewards or incentives were offered.
versus androphilic and that this measure would successfully
discriminate between the groups at a level similar to that obtained Measures
with the IAT and PT. We also predicted that the overall IRAP
effect would yield similarly high correlations with the explicit An information and consent booklet was used to brief par-
measuresofsexual orientationtothosereportedbySnowdenetal. ticipants. This consisted of the following brief summary of the
The second aim of the present research was more exploratory. general nature ofthestudy,as well asreproductionsofthe10 nude
Specifically, we sought to examine the separate IRAP effects stimuli to be used in the study and a copy of the consent form:
generated by the male and female pictures by addressing the ‘‘Our research investigates cognitive processes that are used in
following five questions. First, would the IRAP effects for the decisions that involve memory. We are seeking to develop and
male and female stimuli differ significantly for both the gyne- test theories of cognitive processes that occur inside and outside
philic and androphilic groups? Second, would both groups show of awareness in the routine use of memory. In this case, the cog-
significant IRAP effects consistent with their self-reported sexual nitive processes involved in making decisions about the sexual
orientation (i.e., an attraction bias for males only for the appeal of males and females are being investigated. As such, nude
androphilic group and an attraction bias for females only for the images of both males and females will be presented multiple

123
832 Arch Sex Behav (2016) 45:829–841

times during the experiment. Your identity as a subject is con- The semantic differentials involved two identical sets of
fidential. Further, you are free to discontinue participation at any six bipolar Likert scales, one for the concept‘‘sex with men is
time, without penalty.’’ (to me)’’ and another for the concept ‘‘sex with women is (to
The same five male and four of the five female picture stimuli me).’’The Likert scales each had a pair of opposite adjectives
used by Snowden et al. (2008), taken from the International at either end. These pairs were‘‘good/bad,’’‘‘beautiful/ugly,’’
Affective Picture System (IAPS) (Lang, Bradley, & Cuthbert, ‘‘pleasant/unpleasant,’’‘‘exciting/boring,’’‘‘nice/awful,’’ and
1997), were employed in the current study (male picture num- ‘‘attractive/unattractive.’’The scales ranged from 1 to 7, with
bers: 4460, 4500, 4534, 4550, 4561; female picture numbers: 4 as the neutral point. Higher numbers indicated a more favorable
4141, 4142, 4210, 4240). A fifth female picture (picture number: attitude, except in the case of the‘‘pleasant/unpleasant’’scales, in
4235) was chosen from the IAPS in lieu of the original fifth which the labels were reversed (to control for repetitive respond-
picture used by Snowden et al. (picture number: 4332) due to its ing). The data for this scale were recoded before the data analysis
unavailability. All pictures chosen by Snowden et al. were picked to render the direction of effects consistent with the other data.
for their erotic, but not pornographic content, as was the fifth The IRAP software, which was run on a standard personal
female picture in the current study; subjects in the pictures were computer, was written by the second author and is available
completely or almost completely nude, while not visibly sexu- upon request. Participants completed the study alone in a
ally aroused nor engaged in sexual activity. small quiet room free of distraction.
The five word stimuli pertaining to‘‘sexually attractive’’orig-
inally used by Snowden et al. (2008) were also employed in the Procedure
current study (i.e.,‘‘arousing,’’‘‘erotic,’’‘‘attractive,’’‘‘sensual,’’and
‘‘exciting’’). However, only four of the five original words per- Participants were informed that the study would consist of a short
taining to‘‘sexually unattractive’’were used (i.e.,‘‘repulsive,’’ questionnaire about their sexual orientation and behavior, fol-
‘‘repelling,’’‘‘repugnant,’’and‘‘repellent’’). During pilot testing, lowed by a computerized task. For ethical reasons, participants
the fifth word (‘‘forbidding’’) was deemed to be ambiguous in were also informed that both were intended as measures of sexual
the context of the IRAP because it had moralistic connotations, preference, but that the data were being collected anonymously
which applied to all of the nude images (both male and female) and as such could not be directly traced to them. In addition, the
irrespective of sexual orientation. Consequently, the word participants were informed they had the right to cease participa-
‘‘awful’’ was used in its place. tion at any time, as well as retract their data afterwards. Partici-
The explicit attitude measures consisted of the semantic dif- pants who inquired further as to how the IRAP measures sexual
ferential measures used by Snowden et al. (2008), as well as a preference were informed that it determines it based on their
version of the KSOG, modified to reflect the results of a factor responses to the stimuli, but no more specific information was
analysis of the instrument (Weinrich et al., 1993). The KSOG given.
consisted of five dimensions of sexual orientation (sexual attrac- If participants confirmed they were willing to continue, they
tion, sexual behavior, sexual fantasies, hetero/gay lifestyle, and were presented with the information and consent booklet,
self identification), all of which were assessed on a seven point described previously. Participants were then offered a minimum
scale across two temporal dimensions (past, defined as up to a of a 24-h ‘‘change-of-mind’’ period to allow them to reconsider
year ago, and present, defined as the last 12 months), as well as a their participation. To avoid inconveniencing participants unnec
third dimension of ideality (defined as what the participant would essarily, those who wished to continue with the study immedi-
like). Higher scores indicated a more androphilic attitude and ately were allowed to do so.
lower scores indicated a more gynephilic attitude. This resulted No participants chose to cancel their participation after the
in a total of 15 scores of sexual orientation (Cronbach’s alpha for change-of-mind period, and, upon their return, they were again
present study = .98). presented with the booklet and asked to sign the consent form
Mean scores were rounded off to the nearest whole number, if they wished to continue. Having signed, participants then com-
and this final score was used as a screening measure for the study pleted the explicit measures (the KSOG and semantic differ-
(KSOG scores were not rounded off to the nearest whole number entials).
for anything other than this screening). Scores of 1–3 were Subsequently, participants were seated in front of the com-
deemed to represent an overall sexual preference for women, puter, which presented the instructions and stimuli and recorded
scores of 5–7 an overall sexual preference for men, and a score of all responses. The IRAP software began by presenting a set of
4 a relative lack of definite preference for either men or women. instructions, which explained the IRAP task using illustrative
No participant had a score of 4 and all participants’ scores were in examples of the different types of trials, and giving a detailed
accordance with their reported sexual orientation, with gyne- account of what participants were required to do.
philic individuals scoring between 1 and 3 and androphilic The IRAP was presented in blocks of 40 trials. Trials consisted
individuals scoring between 5 and 7. of the simultaneous presentation of either a male or female nude

123
Arch Sex Behav (2016) 45:829–841 833

picture stimulus at the top of the screen, either an attractive or the words‘‘Too Slow’’appeared towards the center bottom of the
unattractive word stimulus in the middle of the screen and screen and remained there until the participant chose one of the
response options of‘‘True’’and‘‘False’’in the bottom left- and response options.
right-hand corners, with the instructions‘‘Press ‘D’ for’’and Participants were first presented with a set of two practice
‘‘Press ‘K’ for’’directly above the left and right response options, blocks. Participants were required to achieve an accuracy crite-
respectively. The left–right positioning of the two response rion of C80 % correct responses and a median response latency of
options, and therefore the keys required to select them, varied ran- B2000 ms. If these criteria were achieved, participants were then
domly across trials, with the constraint that they could not appear exposed to fixed set of six test blocks. If they were not achieved,
in the same positions across more than three successive trials. The the practice blocks were repeated until they were. Participants
different combinations of male/female and positive/negative were not required to achieve any performance criteria during the
words resulted in four possible trial types: Male-Attractive, Male- test blocks in order to proceed. However, accuracy and latency
Unattractive, Female-Attractive, and Female-Unattractive (see feedback were presented at the end of each block to encourage
Fig. 1). participants to maintain the performance criteria achieved during
During each block, participants had to respond in accordance the practice blocks.
with one of two rules, regardless of their own personal feelings: Blocks were presented in one of two possible sequences, each
(1)‘‘all females are attractive and all males are unattractive’’(de- alternating between the presentation of a female-attractive and a
fined as a female-attractive block) or (2)‘‘all males are attractive male-attractive block. In one sequence, participants were first
and all females are unattractive’’ (defined as a male-attractive exposed to a female-attractive block, whereas in the other
block). The trials were presented quasi-randomly with the con- sequence participants were first exposed to a male-attractive
straint that each of the four trial-types appeared 10 times within block. Block sequence was counterbalanced across participants.
each 40-trial block, all 10 picture and 10 word stimuli were pre- Upon completion of the IRAP, participants were thanked and
sented twice within each block and the same trial-type was not debriefed and reminded that if they wished they could still revoke
presented across successive trials. their data.
Choosing the response option deemed correct cleared the
screen for a 400 ms inter-trial interval and then the next trial was Data Analysis
presented. If the incorrect response option was chosen, a red X
appeared directly underneath the target word and remained there The primary datum for the IRAP was response latency defined as
until the participant chose the correct response option. If a par- time in milliseconds from the onset of a test trial until the emission
ticipant failed to respond within 2000 ms from the start of a trial, of a correct response. Consistent with the majority of published

Female-Attractive Male-Attractive

(Picture of a nude female) (Picture of a nude male)

Attractive Attractive
Females Females Males Males
Attractive Unattractive Attractive Unattractive

Select ‘d’ for Select ‘k’ for Select ‘d’ for Select ‘k’ for
True False True False

Female-Unattractive Male-Unattractive

(Picture of a nude female) (Picture of a nude male)

Awful Awful
Females Females Males Males
Unattractive Attractive Unattractive Attractive

Select ‘d’ for Select ‘k’ for Select ‘d’ for Select ‘k’ for
True False True False

Fig. 1 The four IRAP trial-types. The nude picture stimuli, word stimuli and response options (‘‘True’’and‘‘False’’) appeared simultaneously on each
trial. Arrows with superimposed text show which responses indicate which bias (text and arrows did not appear on screen)

123
834 Arch Sex Behav (2016) 45:829–841

IRAP studies, individual response latency data were transformed

into D-IRAP scores (see Barnes-Holmes, Barnes-Holmes,
Stewart, & Boles, 2010a) using an adaptation of the Greenwald,
Nosek, and Banaji (2003) D-algorithm.
TheD-algorithmproducedaD-IRAPscoreforeachofthefour
trial types. For the two female trial type scores, a positive score
indicated an attraction bias and a negative score indicated an
aversionbias,whereasforthetwomaletrial typescoresanegative
score indicated an attraction bias and a positive score indicated an
aversion bias. The mean of the two female trial type scores con-
stituted the female pictures D-IRAP score, and the mean of the
two male trial type scores multiplied by -1 constituted the male Fig. 2 Mean D-IRAP scores with standard error bars for female picture
and male picture trial-types for Gynephilic and Androphilic participants.
pictures D-IRAP score. A positive score thus indicated an attrac-
A positive score indicates a positive bias (attraction) and a negative score
tion bias, whereas a negative score indicated the opposite. The indicates a negative bias (aversion)
mean of the four trial-types scores constituted the overall mean D-
IRAP score. A positive score thus indicated a gynephilic bias (i.e.,
stronger attraction to female than male pictures) whereas a neg-
scores differed significantly from zero. Both scores for the gyne-
ative score indicated a androphilic bias (i.e., stronger attraction to
philic group were significant: female pictures, t(15) = 10.49, p\
male than female pictures).
.0001, d = 5.42; male pictures, t(15) = -3.15, p = .0066, d =
1.63. The male picture scores for the androphilic group also dif-
fered significantly from zero, t(15) = 4.95, p = .0002, d = 2.56,
Results
but the female picture scores did not.
The overall mean D-IRAP score was -.15 (SE = .04) for the
Implicit Measure
androphilic group and .45 (SE = .07) for the gynephilic group,
and this difference proved to be significant, t(30) = 7.69, p\
A preliminary analysis showed that block sequence (female-
.0001, with a very large effect size (d = 2.72).
attractive-first versus male-attractive-first) did not have a
significant effect on performance; hence, this variable was
removed from subsequent analyses. Prediction of Sexual Orientation
The mean D-IRAP scores for the male and female pictures are
shown in Fig. 2. The scores for the female pictures were .59 A main aim of the current research was to determine if an implicit
(SE = .06) for the gynephilic participants and .01 (SE = .06) for measure could be used to differentiate between the sexual pref-
the androphilic participants; for the male pictures, the scores were erences of gynephilic and androphilic men, and to measure this
-.31(SE = .10)forthegynephilic participants and.31 (SE = .06) predictive ability. As such, the same signal detection test
for the androphilic participants. The gynephilic participants thus employed by Snowden et al. was used here, which involved
showed a strong positive (attraction) bias towards the female constructing the Receiver Operator Characteristic (ROC). A
pictures with a negative (aversion)bias towards the male pictures. ROC is a graph in which the probability of a true positive, or a
In contrast, the androphilic group showed a strong positive bias ‘‘hit,’’isplottedagainst theprobability ofafalsepositiveora‘‘false
towards the male pictures, but virtually no directional bias for the alarm’’ (Fawcett, 2006). From this, the AUC can be calculated,
female pictures. which essentially is the statistical likelihood that a randomly
The mean D-IRAP scores for the male and female pictures chosen member of the‘‘positive’’group (in this case, gynephilic
were subjected to a 2 9 2 mixed repeated measures analysis of participants) will have a higher score than a randomly chosen
variance (ANOVA) with sexual orientation as a between-par- member of the ‘‘negative’’ group (in this case androphilic indi-
ticipant variable and IRAP trial-type (male versus female) as the viduals). Therefore, a test with perfect ability to predict group
within participant variable. The ANOVA yielded a significant membership would have an AUC = 1.0, and a test with no ability
interaction effect, F(1, 30) = 59.07, p\.0001, g2p = .66. Four to detect group membership would have an AUC = *0.5.
t tests were used to explore the interaction. Two unpaired t tests The ROCs for the overall D-IRAP score, female picture, and
showed a significant effect for both the female trial-type, t(30) = male picture scores are shown in Fig. 3. Using the overall score,
7.21, p\.0001, d = 2.55 and the male trial-type, t(30) = -5.29, the IRAP proved to be a perfect predictor of sexual orientation,
p\.0001, d = 1.89. Two paired t tests showed a significant effect with an AUC = 1.0 (p\.001). The ROC analysis for the female
for both the gynephilic group, t(15) = 6.7, p\.0001, d = 1.70, and male picture scores showed similarly strong abilities to iden-
and androphilic group, t(15) = -3.8, p = .0018, d = .95. Four tify sexual orientation, although these were not perfect, with
one sample t tests were conducted to determine if the D-IRAP female AUC = 0.95 (p\.001) and male AUC = 0.94 (p\.001).

123
Arch Sex Behav (2016) 45:829–841 835

Differential Preference Scales preference). All 12 t tests (six for the gynephilic group and six for
the androphilic group) proved to be highly significant (all ps\
To make the semantic differential measures compatible with .0001). A further 24 one-sample t tests were employed to deter-
the overall D-IRAP scores, the scores for the‘‘Sex with Men’’ mine if the‘‘Sex with Women’’and‘‘Sex with Men’’Likert ratings
Likert scales were subtracted from the scores for the‘‘Sex with differed significantly from 4 (i.e., a neutral preference). For the
Women’’Likert scales. Positive scores thus indicate a preference gynephilic group, 11 of the t tests were significant (ps\.01,
for women and negative scores a preference for men. These ds[1.5). Similarly, for the androphilic group, 11 of the t tests
measures arereferred toas Differential Preference Scales(DPSs), were significant (ps\.02, ds[1.4).
and the overall means for these data along with the overall means
from the two separate Likert scales are shown in Table 1. Klein Sexual Orientation Grid
In all cases, the direction of the effects was consistent with
predicted group differences. The data for the DPSs were entered The data for the KSOG were similarly entered into an unpaired t
into a 2 9 6 multivariate analysis of variance (MANOVA) with test. As would be expected, given that the KSOG was used as a
sexual orientation as a between-participant variable and the 6 screening measure, it too produced a large and significant differ-
DPS scores as within participant variables. The MANOVA ence (see Table 1).
yielded a significant main effect, F(1, 30) = 57.26, p\.001,
gp2 = .93. Six unpaired t tests were used to explore the nature of Relationship Between the Measures
this main effect and all 6 t tests were significant (see Table 1).
Twelve one-sample t tests were used to determine if the results Pearson correlation coefficients were computed to examine the
for the DPSs differed significantly from zero (i.e., a neutral relationships between the variables (see Table 2). The Overall D-

Fig. 3 Receiver operating characteristics of the ability of the overall under the curve (AUC) is 1.0 (p\.001) for the overall D-IRAP scores,
mean D-IRAP, female and male picture bias scores to predict sexual 0.95 (p\.001) for the female picture bias scores and 0.94 (p\.001) for
orientation. The straight diagonal lines represent chance level. The area the male picture bias scores

123
836 Arch Sex Behav (2016) 45:829–841

IRAP scores were correlated with the DPSs and the two D-IRAP attraction bias to one gender predicted an aversion bias to the
picture bias scores were correlated with their corresponding gen- opposite gender).
der-specific Likert scales (i.e., male picture bias scores with‘‘Sex
with Men’’ratings and female picture bias scores with‘‘Sex with
Women’’ratings). The three D-IRAP measures were also corre- Discussion
lated with the KSOG. All correlations with the Overall D-IRAP
scores were very high and significant, ranging between r = .77 The results of the current study supported Snowden et al.’s
(‘‘Pleasant’’Differential Preference Scale) and r = .84 (‘‘Good’’ (2008) conclusion that implicit measures can be used to dis-
Differential Preference Scale). All of the correlations between tinguish between men of different sexual orientations. In addi-
each of the two picture bias scores and the explicit measures were tion, the data indicated that the IRAP had a level of predictive
also significant, although in general they were slightly weaker validity that compared favorably with the levels reported by
than the overall D-IRAP correlations. Finally, a Pearson corre- Snowden et al. for the IAT and the PT. Furthermore, high
lation coefficient was calculated between the two D-IRAP pic- correlations between the IRAP and the explicit measures were
ture bias scores and this proved to be negative and significant, found, which again compared favorably with those reported
r = -.65, p\.0001 (the correlation was negative because an for the IAT and the PT, which Snowden et al. pointed out were

Table 1 Descriptive and inferential statistics for the explicit measures

Group Between groups comparison
Gynephlic Androphilic
M SD M SD t d

Measure
Good
Preference 4.4 1.7 -3.7 1.4 14.41*** 5.09
Sex w/women 6.8 0.4 3.1 1.2
Sex w/men 2.4 1.6 6.8 0.5
Beautiful
Preference 3.6 1.1 -2.4 1.8 11.42*** 4.04
Sex w/women 6.0 0.9 2.8 1.4
Sex w/men 2.4 1.3 5.3 1.4
Pleasant
Preference 4.7 1.7 -3.6 1.8 13.64*** 4.82
Sex w/women 6.9 0.3 2.8 1.4
Sex w/men 2.2 1.6 6.4 1.1
Exciting
Preference 2.7 1.6 -3.6 1.7 10.85*** 3.84
Sex w/women 6.3 1.0 2.9 1.6
Sex w/men 3.6 1.0 6.5 0.6
Nice
Preference 3.9 1.3 -2.9 2.0 11.44*** 4.05
Sex w/women 6.9 0.3 3.4 1.6
Sex w/men 3.0 1.3 6.3 1.1
Attractive
Preference 4.9 1.2 -3.4 2.0 13.99*** 4.95
Sex w/women 6.8 0.5 2.8 1.6
Sex w/men 1.9 1.2 6.2 1.3
KSOG 1.5 0.4 5.9 0.5 27.47*** 10.03
Range for preference differentials, -6.0 to 6.0. Range for‘‘Sex with Women’’and‘‘Sex With Men’’differentials, 1.0–7.0. Range for KSOG-m, 1.0–7.0
*** p\.0001

123
Arch Sex Behav (2016) 45:829–841 837

Table 2 Correlations (Pearson r) between D-IRAP scores and explicit orthogonal pattern for the IRAP was somewhat reflected in the
measures KSOG scores, in which the mean for the androphilic group was
Overall Female Male 5.9 (i.e., 1.1 away from maximum exclusivity) whereas the mean
D-IRAP score picture bias picture bias for the gynephilic group was 1.5 (i.e., 0.5 away from maximum
exclusivity). On balance, five of the six one-sample t tests for the
Measure
Likert scales were significantly different from neutral in a nega-
Good
tive direction for the androphilic group when rating ‘‘Sex with
Preference .84***
Women.’’ Thus, although the androphilic group produced self-
Sex w/women .79***
reports that suggested a lower level of exclusivity relative to the
Sex w/men .70***
gynephilic group in terms of general sexual preference, the rat-
Beautiful
ings of the androphilic group with respect to sexual attraction to
Preference .82***
women were far from neutral. How might we account for this
Sex w/women .68***
apparent divergence between the implicit and explicit measures?
Sex w/men .56**
One important factor that might have served to reduce the
Pleasant implicit female picture bias for the androphilic group to near zero
Preference .77*** is the life-long repeated media presentations of women as sexual
Sex w/women .71*** objects and as possessing great sexual appeal. For example, in
Sex w/men .55** advertising not only is the sexual appeal of women portrayed
Exciting more often than men, but also the female models used for this pur-
Preference .82*** pose tend to be more attractive, more slender, and younger than
Sex w/women .69*** males who are used for their sex appeal (Lin, 1998). Almost daily
Sex w/men .76*** exposure to this focus on females as attractive, sexual beings may
Nice thus have impacted upon the automatic responses to the female
Preference .82*** stimuli. For the gynephilic group, the portrayal of women as pri-
Sex w/women .62*** marily sexual would only serve to support those automatic
Sex w/men .68*** responses that were consistent with self-reported sexual orien-
Attractive tation. In the case of the androphilic group, however, constant
Preference .80*** exposure to females as sexual in the wider culture may influence
Sex w/women .72*** automatic responses in a manner that diverges from self-reported
Sex w/men .57** levels of attraction to the opposite sex.
KSOG .82*** .81*** .69*** Alsoofnoteisthatandrophilic menaremorelikelyto havehad
sexual experience with their non-preferred gender than gyne-
** p\.001; *** p\.0001
philic men (Layte et al., 2006). It is possible that this exposure
may have affected the implicit bias in the androphilic men in this
sample or even vice versa. However, without non-relativistic
already higher than all previously published comparisons. Criti- information on numbers of same and opposite sex partners,
cally, the level of predictive validity and correlation with explicit this hypothesis was untestable using the current data set.
measures remained high even when bias scores were calculated Of course, both explanations remain speculative, but there is
only using the implicit responses to either the male or female considerable evidence that implicit measures are sensitive to the
pictures. The current findings thus supported the conclusion impact of evaluative conditioning (Olson & Fazio, 2001, 2002)
that the sexual orientation of gynephilic and androphilic men and other iterative learning procedures (Cullen, Barnes-Holmes,
may be distinguished based not only on relative preference scores Barnes-Holmes, & Stewart, 2009; Hughes & Barnes-Holmes,
for male and female erotic stimuli, but also on scores obtained 2011). Consequently, exposure to women in a sexual context in
separately for each gender. the above-described ways may indeed impact on measures of
Although implicit responses to both types of stimuli (male and implicit sexual response biases. In any case, the fact that the IRAP
female) yielded very high levels of predictive validity, the pattern yielded an effect that diverged somewhat from the explicit ratings
ofbiasesshownforthetwotypesofstimulifortheandrophilicand serves to highlight the potential utility of employing such mea-
gynephilic men was not strictly orthogonal. Specifically, the sures in the investigation of sexual orientation.
gynephilic group showed clear attraction and aversion biases for Of key interest would be to investigate to what degree implicit
the female and male pictures, respectively, whereas the andro- sexual orientation as measured by the IRAP predicts sexual
philic group only showed the opposite pattern for the male stim- behavior and arousal, especially in the context of a discrepancy
uli. Interestingly, the female nudes produced a near neutral IRAP with explicit attitudes, such as that displayed by the androphilic
effect for the androphilic group. It is worth noting that the non- men in this study. As discussed previously, explicit and implicit

123
838 Arch Sex Behav (2016) 45:829–841

measures may tap into distinct aspects of sexual orientation. Afurtherlimitationwasthattheparticipantswereinformedthat

Identifying the conditions under which the IRAP and explicit the research involved measures of sexual orientation and were
measures predict other aspects of sexuality and which aspects asked only to volunteer if they felt they had a strong preference for
each predict could help tease out what dimensions of sexual ori- one gender over the other. As such, participants were relatively
entation these two types of measures assess. For example, if open about their sexuality and sexual matters generally and it
implicit measures predicted genital arousal to novel images and seems unlikely that any participant would have been trying to hide
opportunistic sexual interaction, but explicit measures predicted or fake his sexual orientation. The current findings do not, there-
deliberately sought interactions and long term sexual interest in fore, indicate if the IRAP could be used to tap into sexual response
particular persons, one might conclude that the IRAP taps into biases when participants are attempting to engage in dissimulation
immediate sexual arousal, whereas explicit measures tap into or lack awareness of such biases. Indeed, a true test of its utility in
desires to act on one’s arousal or arousal that may be sustained or that regard wouldrequire participantsforwhom theiractual sexual
induced at later stages of sexual interaction. An additional area of attraction patterns are inconsistent with those they explicitly report
interest would be sexual behavior in cases of neither attraction nor and ideally the results would also be compared with phallometric
aversion. Theoretically, the former should motivate sexual testing. Nevertheless, previous research has indicated that the
behavior, and the latter should deter it, but if an individual lacks IRAP is difficult to fake (McKenna et al., 2007; see also Barnes-
either response to a particular gender they could be motivated to Holmes et al., 2010a), and psychologically sensitive biases not
engage in such behaviors with members of that gender by other revealed by explicit measures have been obtained with the IRAP
factors, such as curiosity, sensation seeking and/or miscellaneous (Barnes-Holmes et al., 2010b; Dawson et al., 2009; Roddy et al.,
social benefits. 2010). Consequently, further research with the IRAP to deter-
A limitation of the current study was that participation was mine its resistance to faking (or sensitivity to unconscious biases)
restricted to gynephilic and androphilic men only. As such, the in the context of sexual preferences certainly seems worthwhile.
current findings tell us little about androphilic women, gynephilic A recent attempt to develop the IRAP as a forensic measure for
womenandbisexualindividuals.Theimplicitsexualresponsesof distinguishingthe implicit sexual responses ofchild sex offenders
women would be of particular interest, given that gynephilic from non-offenders showed moderate predictive validity (Daw-
women typically do not display a category specific response son et al., 2009). It was suggested that the accuracy of the IRAP
pattern on genital measures (Chivers et al., 2007, 2010). Since the may have been compromised by the heterogeneity of the offend-
IRAP is an objective measure, one might predict that it would ing group (Gudjonsson & Sigurdsson, 2000), and a more precise
show a similar response pattern to that exhibited by genital understanding of these differences may lead to more effective
arousal measures. However, the IRAP is also essentially a verbal means of their assessment (Robertiello & Terry, 2007). Indeed,
measure (Barnes-Holmes et al., 2010a), which suggests that the the current study’s success suggests that when sexual preference
construct it measures is likely more directly related to explicit is more clearly defined, distinguishing between the implicit sex-
self-reports than genital measures are. As such, gynephilic ual responses of groups with different sexual preferences may be
women may yield category-specific IRAP scores concordant achieved with very high levels of accuracy using the IRAP, as
withtheirreported sexualorientation.Suchafindingwouldimply well as the IAT and PT. It is worth noting, however, that other
that while gynephilic women are category-specific at a verbal or variables may play a role here. For example, words rather than
cognitive level, this is not the case in terms of genital arousal, pictures were employed with the IRAP in the Dawson et al. study,
which would in turn help explain why the majority of women and the latency criterion was set at 3000 rather than 2000 ms (the
consider themselves to be gynephilic despite what research latter was used in the current study). Pictures, and particularly
utilizing genital arousal measures seems to suggest. nudes, may elicit relatively strong sexual response biases and
Likewise, the IRAP could be used to investigate the phenome- recent research indicates that stronger and more reliable IRAP
nal nature of bisexuality. Notably, some bisexual men display a effects are produced when a shorter response latency criterion is
category-specific response pattern on genital arousal measures employed (Barnes-Holmes et al., 2010b). Numerous variables
(Rieger, Chivers, & Bailey, 2005, but see Rosenthal, Sylva, will thus require systematic analysis in the search for increas-
Safron, & Bailey, 2011). If those who exhibit this pattern on ingly accurate measures of sexual preference using the IRAP
genital measures display significant attraction biases to both and indeed other measures of implicit attitudes (see O’Ciardha
males and females on the IRAP, this would suggest that bisexu- & Gormley, 2009, 2012).
ality can exist in men at the verbal or cognitive level even when it As mentioned previously, one advantage the IRAP has over
cannot be detected using phallometry. Similarly, bisexual indi- the IAT is its ability to measure biases in a non-relative manner. It
viduals that display dual attraction biases may constitute multiple is worth noting that this disadvantage can be worked around by
subgroups, such that some may possess equally strong biases for replacing one of the target stimulus sets with a set of presumably
each gender and others may have a bias that is more pronounced neutral stimuli, thereby theoretically creating a non-relative mea-
for one gender or the other. sure of implicit biases to a single concept. This has been done

123
Arch Sex Behav (2016) 45:829–841 839

specifically with the IAT in order to measure individual implicit Additionally, and as mentioned previously, the IRAP is in
sexual responses to male and female stimuli (Snowden & Gray, essence a verbal measure (Barnes-Holmes et al., 2010a) which
2013). This was performed as a follow up to an IAT which mea- suggests that it (7) may measure a construct that is distinct from,
sured sexual responses relatively. However, this solution still has albeit related to, that tapped into by the alternative measures and
some inherent issues. Firstly, research suggests that exposure to a (8) is more directly comparable with explicit measures. Indeed,
single IAT results in vulnerability to faking in subsequent IATs, these qualities also make the IRAP a good potential candidate to
even without explicit instructions on how (Fiedler & Bluemke, utilize in tandem with genital response or other measures to
2005), which may limit this form of the IAT’s ability to index an produce more detailed and perhaps even more accurate results.
individual’s full profile of implicit sexual responses to male and As such, future research should compare the IRAP with other
female stimuli. measuresto determinewhether they measurethe sameconstructs,
Secondly, unlessanother non-relativeimplicit measure is used their comparative performance and their complementary utility.
to assess them in advance, the neutrality of the replacement stim- In conclusion, we have shown that the IRAP has a powerful
uli is somewhat of an a priori assumption, which can complicate ability to identify the sexual orientations of gynephilic and
interpretation. For example, according to the mentioned IATs androphilic men, and critically its accuracy in this regard was
utilized by Snowden andGray(2013), gynephilic malesappeared maintained when measuring separate response biases for male
to be more sexually attracted to female pictures versus male pic- and female stimuli. Indeed, these separate measurements indi-
tures, equally sexually attracted to male pictures versus neutral cated that the response biases of the two sexual orientations tar-
pictures, and more sexually attracted to female pictures versus geted here are not strictly orthogonal. This finding raises some
neutral pictures. This was interpreted as category-specific attrac- interesting questions concerning the variables responsible for the
tion to the female stimuli, as it is quite unlikely that these partici- absence of a negative bias among the androphilic men for the
pants would display sexual attraction to the neutral stimuli at a female stimuli, especially given that this group rated sex with
group level. However, it was unclear whether these men had no females negatively on the explicit measure. In any case, these
biases to both the male and neutral stimuli or aversion biases to findings provide further support for Snowden et al.’s (2008)
boththemaleandneutral stimuli,bothofwhichcouldbepossible. conclusion that implicit measurements could prove to be of
Intuitively, a neutral score implies a neutral attitude to the male considerable utility in the study of sexual orientation and sexual
stimuli on the male versus neutral IAT; however, the data from preferences.
the IRAP imply that gynephilic men may have an implicit aver-
sion bias to male stimuli. Follow up research could test this by Open Access This article is distributed under the terms of the Creative
Commons Attribution 4.0 International License (http://creativecommons.
administering the IRAP and these IATs to gynephilic male par- org/licenses/by/4.0/), which permits unrestricted use, distribution, and
ticipants, and indeed the IRAP could potentially beused to test the reproduction in any medium, provided you give appropriate credit to the
validity of the male versus neutral and female versus neutral IATs original author(s) and the source, provide a link to the Creative Commons
used by Snowden and Gray. license, and indicate if changes were made.
Finally, one might ask why the IRAP should be used in
research on sexual preferences when other established alterna-
References
tives exist, such as genital response measures, or are in develop-
ment, such as eye tracking (Rupp & Wallen, 2007), eye dilation Arkes, H. R., & Tetlock, P. E. (2004). Attributions of implicit prejudice,
(Rieger & Savin-Williams, 2012), and functional magnetic res- or ‘‘Would Jesse Jackson ‘fail’ the Implicit Association Test?’’
onance imaging (Safron et al., 2007). While it remains to be seen Psychological Inquiry, 15, 257–278.
whether any of these measures outperform the IRAP or vice versa Barnes-Holmes, D., Barnes-Holmes, Y., Power, P., Hayden, E., Milne, R., &
Stewart, I. (2006). Do you really know what you believe? Developing
the IRAP does present a number of intrinsic advantages in that it: the Implicit Relational Assessment Procedure (IRAP) as a direct mea-
(1) requires relatively little training to use, (2) does not require any sure of implicit beliefs. The Irish Psychologist, 32, 169–177.
equipment beyond a basic computer, allowing for large amounts Barnes-Holmes, D., Barnes-Holmes, Y., Stewart, I., & Boles, S. (2010a). A
of parallel participant testing, (3) does not have any additional sketch of the Implicit Relational Assessment Procedure (IRAP) and the
Relational Elaboration and Coherence (REC) model. Psychological
runningorequipmentcosts,(4)isnotphysicallyuncomfortableor Record, 60, 527–542.
invasive for participants, (5) has the potential to be developed to Barnes-Holmes, D., Murphy, A., Barnes-Holmes, Y., & Stewart, I. (2010b).
be useable with words rather than pictures, and thus is more likely The Implicit Relational Assessment Procedure: Exploring the impact
to be acceptable for use in research with minors, and (6) is cur- of private versus public contexts and the response latency criterion on
pro-white and anti-black stereotyping among white Irish individuals.
rently being developed into an online measure for remote data Psychological Record, 60, 57–66.
collection, which should allow for large scale data collection in a Barnes-Holmes, D., Murtagh, L., Barnes-Holmes, Y., & Stewart, I. (2010c).
similar manner to the IAT (Nosek, Greenwald, & Banaji, 2005). Using the Implicit Association Test and the Implicit Relational Assess-

123
840 Arch Sex Behav (2016) 45:829–841

ment Procedure to measure attitudes towards meat and vegetables in Gawronski, B., & Bodenhausen, G. V. (2006). Associative and propositional
vegetarians and meat-eaters. Psychological Record, 60, 287–306. processes in evaluation: An integrative review of implicit and explicit
Barnes-Holmes, D., Waldron, D., Barnes-Holmes, Y., & Stewart, I. (2009). attitude change. Psychological Bulletin, 132, 692–731.
Testing the validity of the Implicit Relational Assessment Procedure Greenwald, A. G., & Banaji, M. R. (1995). Implicit social cognition: Atti-
(IRAP) and the Implicit Association Test (IAT): Measuring attitudes tudes, self-esteem, and stereotypes. Psychological Review, 102, 4–27.
towards Dublin and country life in Ireland. Psychological Record, 59, Greenwald, A. G., Nosek, B. A., & Banaji, M. R. (2003). Understanding and
389–406. using the Implicit Association Test: I. An improved scoring algorithm.
Chivers, M. L., Rieger, G., Latty, E., & Bailey, J. M. (2004). A sex difference Journal of Personality and Social Psychology, 85, 197–216.
in the specificity of sexual arousal. Psychological Science, 15, 736– Gudjonsson, G. H., & Sigurdsson, J. F. (2000). Differences and similarities
744. between violent offenders and sex offenders. Child Abuse and Neglect,
Chivers, M. L., Seto, M. C., & Blanchard, R. (2007). Gender and sexual ori- 24, 363–372.
entation differences in sexual response to sexual activities versus gen- Hughes, S., & Barnes-Holmes, D. (2011). On the formation and persistence
der of actors in sexual films. Journal of Personality and Social Psy- of implicit attitudes: New evidence from the Implicit Relational
chology, 93, 1108–1121. Assessment Procedure (IRAP). Psychological Record, 61, 391–410.
Chivers,M.L.,Seto,M.C.,Lalumière,M.L.,Laan,E.,&Grimbos,T.(2010). Hughes, S., Barnes-Holmes, D., & De Houwer, J. (2011). The dominance of
Agreement of self-reported and genital measures of sexual arousal in associative theorizing in implicit attitude research: Propositional and
men and women: A meta-analysis. Archives of Sexual Behavior, 39, 5– behavioral alternatives. Psychological Record, 61, 465–496.
56. Israel, E., & Strassberg, D. S. (2009). Viewing time as an objective measure
Cullen, C., Barnes-Holmes, D., Barnes-Holmes, Y., & Stewart, I. (2009). of sexual interest in heterosexual men and women. Archives of Sexual
The Implicit Relational Assessment Procedure (IRAP) and the mal- Behavior, 38, 551–558.
leability of ageist attitudes. Psychological Record, 59, 591–620. Klein, F. (1993). The bisexual option. New York: Hawthorn Press.
Dawson, D. L., Barnes-Holmes, D., Gresswell, D. M., Hart, A. J., & Gore, N. Klein, F., Sepekoff, B., & Wolf, T. J. (1985). Sexual orientation: A multi-
J. (2009). Assessing the implicit beliefs of sexual offenders using the variable dynamic process. Journal of Homosexuality, 11, 35–49.
Implicit Relational Assessment Procedure. Sexual Abuse, 21, 57–75. Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (1997). International affective
De Houwer, J. (2003). The extrinsic affective Simon task. Experimental picture system (IAPS): Affective ratings of pictures and instruction
Psychology, 50, 77–85. manual. Technical Report A-6. Gainesville: University of Florida.
De Houwer, J. (2006). What are implicit measures and why are we using Layte, R., McGee, H., Quail, A., Rundle, K., Cousins, G., Donnolly, C., et al.
them? In R. Wiers & A. Stacy (Eds.), Handbook of implicit cognition (2006). The Irish study of sexual health and relationships. Dublin:
and addiction (pp. 11–28). Thousand Oaks, CA: Sage. Crisis Pregnancy Agency and Department of Health and Children.
De Houwer, J., Teige-Mocigemba, S., Spruyt, A., & Moors, A. (2009). Lin, C. A. (1998). Use of sex appeals in prime-time television commercials.
Implicit measures: A normative analysis and review. Psychological Sex Roles, 38, 461–475.
Bulletin, 135, 347–368. McConnell, A. R., & Leibold, J. M. (2001). Relations among the Implicit
Dovidio,J.F.,Kawakami,K.,Johnson,C.,Johnson,B.,&Howard,A.(1997). Association Test,discriminatory behavior, and explicit measuresofracial
Onthe natureofprejudice:Automatic andcontrolled processes. Journal attitudes. Journal of Experimental Social Psychology, 37, 435–442.
of Experimental Social Psychology, 33, 510–540. McKenna, I. M., Barnes-Holmes, D., Barnes-Holmes, Y., & Stewart, I.
Dovidio, J. F., Kawakami, K., Smoak, N., & Gaertner, S. L. (2009). The roles (2007). Testing the fake-ability of the Implicit Relational Assessment
of implicit and explicit processes in contemporary prejudice. In R. E. Procedure (IRAP): Thefirststudy.International Journalof Psychology
Petty, R. H. Fazio, & P. Brinol (Eds.), Attitudes: Insights from the new and Psychological Therapy, 7, 123–138.
implicit measures (pp. 165–192). New York: Psychology Press. Nosek, B. A. (2005). Moderators of the relationship between implicit and
Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition explicit evaluation. Journal of Experimental Psychology, 134, 565–
Letters, 27, 861–874. 584.
Fazio, R. H., & Olson, M. A. (2003). Implicit measures in social cognition Nosek, B. A. (2007). Implicit-explicit relations. Current Directions in
research: Their meaning and use. Annual Review of Psychology, 54, Psychological Science, 16, 65–69.
297–327. Nosek, B. A., & Banaji, M. R. (2001). The Go/No-Go Association task.
Fiedler, K., & Bluemke, M. (2005). Faking the IAT: Aided and unaided Social Cognition, 19, 625–666.
response control on the Implicit Association Tests. Basic and Applied Nosek, B. A., Greenwald, A. G., & Banaji, M. R. (2005). Understanding and
Social Psychology, 27, 307–316. using the Implicit Association Test: II. Method variables and construct
Freund, K., Langevin, R., Chamberlayne, R., Deosoran, A., & Zajac, Y. validity. Personality and Social Psychology Bulletin, 31, 166–180.
(1974a). The phobic theory of male homosexuality. Archives of O’Ciardha, C., & Gormley, M. (2009). Comparing two implicit cognitive
General Psychiatry, 31, 495–499. measures of sexual interest: A pictorial Stroop task and the Implicit
Freund, K., Langevin, R., Cibiri, S., & Zajac, Y. (1973). Heterosexual Association Test. In D. Thompson & D. R. Laws (Eds.), Cognitive
aversion in homosexual males. British Journal of Psychiatry, 122, approaches to the assessment of sexual interest in sexual offenders
163–169. (pp. 177–202). Chichester, UK: Wiley.
Freund, K., Langevin, R., & Zajac, Y. (1974b). Heterosexual aversion in O’Ciardha, C., & Gormley, M. (2012). Using a pictorial-modified Stroop
homosexual males: A second experiment. British Journal of Psychi- task to explore the sexual interests of sexual offenders against children.
atry, 125, 177–180. Sexual Abuse, 24, 175–197.
Friese, M., Hofmann, W., & Wänke, M. (2008). When impulses take over: Olson, M. A., & Fazio, R. H. (2001). Implicit attitude formation through
Moderated predictive validity of explicit and implicit attitude measures classical conditioning. Psychological Science, 12, 413–417.
in predicting food choice and consumption behavior. British Journal of Olson, M. A., & Fazio, R. H. (2002). Implicit acquisition and manifestation
Social Psychology, 46, 397–419. of classically conditioned attitudes. Social Cognition, 20, 89–104.
Galdi, S., Arcuri, L., & Gawronski, B. (2008). Automatic mental associ- Power, P., Barnes-Holmes, D., Barnes-Holmes, Y., & Stewart, I. (2009).
ations predict future choices of undecided decision makers. Science, The Implicit Relational Assessment Procedure (IRAP) as a measure
321, 1100–1102. of implicit relative preferences: A first study. Psychological Record,
Gawronski, B. (2009). Ten frequently asked questions about implicit mea- 59, 621–640.
sures and their frequently supposed, but not entirely correct answers. Rieger, G., Chivers, M. L., & Bailey, J. M. (2005). Sexual arousal patterns of
Canadian Psychology, 50, 141–150. bisexual men. Psychological Science, 16, 579–584.

123
Arch Sex Behav (2016) 45:829–841 841

Rieger, G., & Savin-Williams, R. C. (2012). The eyes have it: Sex and sexual Snowden, R. J., & Gray, N. S. (2013). Implicit sexual associations in
orientation differences in pupil dilation patterns. PLoS One, 7, e40256. heterosexual and homosexual women and men. Archives of Sexual
Robertiello, G., & Terry, K. J. (2007). Can we profile sex offenders? A review Behavior, 42, 475–485.
of sex offender typologies. Aggression and Violent Behavior, 12, 508– Snowden, R. J., Wichter, J., & Gray, N. S. (2008). Implicit and explicit
518. measurement of sexual preference in gay and heterosexual men: A
Roddy, S., Stewart, I., & Barnes-Holmes, D. (2010). Anti-fat, pro-slim, or comparison of priming techniques and the Implicit Association Task.
both? Using two reaction time based measures to assess implicit atti- Archives of Sexual Behavior, 37, 558–565.
tudes to the slim and overweight. Journal of Health Psychology, 15, Timko, C. A., England, E. I., Herbert, J. D., & Forman, E. M. (2010). The
416–425. Implicit Relational Assessment Procedure as a measure of self-esteem.
Rosenthal, A. M., Sylva, D., Safron, A., & Bailey, J. M. (2011). Sexual Psychological Record, 60, 679–698.
arousal patterns of bisexual men revisited. Biological Psychology, Vahey, N. A., Barnes-Holmes, D., Barnes-Holmes, Y., & Stewart, I. (2009).
88, 112–115. A first test of the Implicit Relational Assessment Procedure (IRAP) as a
Rullo, J. E., Strassberg, D. S., & Israel, E. (2010). Category-specificity in measure of self-esteem: Irish prisoner groups and university students.
sexual interest in gay men and lesbians. Archives of Sexual Behavior, Psychological Record, 59, 371–388.
39, 874–879. Weinrich, J. D., Snyder, A. J., Pillard, R. C., Grant, I., Jacobson, D. L.,
Rupp, H. A., & Wallen, K. (2007). Sex differences in viewing sexual stimuli: Robinson, S. R., & McCutchan, J. A. (1993). A factor analysis of the
An eye-tracking study in men and women. Hormones and Behavior, Klein Sexual Orientation Grid in two disparate samples. Archives
51, 524–533. of Sexual Behavior, 22, 157–168.
Rydell, R. J., & McConnell, A. R. (2006). Understanding implicit and Wilson, T. D., Lindsey, S., & Schooler, T. Y. (2000). A model of dual atti-
explicit attitude change: A systems of reasoning analysis. Journal tudes. Psychological Review, 107, 101–126.
of Personality and Social Psychology, 91, 995–1008.
Safron, A., Barch, B., Bailey, J. M., Gitelman, D. R., Parrish, T. B., & Reber,
P. J. (2007). Neural correlates of sexual arousal in homosexual and
heterosexual men. Behavioral Neuroscience, 121, 237–248.

123