International Journal of Drug Policy 23 (2012) 54–61
Contents lists available at ScienceDirect
International Journal of Drug Policy
journal homepage: www.elsevier.com/locate/drugpo
Research methods
Reconciling incongruous qualitative and quantitative findings in mixed methods
research: Exemplars from research with drug using populations
Karla D. Wagner a,∗ , Peter J. Davidson a , Robin A. Pollini a , Steffanie A. Strathdee a , Rachel Washburn b ,
Lawrence A. Palinkas c
a
Division of Global Public Health, Department of Medicine, University of California San Diego, 9500 Gilman Drive MC 0507, La Jolla, CA 92093-0507, USA
Department of Sociology, Loyola Marymount University, One LMU Drive, Suite 4314, Los Angeles, CA 90045-2659, USA
c
School of Social Work, University of Southern California, Montgomery Ross Fisher Building, Los Angeles, CA 90089-0411, USA
b
a r t i c l e
i n f o
Article history:
Received 23 November 2010
Received in revised form 11 May 2011
Accepted 14 May 2011
Keywords:
Qualitative methods
Mixed methods research
Injection drug use
a b s t r a c t
Mixed methods research is increasingly being promoted in the health sciences as a way to gain more
comprehensive understandings of how social processes and individual behaviours shape human health.
Mixed methods research most commonly combines qualitative and quantitative data collection and analysis strategies. Often, integrating findings from multiple methods is assumed to confirm or validate the
findings from one method with the findings from another, seeking convergence or agreement between
methods. Cases in which findings from different methods are congruous are generally thought of as
ideal, whilst conflicting findings may, at first glance, appear problematic. However, the latter situation
provides the opportunity for a process through which apparently discordant results are reconciled, potentially leading to new emergent understandings of complex social phenomena. This paper presents three
case studies drawn from the authors’ research on HIV risk amongst injection drug users in which mixed
methods studies yielded apparently discrepant results. We use these case studies (involving injection
drug users [IDUs] using a Needle/Syringe Exchange Program in Los Angeles, CA, USA; IDUs seeking to
purchase needle/syringes at pharmacies in Tijuana, Mexico; and young street-based IDUs in San Francisco, CA, USA) to identify challenges associated with integrating findings from mixed methods projects,
summarize lessons learned, and make recommendations for how to more successfully anticipate and
manage the integration of findings. Despite the challenges inherent in reconciling apparently conflicting
findings from qualitative and quantitative approaches, in keeping with others who have argued in favour
of integrating mixed methods findings, we contend that such an undertaking has the potential to yield
benefits that emerge only through the struggle to reconcile discrepant results and may provide a sum
that is greater than the individual qualitative and quantitative parts.
© 2011 Elsevier B.V. All rights reserved.
Introduction
Mixed methods research is a re-emerging area of interest in
the field of drug use research, and has received attention across
multiple disciplines, including anthropology, epidemiology and
sociology (Bourgois, 2002; Moss, 2003; Rhodes, 2009). Whilst
there are increasing calls for and advocates of mixed methods
research designs, combining qualitative and quantitative strategies
can present a number of distinct philosophical and logistical challenges, including whether and how to integrate findings that may
initially appear to conflict. Methodological scholars have engaged
with the process of how and when to combine findings from qualitative and quantitative research (for reviews see Denzin, 2010;
∗ Corresponding author. Tel.: +1 619 543 0857; fax: +1 858 534 7566.
E-mail address: kdwagner@ucsd.edu (K.D. Wagner).
0955-3959/$ – see front matter © 2011 Elsevier B.V. All rights reserved.
doi:10.1016/j.drugpo.2011.05.009
Kelle, 2001). In its most recent iteration, termed the “third methodological moment” (Denzin, 2010, p. 422), researchers conducting
mixed methods research have revisited the discussion, and some
note a resurgence of interest in integration in the design, analysis,
and interpretation of mixed methods studies (e.g. Bazeley, 2009).
Some have noted that the process of integration, particularly integration of apparently discrepant data, can itself be a productive
enterprise (Fielding, 2009); though the success of such integration efforts may depend on the perspective with which researchers
approach the discrepancies.
At first glance, integration is often thought of in terms of seeking consistency between qualitative and qualitative findings. In
one commonly used form, integration through the “triangulation”
of findings from multiple methods implies a search for increased
validity; the use of multiple methods producing similar results
suggests that the phenomenon of interest has been more accurately measured than it could have been using a single method
K.D. Wagner et al. / International Journal of Drug Policy 23 (2012) 54–61
(Moran-Ellis et al., 2006). In the social sciences, triangulation has
its roots in the classic multitrait-multimethod matrices proposed
by Campbell and Fiske (1959) as a means of establishing convergent validity of psychological scales. Though ultimately favouring
the classic conceptualisation of triangulation in the trigonometrical
sense, Kelle (2001) also offers a third interpretation of triangulation: triangulation as a means of integrating multiple perspectives
(complementarity). From a more postmodern perspective, the use
of multiple methods can been seen to extend the “scope and depth
of understanding” (Fielding, 2009, p. 429), and can provide a richer
understanding of complex social issues.
In the fields of drug use and HIV prevention research, scholars
have increasingly employed mixed methods designs to investigate
the complex environmental, social, and individual factors that are
associated with negative health outcomes (Bourgois et al., 2006;
Clatts, Welle, Goldsamt, & Lankenau, 2002; Pollini et al., 2010;
Torrone et al., 2010), though the complementary use of epidemiology and ethnography in the study of drug use is not new (Agar,
1997; Clatts, Southeran, Heimer, & Goldsamt, 1999). In the process, some have noted difficulties in integrating findings from
mixed methods research. Critical to this discourse is attention to
the assumptions that underlie the methods that researchers are
attempting to combine. Whilst differences between qualitative and
quantitative approaches to complex social situations can often
be reduced to differences in data collection processes and analytic methodologies, these superficial differences can mask more
fundamental epistemological differences – quantitative methods
emphasize deductive approaches that test a priori hypotheses;
qualitative methodologies emphasize inductive approaches that
“stress how social experience is created and given meaning”
(Denzin & Lincoln, 2000, p. 8, emphasis in the original). For example, in complementary articles Bourgois (2002) and Moss (2003)
articulate some of the challenges experienced in the context of
a large mixed methods study of disease transmission amongst
young injection drug users, which employed participant observation, ethnographic interviews, and epidemiological surveys. These
included reconciling differences between clinical and statistical
significance, discipline-specific constructions of scientific validity
and reliability, and different levels of disciplinary focus (i.e., ethnographic focus on macro-level social structures vs. epidemiological
focus on individual-level behaviour).
In this paper, we extend this discussion and argue that the
process of engaging with divergent results can yield a more comprehensive and nuanced understanding of the situation under
study, offering multiple viewpoints and encouraging a “dialogue
between differently committed minds” (Fielding, 2009, p. 436).
In an effort to engage critically and reflexively with the process
of doing mixed methods research, we present three case studies
from our own research focused on the association between injection drug use and HIV infection as exemplars (for more details
we refer the reader to the original publications: Davidson, 2009;
Pollini et al., 2010; Wagner et al., 2010; Wagner et al., 2011). The
projects differ in their specific research questions, research designs
and in their methods of attempting to reconcile conflicting findings.
The common theme, however, is that the work of seeking to integrate initially conflicting findings yielded rich insights into both the
nature of the phenomena and of the methods used to investigate
them. We provide a descriptive account of each case study, briefly
summarising the study design and implementation, analysis and
results; and discuss the efforts to reconcile apparently incongruous results. We conclude with a discussion of the theoretical and
methodological lessons learned from implementing these mixed
methods projects, and with recommendations for how investigators engaged in research with drug using populations may more
successfully anticipate and manage the integration of findings from
multiple methods.
55
Case Study 1: the role of “dopesickness” in influencing risky
injection behaviour amongst IDUs
Background
Current public health recommendations hold that using a new
needle/syringe for every injection event is necessary to avoid the
spread of HIV and other blood-borne viruses amongst IDUs (Gayle,
O’Neill, Gust, & Mata, 1997). Considerable research has examined
the role of the perceived risks associated with sharing injection equipment in influencing needle/syringe sharing behaviour.
However, less research has been dedicated to the converse – understanding the influence of the perceived risk of refusing to share
injection equipment. That is, the consequences that IDUs associate
with engaging in “safer” behaviour than can reduce the risk for HIV
infection. Epidemiological and ethnographic studies have identified some factors that might be identified as potential consequences
of refusing to share needle/syringes, such as the threat of becoming “dopesick” (i.e., experiencing withdrawal symptoms associated
with the delayed use of opioids), risking citation or arrest for possession of drug injection paraphernalia or violating social norms
associated with sharing. This study was designed to qualitatively
identify the consequences that IDUs associate with refusing to
share injection equipment, and to examine the quantitative associations amongst those perceived consequences, other psychosocial
correlates of injection risk behaviour and needle/syringe and paraphernalia sharing.
Method
This study was designed as a two-phase, sequential mixed methods study, in which the exploratory qualitative phase provided data
that were used to inform the development of items for a quantitative questionnaire. In Phase I, semi-structured interviews were
conducted with 26 IDUs recruited from a single Needle/Syringe
Exchange Program (SEP) in Los Angeles, CA, USA. The interviews
focused on the most recent injection event in which the participant shared injection equipment, and participants were asked to
describe the potential consequences to them if they had refused to
share equipment at that time. Interviews were digitally recorded
and transcribed in their entirety. A theoretical framework that provided some a priori thematic codes derived from the ethnographic
and epidemiological literature guided the qualitative analysis, and
emergent codes were added to the codebook throughout the analysis.
For the second phase of the study, 187 IDUs were recruited
from the same SEP to participate in a structured interview using
Audio Computer Assisted Self Interview (ACASI). Qualitative data
were distilled into 11 possible consequences of refusing to share,
and in the quantitative survey participants were asked how frequently each of the 11 consequences “influenced their decision
whether or not to share injection equipment in the past 30 days”.
The consequences were weighted by their importance (e.g. “how
much of a problem would [that consequence] be for you?”), since
more severe consequences may be more salient in their influence
(Fishbein, 1967). The perceived consequence and importance questions were included in the quantitative survey, along with measures
of other psychosocial constructs commonly used to explain injection risk behaviour (e.g., perceived risk of HIV or HCV, self-efficacy
for safer injection, response efficacy, perceived severity of HIV or
HCV, knowledge and peer norms for safer injection).
The aim of the quantitative analysis was to determine
whether the perceived consequences of refusing to share injection equipment were associated with injection risk behaviour when
controlling for other theoretically relevant psychosocial correlates.
To investigate this question, we first created two sub-scales cre-
56
K.D. Wagner et al. / International Journal of Drug Policy 23 (2012) 54–61
ated from the perceived consequences items using exploratory
factor analysis, which largely conformed to the theory that emerged
from the qualitative interviews: social/internal consequences
and structural/external consequences. We then employed logistic
regression analysis in which two dependent variables (receptive
needle/syringe sharing and receptive paraphernalia sharing) were
regressed on the psychosocial correlates and the perceived consequences sub-scales.
Results
In the qualitative narratives, almost all participants described
risky injection events in which they were already experiencing, or
were trying to avoid, the intense drug withdrawal symptoms (i.e.
“dopesickness”) that results from delayed use of heroin amongst
heroin-dependent injectors. They described events in which, if they
did not use the available injection equipment, they would most
certainly become dopesick, and therefore said they injected with
shared equipment in an attempt to avoid it. However, this key finding from the qualitative analysis – that risky injection events were
overwhelmingly contextualized by an omnipresent risk of experiencing dopesickness – was less clearly borne out by the quantitative
data. In the quantitative analyses, the perceived consequence of
becoming “dopesick” yielded slightly inconsistent results in the
exploratory factor analysis, showing moderate double loadings on
both sub-scales. Due to the salience of this particular consequence
in the qualitative narratives, this finding inspired additional quantitative analyses in which we removed the item from the perceived
consequences scales and evaluated its significance as an independent covariate. Results from these supplemental analyses showed
that, on its own, “dopesickness” was not associated with either
receptive or distributive sharing, and removing it from the subscales did not substantially change the findings regarding the main
effects of the perceived consequences on injection risk behaviour.
Integration challenges and lessons learned
Rather than use this discrepancy between the quantitative and
qualitative findings as a rationale for discrediting the results of
either analysis, we attempted to understand the circumstances
under which such a discrepancy could arise, and its implications for
future research of this type, in which qualitative data are collected
in order to inform the design of a quantitative survey. We suggest
two potential frameworks for understanding this discrepancy: the
first rooted in an understanding of the politics of representation
and social roles and biases inherent in the conduct of public health
research amongst drug users, and the second grounded in a study
design perspective that emphasizes a reciprocal, rather than linear,
relationship between quantitative and qualitative data.
This particular case study highlights issues related to the politics
of representation that result from the interaction between public
health researchers and IDUs. Though the interviewer/author had
an ongoing collaborative relationship with the SEP where the data
collection took place and had interacted with many of the interview
participants for over two years at the time that the interviews took
place, she was not a member of the target population. Also, since the
interviews occurred at the SEP, the researcher was often identified
by participants as part of the SEP staff (even though she emphasized
that she was not employed by the SEP). The findings from the qualitative portion of the project, then, must be understood within this
context. It is possible that the emphasis on the role of dopesickness
was offered as a way to avert anticipated blame from the researcher
for engaging in behaviour that both parties understood to confer a
disease transmission risk. Attributing their risky behaviour to the
threat of drug withdrawal, then, could represent an attempt by
interview participants to avoid violating the shared expectations
of the participant and the SEP regarding acceptable behaviour. In
the quantitative interviews, on the other hand, participants had
the freedom to enter their responses to questions about the importance of dopesickness in influencing their behaviour directly into
the computer without having to explain or justify their actions the
research team. Thus, one explanation for the observed discrepancy
between the relative importance of dopesickness in the two phases
of the study may have to do with the social context of describing
risky injection events to a researcher. In short, this finding could
have been influenced by social desirability bias (a risk faced in all
behavioural research relying on respondent self-report). Whilst the
absence of statistical correlation in the quantitative phase neither
confirms nor refutes the presence of social desirability bias, it does
highlight the need for sensitivity to and discussion of its potential
effects. One contribution of the mixed methods approach, in this
case, is its ability to highlight the need for a reflexive examination
of identity politics and the influence of the social-spatial context in
which research occurs (Bourgois, 2002).
The discrepancies highlighted in this case study also point
towards another contribution of the mixed methods approach –
the ability of a mixed methods design to create a reciprocal dialogue between quantitative and qualitative data, such that findings
from one method inform the development and analysis of the
next, which can then be used to refine subsequent phases of the
investigation. This study was designed in two phases, in which the
qualitative results were used to inform the development of a quantitative scale. Unexpectedly, the initial lack of agreement between
the qualitative and quantitative results on the importance of the
influence of “dopesickness” inspired the investigators to pursue
additional quantitative analyses that would have remained unexplored in the absence of the compelling qualitative results. Though
not feasible under the funding mechanism and timeline for the current study, a more comprehensive use of a mixed methods design
could capitalise on the apparently discrepant results to refine the
data collection instruments based on lessons learned through the
attempt to reconcile the results. For example, it is possible that the
quantitative questions did not accurately measure the same construct that was discussed in the qualitative interviews, or that the
difference in referent timeframes in the two phases influenced the
results (i.e. the most recent risky injection event described in the
qualitative interviews versus quantitative questions that asked participants to think more generally about behaviour over the past 30
days). Simply piloting the quantitative interviews (which was also
done in this study) would not have identified this issue. Additional
data collection in the form of cognitive interviews to understand
how participants interpret the quantitative questions or “memberchecking” interviews or focus groups to discuss with participants
whether the findings from the research resonate with their experience, could shed some light onto the validity of the findings. In the
absence of the mixed methods design that explicitly attempted to
integrate the findings from the two phases, however, the discrepancies may not have been discovered and the lessons for future
study designs, data collection instruments and analysis strategies
would have been lost.
Case Study 2: barriers to needle/syringe purchase at retail
pharmacies in Tijuana, Mexico
Background
In Mexico, it is legal to purchase and possess sterile needle/syringes without a prescription. However, findings from
qualitative research with IDUs in the north-western border city of
Tijuana, Baja California, Mexico, suggested that some pharmacies
refuse to sell or charge higher prices to persons suspected of being
K.D. Wagner et al. / International Journal of Drug Policy 23 (2012) 54–61
IDUs (Strathdee et al., 2005). In prior qualitative interviews, IDUs
directly linked these pharmacy-based barriers to needle/syringe
access to the practice of needle/syringe sharing, thus increasing
their risk for HIV infection. To understand more completely the context of pharmacy sales in Tijuana, we undertook a mixed methods
study designed to (1) quantify the frequency of refusals and overcharges reported by IDUs attempting to purchase needle/syringes
from pharmacies in Tijuana, (2) identify correlates of experiencing
these barriers to needle/syringe access and (3) provide a comprehensive understanding of these correlates within the context of
IDUs’ experiences.
Method
A detailed account of the study methods and results has been
published elsewhere (Pollini et al., 2010). Briefly, the parent study
(Proyecto El Cuete Phase III) recruited 1056 IDUs via respondent driven sampling (RDS; Heckathorn, 1997) between 2006
and 2007 into a longitudinal study of behavioural and contextual risk factors for HIV, syphilis and TB infection (Strathdee et al.,
2008). Participants completed an interviewer-administered survey
at baseline and every six months thereafter that included questions
on demographics, drug use behaviours, needle/syringe acquisition
and use and experiences with police. At the second study visit,
participants were asked whether they had attempted to purchase
needle/syringes at a pharmacy in the past six months and, if so, to
provide information on the outcome of the most recent purchase
attempt. IDUs who reported being overcharged and/or refused
were defined as having experienced barriers to pharmacy-based
purchase and were compared to those who did not experience barriers using univariable and multiple logistic regression methods
with RDS adjustment.
Between October 2008 and March 2009 we recruited a subset
of 47 IDUs from the parent study to participate in focus groups
designed to provide detailed accounts of experiences related to
pharmacy-based needle/syringe purchase. We conducted seven
focus groups, of which five consisted entirely of males and two
entirely of females. The focus groups were audio recorded, transcribed verbatim, translated into English by a certified translator
and then double-coded using a descriptive and thematic approach
rooted in Grounded Theory (Glaser & Strauss, 1967; Willms et al.,
1990). An explanatory design was then used to explain or build
upon the quantitative results (Creswell & Plano Clark, 2006); that
is, we used themes from the focus groups to elucidate why certain characteristics were associated with experiencing barriers to
needle/syringe purchase.
Results
Overall, 649 IDUs completed the visit 2 survey and 627 (96.6%)
responded to the question on pharmacy needle/syringe purchase.
Most (80.7%) reported purchasing a needle/syringe in a pharmacy
in the past 6 months and 71.6% of all participants cited pharmacies
as their primary source of needle/syringes. Only 16.0% experienced
barriers to pharmacy-based needle/syringe purchase, including
being refused purchase, being overcharged or both. Factors independently associated with experiencing barriers to needle/syringe
purchase were being homeless, engaging in receptive needle/syringe sharing, reporting an average of more than five uses per
needle/syringe and having a greater number of lifetime abscesses.
These quantitative findings directly contradicted earlier qualitative interviews by our research team in which pharmacy-based
barriers to needle/syringe access emerged as a key theme
(Strathdee et al., 2005). We investigated this discrepancy in the
seven focus groups by inviting participating IDUs to help us explain
these discordant findings. Participants described a scenario in
57
which pharmacies are reluctant to sell needle/syringes to persons
suspected of illicit drug use, reportedly because of fear of police
(despite the fact that needle/syringes can be legally sold and possessed without a prescription), fear of IDU theft and violence and
overall concerns about the impact IDU customers would have on
the “image” of their business. They explained that IDUs adapt to this
restrictive purchasing environment by working to develop a personal relationship with staff at a single pharmacy; by building this
relationship the IDU becomes “known” to the staff as someone who
will not cause trouble for them, thus increasing their willingness to
sell needle/syringes. This pharmacy becomes the primary source of
needle/syringes for that IDU, with purchases attempted elsewhere
only when that pharmacy is not available (e.g., after hours, immediately after release from jail when withdrawal requires immediate
injection). It is in these latter cases that IDUs are most likely to
encounter refusals or overcharging.
Integration challenges and lessons learned
Lessons learned from this case study again point to in part to
a methodological issue that would not have been identified in a
single-method study – our discrepant findings were likely due in
part to how we worded the questions in Phases I and III of the
parent study. In the Phase I qualitative study we asked “Will most
pharmacies sell syringes to drug users in this city?” followed by
probes designed to ascertain the nature of any barriers cited (e.g.
requiring a prescription). In doing so, we asked IDUs to describe
needle/syringe access at the community level. In contrast, the Phase
III quantitative study asked: “In the last 6 months, did you purchase
a syringe from a pharmacy in Tijuana?” thus focusing on access
at the individual level. Whilst the wording of these questions may
not have been optimal, it is often the case with multi-year studies
that question wording and emphasis changes. In this case, using a
mixed methods approach that included a third type of data collection – focus groups in which participants offered their opinions of
and explanations for the data from the contradictory qualitative
and quantitative data – allowed us to provide another perspective that enhanced our understanding of the IDU risk environment.
Specifically, the “member checking” focus groups revealed the
adaptive process of becoming “known” to pharmacies, which would
have remained unobserved in the absence of the discussions about
the apparently contradictory results. Relying on the qualitative or
quantitative interviews alone would have resulted in a highly distorted view of the needle/syringe access situation in Tijuana.
This case study also points to the value of conducting mixed
methods studies that employ a sequential design that explicitly
allowed for an ongoing dialogue between the quantitative and
qualitative phases. The subsequent focus groups afforded us an
opportunity to examine the reasons for our discrepant findings
from El Cuete using input from the participants themselves. We
specifically asked them to identify reasons for the discrepant findings, and what emerged was a very richly detailed account of a
dynamic adaptive process designed to negotiate needle/syringe
access in a highly restrictive and complex environment. This level
of detail would not have been obtained had the previously identified discrepancy not been used as the starting point for our focus
group discussions.
Case Study 3: young injectors, policing and SEP access in San
Francisco, CA, USA
Background
Currently San Francisco, CA, USA has fifteen legal SEP sites,
eleven run by a city agency and the remainder by non-profit organi-
58
K.D. Wagner et al. / International Journal of Drug Policy 23 (2012) 54–61
sations receiving most of their funding from the city (NASEN, 2010;
SAFAF, 2010). In 2005, the city took advantage of a change in California State law to formally legalise the possession of needle/syringes
without a prescription. However, a considerable body of literature
describes social and logistical barriers to accessing sterile injecting
equipment, even in jurisdictions such as San Francisco that are wellserved by SEPs and where the legal consequences of needle/syringe
possession have been minimised. These barriers include fear of
police, fear of being identified as a drug user and limited opening hours and/or service location (Bluthenthal, Kral, Lorvick, &
Watters, 1997; Case, Meehan, & Jones, 1998; Gostin, 1998; Heimer,
Bluthenthal, Singer, & Khoshnood, 1996; Rich, Strong, Towe, &
McKenzie, 1999). Whilst “fear of police” is frequently cited in studies of barriers to utilisation of SEPs, little work has been done
to explore this finding in more detail in order to examine which
aspects of interaction with the judicial system are particularly
likely to influence access to SEPs and injecting equipment. One
factor that has not been sufficiently examined is how mobility patterns amongst young, homeless IDUs may mediate the association
between contact with law enforcement and access to services. We
hypothesized that young IDUs’ experiences with law enforcement
would influence their comfort travelling to various parts of the city,
particularly those neighbourhoods where HIV prevention services
are located (e.g. SEPs), and that would in turn influence their access
to injecting equipment via SEPs.
Method
Simultaneous cross-sectional quantitative (N = 455) and qualitative interviews (N = 17 interviews with 16 individuals) were
conducted as a sub-study within a larger epidemiological study
of HCV transmission amongst young (<30 year old) IDUs in San
Francisco. Quantitative interviews assessed two sets of measures:
current judicial status and injection risk behaviour. Variables
assessing current judicial status included: whether someone had
warrants out for their arrest, was on probation or parole, had been
stopped by police for any purpose in the past three months or
had been incarcerated overnight or longer within the past three
months. Lifetime history of incarceration (ever incarcerated, and
if so for how long in total; ever been on probation or parole) was
also recorded. To assess injection risk behaviour, we created a Safer
Injection Indicator (SII), which measured how closely each individual approximated the U.S. Public Health Service standard of using
one new needle/syringe for every injection (Gayle et al., 1997) in
the last 30 days. The SII was constructed by taking the number of
needle/syringes acquired in the last 30 days from any source and
subtracting the number of self-reported injection events in the last
30 days.
Qualitative interviews were semi-structured and carried out at
the field sites of the parent study with a sub-sample of individuals
who were or had been in the parent study and who were present
at the parent study field site. Qualitative interviews addressed the
possible relationships between engagement with the judicial system and access to SEPs, by more broadly exploring how and where
respondents moved through the city; what made them go to or
avoid certain areas; what role their interactions with the judicial
system and police had in influencing these movements and through
exploration of how these patterns of movement through the city
and their relationships with the judicial system affected their ability
to visit SEP sites.
Qualitative transcripts were coded and thematically analysed based on Grounded Theory/Situational Analysis approaches
(Clarke, 2003; Glaser & Strauss, 1967; Strauss & Corbin, 1998). For
quantitative data, the Mann–Whitney U-test was used to compare
differences between the median SII values grouped by judicial sta-
tus measures. For non-parametric continuous variables, variables
were categorised and compared using a chi-square test.
Results
In the quantitative analysis, those who had been stopped by
police within the past three months were significantly less likely
than those who had not been stopped to report obtaining enough
needle/syringes to meet their injecting needs. No other measure
of judicial status had a statistically significant association with SII
in either direction. These data suggest a direct negative association between frequent contact with police and access to sufficient
needle/syringes to meet one’s needs.
However, in the qualitative interviews, it appeared that there
was very little association between being stopped by police and
IDUs’ ability to access needle/syringes; participants did not identify
contact with police as a barrier to needle/syringe access. Instead,
participants in the qualitative interviews described a situation
in which contact with police influenced where and how they
moved around the city (i.e. an association between police contact and mobility), an association that itself was largely influenced
by participants’ strategies for income generation. Those who used
techniques such as panhandling (i.e. begging), for example, were
used to endemic but “trivial” contact with police, and expressed relatively little concern about police contact. In contrast, those who
shoplifted or sold drugs usually had less contact with police but
the contact that did occur was of more substantial legal significance and hence of higher concern. The combination of economic
necessity and relative attitudes towards contact with police shaped
where people spent time and how comfortable they were in those
areas – a panhandler would, out of necessity, spend a lot of time
in parts of the city with high levels of foot traffic; would be very
“visible” in those places; would be relatively unconcerned that she
was “known” to police in the area and would subsequently have
relatively few barriers to accessing services such as needle/syringe
exchange in those parts of the city. By contrast, a shoplifter would
spend relatively brief periods of time in parts of the city with
specific types of retail stores; would attempt to be “invisible” in
those places; would be highly concerned about becoming “known”
to police anywhere in the city and would have relatively strong
incentives to avoid going to commercial districts other than when
shoplifting due to fear of recognition by merchants or police.
Most SEPs in San Francisco are deliberately located in parts of
the city with a visible population of IDUs – i.e. in commercial areas
with high levels of foot traffic where panhandling is common. As
such, our qualitative data described a situation where frequency
of contact with police did not in itself appear to affect people’s
willingness and ability to visit SEPs. Rather, willingness and ability to visit parts of the city where SEPs were located was shaped by
how someone earned money, and the consequences of that method
of income generation on the nature of their relations with police.
Through integrating the findings from the two methods – one that
found an association between police contact and needle/syringe
access and the other that found an association between police contact and mobility/income generation – we were able to arrive at a
more complete understand of the factors and underlying processes
associated with needle/syringe access.
Integration challenges and lessons learned
The apparent discrepancies between quantitative and qualitative results (that frequent contact with police was statistically
associated with less access to needle/syringes, but qualitatively
the association was less apparent and was mediated by income
generation strategies and consequent physical movement around
the city) are a reminder of the basic truism that correlation is not
K.D. Wagner et al. / International Journal of Drug Policy 23 (2012) 54–61
causation. Unmeasured in the quantitative data, identification of
this third variable through the qualitative interviews helped add
complexity and nuance to an apparently simple direct association.
In doing so, the qualitative approach suggested both future directions for research (such as systematically attempting to measure
associations between income generation strategies, criminal justice consequences and access to needle/syringes), and different
directions for possible interventions designed to improve needle/syringe access (such as delivering needle/syringes in more
dispersed locations, such as community health clinics, or conducting designing satellite exchange programs specifically targeting
groups who under-utilize SEPs). This case study also highlights differences in the level of focus that can be obtained from quantitative
and qualitative data collection. Quantitative assessments are adept
at measuring discrete events (i.e. being stopped by police) and
analyses focuses on determining whether statistically significant
associations exist between those events and pre-defined outcomes.
In contrast, qualitative interviews can reveal larger cultural and
social processes and dynamics (e.g. relations between IDUs and
police) that are more difficult to distill into discrete items to be
measured by a quantitative survey (Bourgois, Prince, & Moss, 2004).
Discussion
We have described three case studies from our own research
with IDUs in which mixed methods designs were employed to
investigate barriers to safer injection. Our selection of these three
case studies is not meant to be globally representative of all mixed
methods research with drug users. Instead, we present them as
examples of different designs and approaches to mixed methods research that share a common theme of working towards a
reconciliation of apparently discordant findings from their mixed
methods designs. By working to integrate initially conflicting qualitative and quantitative findings, we achieved a more nuanced
understanding of the issues than may have been achieved through
the use of a single method. It should be noted, however, that
our interpretations of the mechanisms behind the observed differences are both speculative and partial. Similar to any other
analysis, it is possible that other researchers could identify different explanations. Nonetheless, all three projects served to set a new
expectation regarding agreement between quantitative and qualitative data and highlighted the unique challenges and contributions
of the mixed methods approach when faced with apparently discrepant data. In our case studies, these came in two primary forms:
In Case Studies 1 and 2, our conflicting findings highlighted a need
to be attuned to the methodological differences that may emerge
between quantitative and qualitative data collection phases, and
the implications of those differences for the integration and interpretation of findings. In Case Studies 2 and 3, the use of multiple
methods allowed for different levels of focus, contrasting the
measurement of associations between discrete factors with the
assessment of underlying social, behavioural or cultural processes
that are less easily measured and were not initially identified as a
primary variable or issue under study.
These contributions underscore some of the primary epistemological and methodological differences between quantitative and
qualitative methods. The power to define the issue under study
and the acceptable response options a priori is situated with the
investigator in quantitative research, whilst in qualitative interviews the respondent has more freedom to introduce new topics
or define unexpected response options. Qualitative methods are
particularly useful in highlighting the participant’s “voice” and
allowing him/her to explain responses or provide conditions or
caveats that could be lost in a strictly quantitative assessment.
They also allow the investigator to develop insights that might
59
be difficult to infer quantitatively, working from the “ground up”
to identify variability in behaviour and environmental influences
(Clatts et al., 2002). For example, in Case Study 2, focus group interviews allowed participants to explain the unexpected quantitative
findings and contextualize them in their descriptions of an adaptive
process that was unanticipated and unmeasured in the quantitative survey. However, some qualitative methods may be more
vulnerable to issues of personal representation or social biases
that influence how information is exchanged between respondent and interviewer, encouraging a more reflexive examination
of the process that can be incorporated into the analysis. Qualitative methods such as ethnography are able to uncover cultural and
social meanings and processes that can help explain and contextualize quantitative associations, whilst quantitative methods such as
epidemiological surveys may reveal associations between discrete,
measured variables that participants are unaware of or unable to
explain (Agar, 1997). For example, in Case Study 3, respondents
in the qualitative component spontaneously introduced income
generation techniques as an important factor and made associations between those techniques and their relationships with police.
Through integration of the qualitative results with the quantitative
findings, the investigators were able to provide a more holistic view
of the complex system influencing respondents’ risk for HIV. These
underlying issues should be considered in advance when designing
a mixed methods study, and the different methods can be used to
capitalise on their unique strengths.
Whilst we acknowledge that not all mixed methods research
projects will produce conflicting findings, anticipating that possibility may serve to benefit future projects that employ similar
designs. Based on the lessons learned from these case studies we
conclude with a brief discussion of some logistical challenges that
face investigators implementing mixed methods studies, including
challenges to study design, grant writing and manuscript publication.
In terms of the design and implementation of mixed methods
studies, several issues can be considered in advance. These include
project timelines, sampling schemes, and the use of additional
methods. Planning for the work of integrating findings during the
study design process and developing project timelines in advance
that allow time for integration and/or subsequent data collection phases is a critical factor in ensuring the success of mixed
methods studies. Study designs that are flexible enough to incorporate emerging findings during the research process and to allow
one phase to inform the next (rather than holding all integration activities for the end) may have a greater chance of success.
However, funding mechanisms are often oriented around linear
rather than iterative or reciprocal study designs. More complex
iterative designs may require longer timelines than are allowable
under most common funding mechanisms. Successful integration
of discrepant findings will be more likely if project timelines are sufficient, or supplementary funding is specifically available for mixed
methods projects.
Another design issue to be addressed in mixed methods studies
is that of sampling. Priorities for both sample size and sampling
strategy may differ between qualitative and quantitative methods – whilst quantitative methods are concerned primarily with
demonstrating sufficient statistical power, qualitative methods are
more concerned with achieving conceptual or theoretical saturation. Planning a mixed methods study that allows for the collection
of qualitative data in a way that is sufficiently flexible to achieve
saturation can be challenging, particularly given the requirements
of the grant development and ethical review process that sample
sizes be determined a priori (Mason, 2010). Long standing qualitative approaches such as Grounded Theory (Charmaz, 2006; Glaser &
Strauss, 1967; Strauss & Corbin, 1997), in which sampling is guided
by a search for theoretical saturation and can evolve to address
60
K.D. Wagner et al. / International Journal of Drug Policy 23 (2012) 54–61
emergent issues in the data, may be particularly informative here.
The use of participant observation or ethnographic methods to
inform quantitative sampling strategies may also be recommended
(Clatts, Davis, & Atillasoy, 1995).
Planning for subsequent phases of data collection that are
designed to enlist the respondents in the reconciliation process
(e.g. “member checking” in Case Study 2) can aid in the reconciliation of findings from multiple methods, and can provide an
explicit forum for one phase to inform the other. Other designs may
include the use of additional methods of data collection. Whilst the
projects described here made use of particular forms of qualitative
and quantitative methods, including other data collection methods
such as GIS mapping, participant observation, visual ethnography
or social network analysis could provide yet another perspective on the phenomenon under study. Importantly, none of these
approaches negate the need for thorough piloting of interviews
(both qualitative and quantitative). Including community members
in this process may further enhance validity.
Finally, compared to single method studies, manuscripts that
report integrated findings from mixed methods studies are often
more challenging to publish. They are often longer, frequently
include more complicated methods sections, and are likely to
include a range of results from qualitative, quantitative or other
methods. They can also be challenging for both readers and peerreviewers, who may have greater expertise or confidence in one
method but not others (Bryman, 2007). Epistemological differences that underpin journal publishing preferences may further
compound these difficulties (Rhodes, Stimson, Moore, & Bourgois,
2010). Perhaps as a result of some of these challenges, there are still
relatively few published papers available to serve as exemplars of
an integrative mixed methods approach (Bryman, 2007; Creswell
& Tashakkori, 2007).
Conclusion
Though methodologists and scholars continue to debate the
merits of various theoretical approaches to the integration of mixed
methods research, the use of mixed methods designs is increasingly being used in public health research, including research with
drug using populations. Whether these approaches are employed
from a purely pragmatic perspective or from a more theoretical
one, they have the potential to present challenges to integration.
Whether one subscribes to a positivist view of a single, empirically
measurable “truth” that can be derived, or to a more postmodern
view of reality in which findings are viewed as socially situated and
partial, researchers will face challenges in the integration of findings from mixed methods studies. Despite the challenges inherent
in reconciling apparently conflicting findings from qualitative and
quantitative approaches, in keeping with others who have argued
in favour of integrating mixed methods findings, we contend that
such an undertaking has the potential to yield benefits that emerge
only through the struggle to reconcile discrepant results. That is,
the integration of mixed methods results (even if they initially seem
to disagree) may provide a sum that is greater that the individual
qualitative and quantitative parts (Bryman, 2007), and the reflexive process of engaging with the discrepant findings can itself be
a useful undertaking (Fielding, 2009). On a practical level, we suggest that when planning a mixed methods research project there
is considerable value in preparing for the possibility that different
methodological approaches may produce different and often conflicting results, that processes for engaging with these differences
should be planned in advance, and that engaging with the process of integrating results has the potential to produce richer, more
nuanced, and ultimately more useful results than either method
does alone.
Acknowledgements
This research was supported by the following funding
sources: NIDA 2R01DA016017-07, NIDA 1R36DA024968, NIDA
T32DA023356, NIDA R01DA019829, NIDA K01DA022923, NIDA
K01DA031031, and the California HIV/AIDS Research Program D06SF-424.
References
Agar, M. (1997). Recasting the “Ethno” in “Epidemiology”. Medical Anthropology, 16,
391–403.
Bazeley, P. (2009). Integrating data analyses in mixed methods research. Journal of
Mixed Methods Research, 3(3), 203–207.
Bluthenthal, R. N., Kral, A. H., Lorvick, J., & Watters, J. K. (1997). Impact of law enforcement on syringe exchange programs: A look at Oakland and San Francisco.
Medical Anthropology, 18(1), 61–83.
Bourgois, P. (2002). Anthropology and epidemiology on drugs: The challenges of
cross-methodological and theoretical dialogue. International Journal of Drug Policy, 13, 259–269.
Bourgois, P., Martinez, A., Kral, A., Edlin, B. R., Schonberg, J., & Ciccarone, D. (2006).
Reinterpreting ethnic patterns among White and African American men who
inject heroin: A social science of medicine approach. PLoS Medicine, 3(10), e452.
Bourgois, P., Prince, B., & Moss, A. (2004). The everyday violence of hepatitis C among
young women who inject drugs in San Francisco. Human Organization, 63(3),
253–264.
Bryman, A. (2007). Barriers to integrating quantitative and qualitative research.
Journal of Mixed Methods Research, 1(1), 8–22.
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by
the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81–105.
Case, P., Meehan, T., & Jones, T. S. (1998). Arrests and incarceration of injection drug
users for syringe possession in Massachusetts: Implications for HIV prevention. Journal of Acquired Immune Deficiency Syndromes and Human Retrovirology,
18(Suppl. 1 (1)), S71–S75.
Charmaz, K. (2006). Constructing grounded theory: A practical guide through qualitative analysis. London: Sage.
Clarke, A. E. (2003). Situational analysis: Grounded theory after the postmodern
turn. Symbolic Interaction, 26(4), 553–576.
Clatts, M. C., Davis, W. R., & Atillasoy, A. (1995). Hitting a moving target: The use
of ethnographic methods in the development of sampling strategies for the
evaluation of AIDS Outreach Programs for Homeless Youth in New York City.
Qualitative methods in drug abuse and HIV Research. NIDA Research Monograph,
157, 117–135.
Clatts, M. C., Southeran, J. L., Heimer, R., & Goldsamt, L. A. (1999). Interdisciplinary
research on transmission of blood-borne pathogens among drug injectors:
Applications of ethnography in epidemiology and public health. In P. A. Marshall, M. Singer, M. C. Clatts, R. H. Needle, & E. Lambert (Eds.), Integrating cultural,
observational, and epidemiology approaches in the prevention of drug abuse and
HIV/AIDS: Current status and future prospects (pp. 74–93). Rockville, MD: National
Institutes on Drug Abuse.
Clatts, M. C., Welle, D. L., Goldsamt, L. A., & Lankenau, S. E. (2002). An ethnoepidemiological model for the study of trends in illicit drug use: Reflections
on the ‘emergence’ of crack injection. International Journal of Drug Policy, 13(4),
285–295.
Creswell, J. W., & Plano Clark, V. L. (2006). Designing and conducting mixed methods
research. Thousand Oaks, CA: Sage Publications Inc.
Creswell, J. W., & Tashakkori, A. (2007). Developing publishable mixed methods
manuscripts. Journal of Mixed Methods Research, 1(2), 107–111.
Davidson, P. J. (2009). Space, Place, and Young Injecting Drug Users in San Francisco.
Unpublished Ph.D. Dissertation, University of California, San Francisco.
Denzin, N. K. (2010). Moments, mixed methods, and paradigm dialogs. Qualitative
Inquiry, 16(6), 419–427.
Denzin, N. K., & Lincoln, Y. S. (2000). The discipline and practice of qualitative
research. In N. K. Denzin, & Y. S. Lincoln (Eds.), Handbook of qualitative research.
Thousand Oaks, CA: Sage.
Fielding, N. G. (2009). Going out on a limb: Postmodernism and multiple method
research. Current Sociology, 57(3), 427–447.
Fishbein, M. (Ed.). (1967). Readings in attitude theory and measurement. New York:
Wiley.
Gayle, H. D., O’Neill, J. F., Gust, S. W., & Mata, A. (1997). HIV prevention bulletin: Medical advice for persons who inject illicit drugs. Retrieved from
http://www.cdc.gov/idu/pubs/hiv prev.htm [Electronic Version].
Glaser, B., & Strauss, A. (1967). The discovery of grounded theory. Chicago: Aldine.
Gostin, L. O. (1998). The legal environment impeding access to sterile syringes and
needles: The conflict between law enforcement and public health. Journal of
Acquired Immune Deficiency Syndromes and Human Retrovirology, 18(Suppl 1.
(1)), S60–S70.
Heckathorn, D. D. (1997). Respondent-driven sampling: A new approach to the study
of hidden populations. Social Problems, 44(2), 174–199.
Heimer, R., Bluthenthal, R. N., Singer, M., & Khoshnood, K. (1996). Structural impediments to operational syringe-exchange programs. AIDS and Public Policy Journal,
11(4), 169–184.
K.D. Wagner et al. / International Journal of Drug Policy 23 (2012) 54–61
Kelle, U. (2001). Sociological explanations between micro and macro and
the integration of qualitative and quantitative methods. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, 2. Retrieved from
http://qualitative-research.net/fqs/fqs-eng.htm [Electronic Version, On-line
Journal]
Mason, M. (2010). Sample size and saturation in PhD studies using
qualitative
interviews.
Forum
Qualitative
Sozialforschung/Forum:
Qualitative Social Research, 11. Retrieved from http://www.qualitativeresearch.net/index.php/fqs/article/view/1428/3027 [Electronic Version]
Moran-Ellis, J., Alexander, V. D., Cronin, A., Dickinson, M., Fielding, J., Sleney, J.,
et al. (2006). Triangulation and integration: Processes, claims and implications.
Qualitative Research, 6(1), 45–59.
Moss, A. (2003). Put down that shield and war rattle: Response to Philippe Bourgois.
International Journal of Drug Policy, 14(1), 105–109.
NASEN. (2010). Retrieved from http://www.nasen.org/programs/us/ca/
Pollini, R. A., Lozada, R., Gallardo, M., Rosen, P., Vera, A., Macias, A., et al. (2010).
Barriers to pharmacy-based syringe purchase among injection drug users in
Tijuana, Mexico: A mixed methods study. AIDS and Behavior, 14(3), 679–
687.
Rhodes, T. (2009). Risk environments and drug harms: A social science for
harm reduction approach. International Journal of Drug Policy, 20, 193–
201.
Rhodes, T., Stimson, G. V., Moore, D., & Bourgois, P. (2010). Qualitative social research
in addictions publishing: Creating an enabling journal environment. International Journal of Drug Policy, 21(6), 441–444.
Rich, J. D., Strong, L., Towe, C. W., & McKenzie, M. (1999). Obstacles to needle
exchange participation in Rhode Island. Journal of Acquired Immune Deficiency
Syndromes, 21(5), 396–400.
61
SAFAF. (2010). Retrieved from http://www.sfaf.org/prevention/hiv-preventionproject-needle-exchange.html
Strathdee, S. A., Fraga, W. D., Case, P., Firestone, M., Brouwer, K. C., Perez, S. G., et al.
(2005). “Vivo para consumirla y la consumo para vivir” – [“I live to inject and
inject to live”]: High risk injection behaviors in Tijuana, Mexico. Journal of Urban
Health, 82(3), IV58–IV73.
Strathdee, S. A., Lozada, R. M., Pollini, R. A., Brouwer, K. C., Mantsios, A., Abramovitz,
D. A., et al. (2008). Individual, social, and environmental influences associated
with HIV infection among injection drug users in Tijuana, Mexico. Journal of
Acquired Immune Deficiency Syndromes, 47(3), 369–376.
Strauss, A. L., & Corbin, J. (Eds.). (1997). Grounded theory in practice. Thousand Oaks,
CA: Sage.
Strauss, A. L., & Corbin, J. M. (1998). Basics of qualitative research: Techniques and
procedure for developing grounded theory. Thousand Oaks, CA: Sage.
Torrone, E. A., Thomas, J. C., Maman, S., Pettifor, A. E., Kaufman, J. S., Sena, A. C., et al.
(2010). Risk behavior disclosure during HIV test counseling. AIDS Patient Care
and STDs, 24(9), 551–561.
Wagner, K. D., Lankenau, S. E., Palinkas, L. A., Richardson, J. L., Chou, C.-P., & Unger,
J. B. (2010). The perceived consequences of safer injection: An exploration of
qualitative findings and gender differences. Psychology, Health & Medicine, 15(5),
560–573.
Wagner, K. D., Lankenau, S. E., Palinkas, L. A., Richardson, J. L., Chou, C.-P., & Unger,
J. B. (2011). The influence of the perceived consequences of refusing to share
injection equipment among injection drug users: Balancing competing risks.
Addictive Behaviors, 36, 835–842.
Willms, D. G., Best, J. A., Taylor, D. W., Gilbert, J. R., Wilson, D. M. C., Lindsay, E. A.,
et al. (1990). A systematic approach for using qualitative methods in primary
prevention research. Medical Anthropology Quarterly, 4(4), 391–409.