What is Trustworthiness?

Trustworthiness of qualitative research:

A key issue for qualitative research is developing a shared understanding of appropriate

procedures for assessing its credibility or trustworthiness (“validity”). In a broad sense,
trustworthy qualitative research, like quantitative research, needs to be based on systematic
collection of data, using “acceptable” research procedures, and allowing the procedures and
findings to be open to systematic critical analysis from others.

Trustworthiness of Qualitative Data

As in quantitative studies, issues of reliability and validity of the study are encountered in
qualitative approach. Henderson (1991) describes the terms as follows: "

Credibility or internal validity refers to how truthful particular findings are.

Transferability or external validity refers to how applicable or generalizable the research findings
are to another setting or group.

Dependability or reliability refers to how we can be sure that our findings are consistent and
reproducible.

Confirmability or objectivity refers to how neutral the findings are in terms of whether they are
reflective of the subjects and the inquiry and not a product of the researcher's biases and
prejudices.

Approaches to increase trustworthiness in qualitative studies:

Having a thick description of the research process and how the investigator reaches the
conclusions, can greatly help another researcher replicate the study and arrive at the same
general scheme.

Prolonged engagement - investing sufficient time to learn about the culture to be studied,
detecting and minimizing distortions that may slowly shape the data, and building trust with the
respondents.

Persistent observation - identifying characteristics and elements relevant to the research.

Peer debriefing - exposing oneself to a disinterested peers in a manner similar to that of an

analytic session, to explore aspects that may be implicit in the researcher's mind.

Referential adequacy - using mechanically recorded data such as tape recorders, videotapes,
photographs.
Member checks or cross-examination - going back to individuals and checking out conclusions
and to corroborate what has been observed.

Other methods to establish trustworthiness.

Trustworthiness relates to how we determine if “we got it right.” The topic of

trustworthiness in qualitative research is emerging and complex. Creswell recommended using at
least two of the following strategies (including triangulation as mentioned above):

· Reflexivity – Tracking one’s thoughts over the course of the study, by keeping a field
journal of personal reflections.

· Prolonged field experience.

· Triangulation.

· Member checking – Asking participants to review and give feedback on transcripts, notes,
drafts, etc.

· Peer examination – Discussing research with colleagues.

· Negative case analysis – Challenging working hypothesis with evidence that doesn’t fit and
potentially revising the hypothesis.

· Clarifying researcher bias – Comment on past experiences, biases, prejudices, and

orientations that are likely to shape the interpretation and approach of the study.

· External audit – External consultant examine process and product.

· Audit trail – allow for tracking back from findings to data.

· Rich, thick description – Allow reader to make decisions regarding transferability.

· Code-recode – Code, wait a period of time, and then recode.

· Nominated sample – Use panel of judges to help in participant selection.

· Compare sample to demo – Compare participants to demographic data.

· Dense description of sample – Describe participants in depth.

 The University of Akron
~ Department of Counseling ~
Shannon D. Smith PhD, PCC, NCC, DAPA,
NSC

Qualitative Research
"Qualitative research genres have become increseasingly important modes of inquiry for social
sciences and applied fields such as education, regional planning, nursing, social, and applied
fields such as education" (Marshall and Rossman, 1998, p. 1).

References Syllabus Dr. Smith's Home Page

What is Qualitative Research?

"Qualitative research is a loosely defined category of research designs or models, all of which
elicit verbal, visual, tactile, olfactory, and gustatory data in the form of descriptive narratives like
field notes, recordings, or other transcriptions from audio- and videotapes and other written
records and pictures or films." Definition [Judith Preissle] [No nice neat definition really
encapsulates qualitative research. It's as much a perspective as it is method].

Helpful links to understanding Qualitative Research:

1. http://www.uea.ac.uk/care/elu/Issues/Research/Res1Ch1.html
2. http://www.nova.edu/ssss/QR/QR3-3/bowen.html
3. http://don.ratcliff.net/qual/resources.html
4. http://trochim.human.cornell.edu/kb/qual.htm
5. http://www.arts.uwaterloo.ca/~lsnyder/iss251/QualitativeResearch.htm
6. Qualitative Validity: http://trochim.human.cornell.edu/kb/qualval.htm
 7. http://www.okstate.edu/ag/agedcm4h/academic/aged5980a/5980/newpage21.htm
8. http://blake.montclair.edu/~junius/Research/ResearchQualitative.html
9. http://web.syr.edu/~bvmarten/ethno.html
10. http://www.mh.state.oh.us/offices/oper/feature4.html
11. http://scholar.lib.vt.edu/ejournals/JTE/v9n1/hoepfl.html
12. http://www.wlu.ca/~wwwk+pe/faculty/pbryden/kp261/Observing%20Behavior.htm
13. Mix Methods-- http://soegrad.colostate.edu/programs/StudyGuideSM01.htm
14. http://www.slais.ubc.ca/resources/research_methods/qualitat.htm
15. http://www.curriculum.edu.au/tech/articles/choose.htm
16. http://soegrad.colostate.edu/programs/StudyGuideSM01.htm

Basis for the Use of a Qualitative Methodology

There are several considerations when deciding to adopt a qualitative research methodology.
Strauss and Corbin (1990) claim that qualitative methods can be used to better understand any
phenomenon about which little is yet known. They can also be used to gain new perspectives on
things about which much is already known, or to gain more in-depth information that may be
difficult to convey quantitatively. Thus, qualitative methods are appropriate in situations where
one needs to first identify the variables that might later be tested quantitatively, or where the
researcher has determined that quantitative measures cannot adequately describe or interpret a
situation. Research problems tend to be framed as open-ended questions that will support
discovery of new information. Greene’s 1994 study of women in the trades, for example, asked,
"What personal characteristics do tradeswomen have in common? In what way, if any, did role
models contribute to women’s choices to work in the trades?" (p. 524a).

The ability of qualitative data to more fully describe a phenomenon is an important consideration
not only from the researcher’s perspective, but from the reader’s perspective as well. "If you
want people to understand better than they otherwise might, provide them information in the
form in which they usually experience it" (Lincoln and Guba, 1985, p. 120). Qualitative research
reports, typically rich with detail and insights into participants’ experiences of the world, "may
be epistemologically in harmony with the reader’s experience" (Stake, 1978, p. 5) and thus more
meaningful.

A Quick Definition of Qualitative Research in Education

It is difficult to find an unambiguous and definitive statement as to what qualitative research in
education actually is. This is primarily due, as Lancy (1993) points out, to the fact that "...topic,
theory, and methodology are usually closely interrelated in qualitative research (p. 3)."
Therefore, a brief definition of the field will center on the methods, terms, and topics employed
in qualitative research. These have been quite diverse. For instance, Bogdan and Biklen (1994)
point out that qualitative research in education draws from many sources, reflected by the use of
such terms as "...symbolic interactionist, inner perspective, the Chicago School,
phenomenological, case study, interpretive, ethnomethodological, ecological, and descriptive (p.
3)." In a similar fashion, Glesne and Peshkin (1992) observe: "Qualitative inquiry is an umbrella
term for various philosophical orientations to interpretive research. For example, qualitative
researchers might call their work ethnography, case study, phenomenology, educational
criticism, or several other terms (p. 9)."
By Gary Shank
The Qualitative Report, Volume 2, Number 3, December, 1995
(http://www.nova.edu/ssss/QR/QR2-3/shank.html )

What is Naturalistic Inquiry?

Studying real-word situations as they unfold naturally; non-manipulative, unobtrusive, and non-
controlling; openness to whatever emerges---lack of predetermined constraints on outcomes
(Patton, 1990, p. 40)

Websites about Naturalistic Inquiry

1. http://writing.colostate.edu/references/research/observe/com3b1.cfm
2. http://www.cc.gatech.edu/classes/AY2001/cs6455_spring/five_axioms.html
3. http://www.uky.edu/~drlane/cohort/methods/2000ch10.htm

What is the Role of the Researcher in Qualitative Inquiry?

Before conducting a qualitative study, a researcher must do three things. First, (s)he must adopt
the stance suggested by the characteristics of the naturalist paradigm. Second, the researcher
must develop the level of skill appropriate for a human instrument, or the vehicle through which
data will be collected and interpreted. Finally, the researcher must prepare a research design that
utilizes accepted strategies for naturalistic inquiry (Lincoln and Guba, 1985).

Glaser and Strauss (1967) and Strauss and Corbin (1990) refer to what they call the "theoretical
sensitivity" of the researcher. This is a useful concept with which to evaluate a researcher’s skill
and readiness to attempt a qualitative inquiry.

Theoretical sensitivity refers to a personal quality of the researcher. It indicates an awareness of

the subtleties of meaning of data. … [It] refers to the attribute of having insight, the ability to
give meaning to data, the capacity to understand, and capability to separate the pertinent from
that which isn’t (Strauss and Corbin, 1990, p. 42).

Strauss and Corbin believe that theoretical sensitivity comes from a number of sources, including
professional literature, professional experiences, and personal experiences. The credibility of a
qualitative research report relies heavily on the confidence readers have in the researcher’s
ability to be sensitive to the data and to make appropriate decisions in the field (Eisner, 1991;
Patton, 1990).

Lincoln and Guba (1985) identify the characteristics that make humans the "instrument of
choice" for naturalistic inquiry. Humans are responsive to environmental cues, and able to
interact with the situation; they have the ability to collect information at multiple levels
simultaneously; they are able to perceive situations holistically; they are able to process data as
soon as they become available; they can provide immediate feedback and request verification of
data; and they can explore atypical or unexpected responses.

What is Trustworthiness?
Trustworthiness of qualitative research:

A key issue for qualitative research is developing a shared understanding of appropriate

procedures for assessing its credibility or trustworthiness (“validity”). In a broad sense,
trustworthy qualitative research, like quantitative research, needs to be based on systematic
collection of data, using “acceptable” research procedures, and allowing the procedures and
findings to be open to systematic critical analysis from others.

Trustworthiness of Qualitative Data

As in quantitative studies, issues of reliability and validity of the study are encountered in
qualitative approach. Henderson (1991) describes the terms as follows: "

Credibility or internal validity refers to how truthful particular findings are.

Transferability or external validity refers to how applicable or generalizable the research findings
are to another setting or group.

Dependability or reliability refers to how we can be sure that our findings are consistent and
reproducible.

Confirmability or objectivity refers to how neutral the findings are in terms of whether they are
reflective of the subjects and the inquiry and not a product of the researcher's biases and
prejudices.

Approaches to increase trustworthiness in qualitative studies:

Having a thick description of the research process and how the investigator reaches the
conclusions, can greatly help another researcher replicate the study and arrive at the same
general scheme.

Prolonged engagement - investing sufficient time to learn about the culture to be studied,
detecting and minimizing distortions that may slowly shape the data, and building trust with the
respondents.

Persistent observation - identifying characteristics and elements relevant to the research.

Peer debriefing - exposing oneself to a disinterested peers in a manner similar to that of an

analytic session, to explore aspects that may be implicit in the researcher's mind.
Referential adequacy - using mechanically recorded data such as tape recorders, videotapes,
photographs.

Member checks or cross-examination - going back to individuals and checking out conclusions
and to corroborate what has been observed.

Other methods to establish trustworthiness.

Trustworthiness relates to how we determine if “we got it right.” The topic of

trustworthiness in qualitative research is emerging and complex. Creswell recommended using at
least two of the following strategies (including triangulation as mentioned above):

· Reflexivity – Tracking one’s thoughts over the course of the study, by keeping a field
journal of personal reflections.

· Prolonged field experience.

· Triangulation.

· Member checking – Asking participants to review and give feedback on transcripts, notes,
drafts, etc.

· Peer examination – Discussing research with colleagues.

· Negative case analysis – Challenging working hypothesis with evidence that doesn’t fit and
potentially revising the hypothesis.

· Clarifying researcher bias – Comment on past experiences, biases, prejudices, and

orientations that are likely to shape the interpretation and approach of the study.

· External audit – External consultant examine process and product.

· Audit trail – allow for tracking back from findings to data.

· Rich, thick description – Allow reader to make decisions regarding transferability.

· Code-recode – Code, wait a period of time, and then recode.

· Nominated sample – Use panel of judges to help in participant selection.

· Compare sample to demo – Compare participants to demographic data.

· Dense description of sample – Describe participants in depth.

Helpful web sites to help you understand trustworthiness in qualitative research

 1. http://www.okstate.edu/ag/agedcm4h/academic/aged5980a/5980/newpage21.htm
2. http://www.slais.ubc.ca/resources/research_methods/measurem.htm#validity

What is the Biography?

What is Ethnography?
What is Grounded Theory?
What is Phenomenology?

The major approaches within qualitative research that are used in education and the social
sciences.

1. Biography – Focus on life of an individual. Data collection – interviews and documents.

Data analysis – stories, epiphanies, historical context. Narrative – detailed picture of the
individual’s life.

2. Phenomenology – Focus on understanding essences of experiences about a

concept/phenomenon. Data collection – long interviews with up to 10 people. Data analysis –
statements, meanings, meaning themes, general description of the experience. Narrative –
description of the “essence” of the experience.

3. Grounded Theory – Focus on developing a theory ground in the data from the field. Data
collection – interviews with 20-30 people to saturate categories and detail a theory. Data analysis
– open coding, axial coding, selective coding, conditional matrix. Narrative – theory or
theoretical model.

4. Ethnography – Focus on drawing a portrait of a cultural group through description and

interpretation. Data collection – observations and interviews, with additional artifacts, during
extended time in the field. Data analysis – description, analysis, interpretation. Narrative –
description of the cultural behavior of a group or an individual.

5. Case Study – Focus on developing an in-depth analysis of a single case or multiple cases.
Data collection – documents archival records, interviews, observations, and artifacts. Data
analysis – description, themes, assertions. Narrative – in-depth study of a case or cases.

Qualitative Journals
http://www.nova.edu/ssss/QR/web.html
http://www.nova.edu/ssss/QR/calls.html

What is Qualitative Data?

Detailed, thick description; inquiry in depth; direct quotations capturing people's personal
perspectives and experiences
(Patton, 1990, p. 40)
Analysis of Data
Bogdan and Biklen define qualitative data analysis as "working with data, organizing it, breaking
it into manageable units, synthesizing it, searching for patterns, discovering what is important
and what is to be learned, and deciding what you will tell others" (1982, p. 145). Qualitative
researchers tend to use inductive analysis of data, meaning that the critical themes emerge out of
the data (Patton, 1990). Qualitative analysis requires some creativity, for the challenge is to place
the raw data into logical, meaningful categories; to examine them in a holistic fashion; and to
find a way to communicate this interpretation to others.

Sitting down to organize a pile of raw data can be a daunting task. It can involve literally
hundreds of pages of interview transcripts, field notes and documents. The mechanics of
handling large quantities of qualitative data can range from physically sorting and storing slips of
paper to using one of the several computer software programs that have been designed to aid in
this task (see Brown, 1996, for a description of one of these programs).

Analysis begins with identification of the themes emerging from the raw data, a process
sometimes referred to as "open coding" (Strauss and Corbin, 1990). During open coding, the
researcher must identify and tentatively name the conceptual categories into which the
phenomena observed will be grouped. The goal is to create descriptive, multi-dimensional
categories which form a preliminary framework for analysis. Words, phrases or events that
appear to be similar can be grouped into the same category. These categories may be gradually
modified or replaced during the subsequent stages of analysis that follow.

As the raw data are broken down into manageable chunks, the researcher must also devise an
"audit trail"—that is, a scheme for identifying these data chunks according to their speaker and
the context. The particular identifiers developed may or may not be used in the research report,
but speakers are typically referred to in a manner that provides a sense of context (see, for
example, Brown, 1996; Duffee and Aikenhead, 1992; and Sours, 1997). Qualititative research
reports are characterized by the use of "voice" in the text; that is, participant quotes that illustrate
the themes being described.

The next stage of analysis involves re-examination of the categories identified to determine how
they are linked, a complex process sometimes called "axial coding" (Strauss and Corbin, 1990).
The discrete categories identified in open coding are compared and combined in new ways as the
researcher begins to assemble the "big picture." The purpose of coding is to not only describe
but, more importantly, to acquire new understanding of a phenomenon of interest. Therefore,
causal events contributing to the phenomenon; descriptive details of the phenomenon itself; and
the ramifications of the phenomenon under study must all be identified and explored. During
axial coding the researcher is responsible for building a conceptual model and for determining
whether sufficient data exists to support that interpretation.

Finally, the researcher must translate the conceptual model into the story line that will be read by
others. Ideally, the research report will be a rich, tightly woven account that "closely
approximates the reality it represents" (Strauss and Corbin, 1990, p. 57). Many of the concerns
surrounding the presentation of qualitative research reports are discussed in the section "Judging
Qualitative Research" which follows.
Although the stages of analysis are described here in a linear fashion, in practice they may occur
simultaneously and repeatedly. During axial coding the researcher may determine that the initial
categories identified must be revised, leading to re-examination of the raw data. Additional data
collection may occur at any point if the researcher uncovers gaps in the data. In fact, informal
analysis begins with data collection, and can and should guide subsequent data collection. For a
more detailed yet very understandable description of the analysis process, see Simpson and
Tuson (1995).

Help web sites to further understand data analysis:

1. http://writing.colostate.edu/references/research/observe/com3b10.cfm
2. http://www.analytictech.com/geneva97/whatis.htm
3. http://www.slais.ubc.ca/resources/research_methods/qualitat.htm

Qualitative Data Analysis

Web sites for Data Analysis

1. http://www.scolari.co.uk/frame.html?
http://www.scolari.co.uk/hyperresearch/hyperresearch.htm
2. http://www.slais.ubc.ca/resources/research_methods/software.htm
3. Atlas.ti
BEST
C-I-SAID
Decision Explorer
Diction
The Ethnograph
GBSTAT 6.5
HyperRESEARCH
MAXqda
QSR NUD*IST 6
QSR NUD*IST Vivo 1.3
SphinxSurvey
Methodologist's Toolchest
winMAX

Writing a Qualitative Proposal

1. http://www.research.vhhsc.ca/i/presentations/QualitativeProposals/
2. http://www.nrf.ac.za/methods/pwkshop.ppt
3. http://www.tc.umn.edu/~musi0012/guideline2.HTM
4. Guideline and helpful suggestions to writing a proposal, dissertation:
http://faculty.washington.edu/krumme/readings/res+writ.html
References for Qualitative Inquiry

http://gradeng.en.iup.edu/mmwimson/710_Trust_Valid_Bib.htm

Selected List of "Qualitative" Research and "Qual-friendly" Journals

American Educational Research Journal
American Journal of Sociology
American Sociological Review
Annual Review of Anthropology
Anthropological Quarterly
Anthropology of Education Quarterly
Anthropology of Work Review
Cultural Anthropology
Educational Foundations
Ethos (Psychological Anthropology)
Harvard Educational Review
Historica Pedagogica
History of Education Quarterly
Human Organization (Applied Anthropology)
International Journal of Qualitative Research in Education
Journal of Contemporary Ethnography
Journal of Cross-cultural Psychology
Journal of Curriculum Studies
Journal of Education
Journal of Medical Humanities
Journal of Research in Rural Education
Oral History Review
Qualitative Health Research
Qualitative Sociology
Review of Educational Research
Science, Technology, and Society
Sociology of Education
Sociology of Sport
Teachers College Record
Urban Education
Visual Arts Research
Visual Sociology

E-mail comments to Dr. Smith

© 2000 The University of Akron
Last Modified 9/11/02
The University of Akron is an equal education and employment institution