Assignment 5: Qualitative Research

Assignment Description
Many applied research methods were derived from experimental science, which seeks to discover facts and causes. Qualitative or ethnographic methods seek to discover how people see and experience their world, and how they apply meaning to it. Ethnography relies on close observation, in-depth interviews, and extensive description of natural settings (the eld) rather than manipulation of variables in controlled settings (the laboratory). Ethnography brought such concepts as culture, user perspective and user experience, work routines and work-arounds, settings, informal communication, and community to the eld of human-computer interaction. Ethnography also brought to HCI the methods of contextual inquiry and design, and many design research techniques such as photo diaries and focus groups. The goal of this assignment is (1) to get you started in conducting exploratory eldwork and data collection and (2) to get you to practice qualitative data analysis and theory generation using Grounded Theory. You will complete this assignment in two parts: ethnographic data collection and Grounded Theory analysis. While you will have two weeks to complete this assignment, I strongly recommend that you start your assignment early. If you are traveling for Thanksgiving, you can use this opportunity to choose a setting to which you might not otherwise have access. Good luck!

CS/Psych-770 Assignment 5 Qualitative Research!

Page 1 of 17

Part I: Ethnographic Data Collection

In this part of the assignment, you will collect some eld data, write up eld notes, and conduct open-ended interviews. Good qualitative research takes considerable on the job experience, protracted time in the eld, and writing experience. This assignment cannot give you these, but it can offer a structure for getting started and some heuristics for organizing and interpreting eld data. The assignment is designed to serve as a template for getting started on an ethnographic study. Please leave a healthy chunk of time for the observation and to type up eld notes immediately thereafter. An Ethnography Checklist is provided in the next page to guide you through the process for the assignment and in future research.

Resources
The following chapter of the course textbook and papers will provide you with more detail on ethnographic data collection. Lazar, J., Feng, J. H., & Hochheiser, H. (2010). Research Methods in Human-Computer Interaction. Wiley, Chapter 9. 1 Wolnger, N. H. (2002). On writing eldnotes: collection strategies and background expectancies. Qualitative Research, 2 (1), 85-93.2 Mulhall, A. (2003). In the eld: notes on observation in qualitative research. Journal of Advanced Nursing, 41 (3), 306313.3

1 2

Course Textbook https://entwicklungspolitik.uni-hohenheim.de/uploads/media/Day_2_-_Reading_text_4.pdf http://onlinelibrary.wiley.com/doi/10.1046/j.1365-2648.2003.02514.x/full

CS/Psych-770 Assignment 5 Qualitative Research!

Page 2 of 17

Ethnography Checklist
Step 1. Decide general research question or setting Decide initial group or situation comparison(s) you wish to make Select specic setting and hang out or get a tour Rationale for your projects existence (benets anticipated) Identication of stakeholders such as the public, your advisor, your instructors, clients or customers, students, funding sponsor, and so forth Describe the goals and objectives (or purposes) of the project as seen by each group of stakeholders Step 2. Get permissions and plan eldwork Get permissions Decide on your role as participant observer Get organizational permission if needed Complete IRB if necessary (N/A) Decide on video, audio, other data capture and get permission (N/A) Would the answer provide information not now available? Would the information be of continuing interest? What nancial and human resources will be available to answer the question? Will they be sufcient? Will the time span required to get the information meet the needs of decision makers? Decide on time, location, people, groups (N/A) Prepare general interview protocol (N/A) Step 3. Make initial observations and nd informants Draw maps or diagrams of setting (out of sight) Make close observation of everythingsmells, what people wear, tools they use Broad sweep, and then narrow focus on each section of grid Remember jargon, beginning/end of conversations Initial contact with informant(s) and rapport (optional) Step 4. Write up detailed eld notes Note times, locations, people in the setting Write down what everyone did Write down what they said Describe informants and information sources such as documents Write down your questions and how informants answered Write down your interpretations (in parentheses) Use quotes for quoted conversation

CS/Psych-770 Assignment 5 Qualitative Research!

Page 3 of 17

Step 1. Select a general research question or setting

The rst step in ethnography is deciding on a general, initial focus (which will not constrain you if you decide to change later). This focus consists of either: A setting, such as coffee shops, gyms, faculty ofces, poolrooms, DoIT help desk facility, or A phenomenon in which you are interested, such as the online community, driving, coffee culture, or peoples adaptation to new technology. (These are just examples.)

The setting and phenomena you study must exist. You cannot create something to study, because you are trying to understand how people experience their own world and give meaning to it, not what they would do if they lived in your world. (As with all rules, there are exceptions to the stricture to study people in their natural settings. Some ethnographers purposely disrupt something to see how people cope and try to re-create meaning.) Most ethnographers aim to study subjects whose meaning cannot be understood using quantitative methods. For example, Nardi and Whittaker, in writing about the challenges of geographically distributed work, studied the role of restaurant meals in collaboration. In ethnography within sociology, favorite topics concern deviance, social control, and especially the underdog (prostitutes, criminals, shoplifters, poor people, children and mentally ill patients in institutions). In ethnography within anthropology, favorite topics concern kinship patterns, status, and economic processes within peasant or pre-literate societies. In the elds of information systems, design, HCI, and other technology research, favorite topics have included organizational subcultures, managerial control, work practices, structuration (adapting technology use to user needs), relationships with technology, the nature of expertise or know-how, and informal communication and inuence processes. When you choose a setting or phenomenon you wish to study, you also make an initial cut at dening comparisons you wish to make. For example, in Whytes studies of gangs (1943) focused in part on the groups power-leaders and followers, insiders and outsiders. Comparisons can help greatly in understanding a setting or phenomenon. (People are much better at comparing things than making absolute judgments.) Step 1 Choose a high action setting for starting some eldwork. Decide on an initial comparison (such as two groups, or two situations). Choose an unfamiliar setting, so you can look at what happens with fresh eyes.(You should not observe your own workplace, club, or family.) Set aside an hour to closely observe this setting. Below are some suggestions for settings to study. Ask us if you need ideas relevant to your research: DoIT help desk facility The Onion newsroom Coffee shops Nursing home (observe staff and residents both) Halfway house or shelter Fast food restaurant including cooking staff and cooking areas Security checking at the airport all participants Crowded computer clusters Support staff lunchroom during lunch

CS/Psych-770 Assignment 5 Qualitative Research!

Page 4 of 17

Bowling alley at peak hours UW day care center Crowded stores or bars you would never frequent Meeting of underdog or unusual groups such as AA, Green Party, Feminist League.

CS/Psych-770 Assignment 5 Qualitative Research!

Page 5 of 17

Step 2. Get permission

If you want to observe people in an organizational setting, ask the ofcial or person in charge for permission to watch for a while and make some notes. Often it is useful to make contact with the gatekeeper through a friend. Explain to this person that you want to observe for a research methods class, and that you will not be taking down any names or making any judgments. You can say this is for practicing ethnographic techniques. Words such as student and homework are key to avoid intimidating people. If you want to observe people in a public setting (such as a restaurant or park), you do not have to get permission. Taking notes might be sensitive in places like bars, so dont take notes overtly. Better leave note taking for when you get home or can hide in the restroom or your car. You could take pictures and/or record audio or video for this homework as long as you obtain the necessary permissions. For future reference, you should always ask for permission before taking anyones picture or doing audiotaping or videotaping. Research that is part of a class assignment and is simply anonymous observation without video does not need IRB permission. However if this is part of a larger research project, you should obtain IRB approval or ask explicitly for a waiver. For this assignment, you do not need IRB. Step 2 Obtain permission to observe the setting for one hour.

CS/Psych-770 Assignment 5 Qualitative Research!

Page 6 of 17

Step 3. Scope out the setting, do initial observation, and start to nd informants
In participant observation, you enter the eld with the goal of establishing rapport and trust with the people in the setting, and especially with one or more informants. Even if you would prefer to be a y on the wall, you will still need to make those around you comfortable. Ethnographers usually remain passive during their rst days of observation, so for this assignment, thats what youll do. When you are observing, do a broad sweep of the setting, create a mental grid and focus on one section or group of people at a time. Remember key words and jargon, beginning and ends of conversations. Use quotes for what people say. Draw a diagram of the setting as soon as possible. If you arent a fast typist or writer, tape record your observations and interpretations right after you leave the setting. Then use your taped record to help you write up your eld notes.

While you are observing, if things happen that you dont understand, try to nd an informantsomeone who can explain whats happening from the perspective of those in the situation. To create trust with your informant, you need to be sympathetic with his or her point of view and experience. Even if ordinarily this would not be a person you would like, you must have humility with the informantyou are the learner and the informant is the teacher. Step 3 Quietly observe the setting, trying to take in everything that is happening. Watch closely who interacts with whom, who is active and passive, the language people use and what they say and do. Before you do this, see step 4! After youve started to get the sense of the place, start identifying informants to ask about anything you dont understand. When you talk with informants, you should use in-depth interview techniques, with follow up (e.g., asking for other examples). You can also try becoming a participant in the activities of the environment (e.g., helping out a barista in return for information) to gain a more intimate understanding of the environment you are studying. Note: You would typically decide, after scoping out the setting, if you want to use audio or video recording. There are advantages and disadvantages of doing so. Recording can inuence the behavior of people in the setting. They will tend to be worried about looking good. However, some ethnographers insist that people get used to recording devices. As noted earlier, using audio or video recordings is optional. You can choose to rely only on eldnotes.

CS/Psych-770 Assignment 5 Qualitative Research!

Page 7 of 17

Step 4. Write up detailed eld notes

Ethnography involves writing up detailed eld notes each time you observe your setting. Typically eld notes take longer to write than the observation itself; this is not a casual activity you can do well. It is no excuse that nothing happened. What are detailed eld notes? They are notes that depend on your detailed memory of the people, events, and conversations you observed (as well as what you are doing and thinking, in parentheses). Pretend you are a novelist and will have to re-create the setting for readers. Heres what to include. Note rough times down the side of the page, numbering each line/ Next to each number, write your descriptions, describe locations, people in the setting/ Write down what everyone did. Write down what they said, in quotations. Describe informants and information sources such as documents Write down your questions (Obs. Q: xxx) and how informants answered (Joes A: xxxx) Write down your interpretations (in parentheses)

Step 4 Immediately after the observation period, write up your detailed eld notes of everything you observed, what people said (quoted if possible), and interpretations you make of what you observed (in parentheses).

CS/Psych-770 Assignment 5 Qualitative Research!

Page 8 of 17

Part II: Grounded Theory Analysis

In this part of the assignment, you will analyze the qualitative data that you collected in the previous part using the Grounded Theory approach. This approach will provide you with a framework to make sense of the qualitative data you collected, identify concepts, relationships, and generate formal models or theories of real-world phenomena. The nal product of the data analysis should be a model of or theory on your observations at the setting you studied. This model or theory should be in a form that can be translated into design specications or into testable hypotheses for further research.

Resources
The following book chapters will provide you with more detail on the Grounded Theory process: Creswell, J. W. (1998). Qualitative Inquiry and Research Design: Choosing Among Five Traditions. Thousand Oaks, CA: Sage Publications, Chapter 8. 4 Adams, A., Lunt, P, & Cairns, P. (2008). A Qualitative Approach to HCI Research. In Research Methods for Human-Computer Interaction. Paul Cairns & Anna L. Cox (Eds.) Cambridge University Press. Chapter 7. 5

4 5

http://pages.cs.wisc.edu/~bilge/private/Creswell98-Ch8-DataAnalysis.pdf (Log-in using VPN.)

http://pages.cs.wisc.edu/~bilge/private/Adams2008-AQualitativeApproachToHCIResearch.pdf (Log-in using VPN.)

CS/Psych-770 Assignment 5 Qualitative Research!

Page 9 of 17

Step 1. Data Formatting

While qualitative research might involve the use a wide range of media as data, Grounded Theory was developed to analyze textual data. Therefore, it is important that you translate all the data you collected into a textual form. These might include: Fieldnotes (notes from your observations) Interview transcripts Memos (notes for yourself that describe insights, preliminary ideas, models, theories) Documents (archival or other documents that others in the setting you are studied created)

A number of software packages exist for Grounded Theory analysis (e.g., commercial NVivo6 and XSight,7 open-source TAMs Analyzer8 and Weft QTA9) and all expect plan text input. These software packages allow you to handle large amounts of data and build graphical representations. I encourage you to download and explore with one of the open source packages (TAMs is for Mac OS X, Weft QTA is for Windows and Linux). However, you can simply use excel for this assignment. Step 1 Translate all your data into a textual form. Identify a unit of analysis (e.g., a sentence, a paragraph, a response, an instance of an event happening) and insert each unit into a row in Excel. The easiest way to do this is to insert line breaks between each unit in a plan text editor and import the le into Excel. Excel will recognize the line breaks and place each unit into a row. Create a few more columns where you will write down your codes.

6 7 8 9

http://www.qsrinternational.com/ http://www.qsrinternational.com/ http://tamsys.sourceforge.net/ http://www.pressure.to/qda/

CS/Psych-770 Assignment 5 Qualitative Research!

Page 10 of 17

Step 2. Open Coding

In the open coding step, your goal should be identifying concepts (e.g., actions, interactions, responses, events, etc.) that you nd signicant in the context of your study. This step will allow you to go from text to concepts, providing an abstract, conceptual representation of your data. The concepts you identify are called codes and the process of marking your data for these codes is called coding. As you code your data, you will see that some of the concepts you identify reoccur and you start reusing the codes you created at the beginning of the coding process. At some point, the coding reaches a level of maturity and no new codes emerge. This is a natural point to stop open coding and move to axial coding. Step 2 Code each unit in your Excel le with concepts. Each unit might contain multiple signicant events and might need more than one code. In this case, use the second column of your spreadsheet for the primary code and add other codes in subsequent columns. Make sure that you choose codes that can be interpreted by others and make sense in the context of your data.

CS/Psych-770 Assignment 5 Qualitative Research!

Page 11 of 17

Step 3. Axial Coding

In the axial coding step, your goal will be to categorize the codes you created in the open coding step. This step is essential identify the appropriate levels of abstraction in the concepts you identify in the data. If you remember the examples from class, abusing technology might not be abstract enough to build theoretical models and might fall in the more abstract category of negative treatments of technology. However, this does not mean that once you categorize codes, they become irrelevant. It will still be relevant when you are creating your story. The categories you create in this step can be anything. However, most categories fall under the following classes: circumstances, actions/interactions, and consequences. One way of categorizing your codes while keeping these classes in mind is to create your own categories and classify the categories under these three classes. This classication will produce a hierarchical categorization such as Actions > Negative treatments of technology > Yelling at the computer. Categories can also be created considering different dimensions. For instance, yelling at the computer can be categorized under treatments of technology but in a negative direction, while there might also be a positive direction such as Maintaining the computer !positive ! Treatments of technology " negative " Yelling at the computer. You can again use Excel to create categories. Another option is to use a mindmapping software package (e.g., open source XMind,10 Gantt Project,11 FreeMind,12 or Compendium 13). Mind-mapping is a powerful tool for both qualitative research and general information organization and brainstorming. I suggest you become familiar with one of these tools in the long run. Step 3 Categorize the codes you created in the open coding step. You can use Excel or one of the mind-mapping software packages listed above. Like codes, categories should also be interpretable by others. Avoid using abbreviations or shortcuts.

10 11 12 13

http://www.xmind.net/ http://www.ganttproject.biz/ http://freemind.sourceforge.net/wiki/index.php/Main_Page http://compendium.open.ac.uk/institute/download/download.htm

CS/Psych-770 Assignment 5 Qualitative Research!

Page 12 of 17

Step 4. Selective Coding

The selective coding step will allow you to indentify relationships between codes and categories that you created in the previous step. The classes I described in the previous step might be useful in indentifying these relationships. You can ask the question, What consequences arise from what actions/interactions under what circumstances? Once you start answering this question using your codes and categories, you will start nding relationships such as High workload (circumstances) " Technology interrupting workers (actions) " Negative treatments of technology (consequences). These relationships should be short stories that you create through your analysis such as people with high workload treat technology negatively when it interrupts them. Mind-mapping software packages would be extremely useful in identifying these relationships. However, if you do not want to invest in learning to use a new software package, paper and pencil would also be sufcient for this assignment. Step 4 Identify relationships in your codes and categories and translate these relationships into stories that you will further rene in the next step. The outcome of this step should be causal diagrams as well as narrative stories.

CS/Psych-770 Assignment 5 Qualitative Research!

Page 13 of 17

Step 5. Comparative Analysis

Up to this step, you were working with abstract concepts, categories, and relationships. Now, you should go back to your data with the stories you created in the previous step in order to: Test whether the stories t into your data, Compare your stories across different axes (e.g., across the gender axis, test whether it holds for females but not males, or across a location axis, whether the story holds for people who use the computer to nd a reference at the library but not for those who ask the librarian).

This step will allow you to test and rene the provisional stories you created in the previous step and nd examples in your data for these stories. Ideally, you might want to conduct a follow-up study, go back to the eld to collect more data to see if your stories predict new data. However, for the purposes of this assignment, you do not need to collect further data (unless you want to). If you are using excel, you can create a new column for stories and insert your stories in the row of the textual data that conrms or contradicts your data. Make the appropriate modications when there is a contradiction (these might be exceptions or they might occur under different circumstances and, therefore, can be explained by a different story). Step 5 Go through your textual data to look for instances of events conrming or contradicting your stories. Highlight these examples and make appropriate modications to your stories based on your comparisons.

CS/Psych-770 Assignment 5 Qualitative Research!

Page 14 of 17

Step 6. Theory Building

In the last step of the analysis, you will combine the stories that you developed in the previous step into a unifying model. The stories do not have to t into a unifying model, but you should look for higher-level concepts, categories, relationships that you might or might not have identied earlier in your analysis. The nal model might be a collection of rened stories or one unifying model. Ideally, you would embed existing theory into the model or nd a unifying paradigm from existing theory at this stage. However, for the purposes of this assignment, you do not need to do this (unless you want to). The nal model can be in the form of a textual description or a causal diagram. Again, the mind-mapping software packages might be very useful in creating the nal product. Step 6 Look for a high-level paradigm that can connect all the stories you created in your comparative analysis. If you are able to identify a unifying theory, build a diagram or write up a textual description of this paradigm. If not, build diagrams or descriptions of individual stories.

CS/Psych-770 Assignment 5 Qualitative Research!

Page 15 of 17

Final Step: Write-Up

Write a two-page report of your data collection and analysis process and ndings in two sections. In the rst section, provide rich but formal descriptions of your setting and the data collection methods you followed. In particular, describe What question you had in mind, How you chose the setting, What setting you went to, What you observed (important actors, actions, and rough themes that you identied we will do a more detailed analysis in next weeks assignment).

In the second section, describe all the steps you followed in your analysis, how many codes, categories, relationships, and stories you identied. Include diagrams or textual descriptions of your stories. Attach the following to your report: 1. 2. 3. 4. 5. 6. Your eldnotes and transcripts of any interviews (.doc le), The unitized data (.xls), List of codes (.xls), List of categories and which codes they include (.xls), List and brief descriptions of relationships you built (.xls), Diagrams or textual descriptions of your stories (.pdf).

You can include items 2-5 in the same .xls le. Please use the assignment template14 for your report. Send your submission to wisconsinhci@gmail.com by December 12, 11:59 pm following the naming convention Assignment5-StudentLastName.zip.

14

http://www.cs.wisc.edu/hci/courses/hci/content/Assignment5-Template.docx (Log-in using VPN.)

CS/Psych-770 Assignment 5 Qualitative Research!

Page 16 of 17

Grading Criteria
Part I: Ethnographic Data Collection (5 points)
1. Identifying a setting and an appropriate research question (1/1) 2. Accessing the setting and spending an appropriate amount of time (1/1) 3. Following ethnographic principles and methods to understand the setting (e.g., yon-the-wall observations, participant observation, interviews) (1/1) 4. Transcribing eld notes, interviews, and other materials obtained in the setting (1/1) 5. Providing a clear account of your ethnographic data collection process and ndings in the section part of your report (1/1)

Part II: Grounded Theory Analysis (5 points)

1. 2. Formatting the data from your ethnography in a textual form (1/1) Rigor in following in Grounded Theory data analysis process (3/3): 2.1. Open coding (i.e., abstraction of your data into concepts) (0.5/0.5) 2.2. Axial coding (i.e., categorization of your abstract concepts into a structure) (0.5/0.5) 2.3. Selective coding (i.e., the relationships you built and the stories you created) (0.5/0.5) 2.4. Comparative analysis (i.e., substantiation of your stories in the data) (0.5/0.5) 2.5. Theory building (i.e., rening your stories into a nal explanatory theory) (0.5/0.5) 2.6. Use of appropriate tools in the analysis process (i.e., Excel or other coding and mind-mapping tools) (0.5/0.5) 3. Quality of the second section of your report on your Grounded Theory process and ndings (1/1)

CS/Psych-770 Assignment 5 Qualitative Research!

Page 17 of 17