Studying interaction with documentation: methodological lessons learned in two semi-naturalistic designs
Pages 138 - 143
Abstract
Semi-naturalistic research designs allow for studying behaviour in a realistic setting, achieving a fair degree of ecological validity without the disadvantages of purely naturalistic designs. A semi-naturalistic study sets boundaries for the behaviour under investigation, within which respondents still act freely. In order to allow for between-subjects comparisons, the raw data obtained must be structured, either through pre-structuring or through post-structuring.
Motivation -- There exists little methodological guidance in the field of documentation design. Qualitative and quantitative studies alike are carried out using an amalgam of methods that were developed for other disciplines. This paper contributes to awareness of the pitfalls (but also the benefits) of doing so.
Research approach -- This paper considers two semi-naturalistic studies into interaction with software and documentation from a methodological point of view. In the first, the data was collected in the respondents' workplace and then post-structured. In the second, the data was collected in a laboratory setting and pre-structured through the use of an observation tool.
Findings/Design -- Both methods are described in some detail, followed by a discussion of methodological issues discovered after the design had been executed. Finally, the relative advantages and disadvantages of the two approaches are highlighted.
Take-away message -- Documentation design can fruitfully combine methodological approaches originally developed for other disciplines, provided these are adapted for the purpose with care and discretion.
References
[1]
Chi, M. T. H. (1997). Quantifying Qualitative Analyses of Verbal Data: A Practical Guide. The journal of the learning sciences, 6(3), 271--315.
[2]
Chipman, S. F., Schraagen, J. M., & Shalin, V. L. (2000). Introduction to Cognitive Task Analysis. In J. M. Schraagen, S. F. Chipman & V. L. Shalin (Eds.), Cognitive Task Analysis (pp. 3--23). Mahwah, NJ: Lawrence Erlbaum Associates.
[3]
Ericsson, K. A., & Simon, H. A. (1980). Verbal Reports as Data. Psychological Review, 87(3), 215--251.
[4]
Flanagan, J. C. (1954). The Critical Incident Technique. Psychological Bulletin, 51(4), 327--358.
[5]
Hall, E. P., Gott, S. P., & Pokorny, R. A. (1995). A procedural guide to cognitive task analysis: the PARI methodology (No. OCLC: 44402460): Brooks Air Force Base, Tex.: Armstrong Laboratory Human Resources Directorate, Manpower and Personnel Research Division, Air Force Materiel Command.
[6]
Holtzblatt, K. (2005). Customer-centered design for mobile applications. Pers Ubiquit Comput, 9, 227--237.
[7]
Karreman, J. (2004). Use and effect of declarative information in user instructions. Amsterdam: Editions Rodopi B. V.
[8]
Mehlenbacher, B. (2003). Documentation: not yet implemented, but coming soon The HCI Handbook: Fundamentals, Evolving Technologies, and Emerging Applications (pp. 527--543): Routledge.
[9]
van der Meij, H. (1997). The ISTE Approach to Usability Testing. IEEE Transactions on Professional Communication, 40(3), 209--223.
[10]
Nisbett, R. E., & Wilson, T. D. (1977). Telling More Than We Can Know: Verbal Reports on Mental Processes. Psychological Review, 84(3), 231--253.
[11]
Norman, D. A. (1999). The Invisible Computer. Cambridge, MA: The MIT Press.
[12]
Norman, D. A. (2010). Living with Complexity. Cambridge, MA: The MIT Press.
[13]
Sellen, A. J., & Harper, R. H. R. (2001). The Myth of the Paperless Office. Cambridge, MA: MIT Press.
[14]
Tobin, G. A., & Begley, C. M. (2004). Methodological rigour within a qualitative framework. Journal of Advanced Nursing, 48(4), 388--396.
[15]
Van Gog, T., Paas, F., Van Merriënboer, J. J. G., & Witte, P. (2005). Uncovering the problem-solving process: cued retrospective reporting versus concurrent and retrospective reporting. Journal of Experimental Psychology: Applied, 11, 237--244.
[16]
Wilson, T. D. (1981). Structured observation in the investigation of information needs. Social Science Information Studies(1), 173--184.
Recommendations
The turn to practice in HCI: towards a research agenda
CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsThis paper argues that a new paradigm for HCI research, which we label the 'practice' perspective, has been emerging in recent years. This stands in contrast to the prevailing mainstream HCI paradigm, which we term the 'interaction' perspective. The '...
Enactment of inter-subjectivity in phenomenological bodily interaction: a conceptual framework
DPPI '11: Proceedings of the 2011 Conference on Designing Pleasurable Products and InterfacesHow can design teams constitute human bodily attributions as the locus of productive forces, the site in which an interaction language links up with phenomenological practices of human users? I propose that the salient characteristic of this matter is ...
Towards Informed Practice in HCI for Development
Our paper provides an enriched understanding of the relationship between research and practice through the study of practitioners variously engaged in field research on technology interventions in the context of global development. By conducting a ...
Comments
Information & Contributors
Information
Published In
Copyright © 2012 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
- EACE: European Association for Cognitive Ergonomics
- Edinburgh Napier University, UK: Edinburgh Napier University, UK
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 August 2012
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
ECCE '12
Sponsor:
- EACE
- Edinburgh Napier University, UK
Acceptance Rates
Overall Acceptance Rate 56 of 91 submissions, 62%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 57Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 04 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in