research-article

Qualitative Methods in Empirical Studies of Software Engineering

Author:

Carolyn B. SeamanAuthors Info & Claims

IEEE Transactions on Software Engineering, Volume 25, Issue 4

Pages 557 - 572

https://doi.org/10.1109/32.799955

Published: 01 July 1999 Publication History

Abstract

While empirical studies in software engineering are beginning to gain recognition in the research community, this subarea is also entering a new level of maturity by beginning to address the human aspects of software development. This added focus has added a new layer of complexity to an already challenging area of research. Along with new research questions, new research methods are needed to study nontechnical aspects of software engineering. In many other disciplines, qualitative research methods have been developed and are commonly used to handle the complexity of issues involving human behavior. This paper presents several qualitative methods for data collection and analysis and describes them in terms of how they might be incorporated into empirical studies of software engineering, in particular how they might be combined with quantitative methods. To illustrate this use of qualitative methods, examples from real software engineering studies are used throughout.

References

[1]

T. Athey, “Leadership Challenges for the Future,” IEEE Software, vol. 15, no. 3, pp. 72–77, May 1998.

Digital Library

[2]

S.R. Barley, “The Alignment of Technology and Structure through Roles and Networks,” Administrative Science Quarterly, vol. 35, pp. 61–103, 1990.

[3]

V.R. Basili R.W. Selby and D.H. Hutchens, “Experimentation in Software Engineering,” IEEE Trans. Software Eng., vol. 12, no. 7, pp. 733–743, July 1986.

Digital Library

[4]

F. Brooks Jr., The Mythical Man-Month. Reading, Mass.: Addison-Wesley, 1975.

Digital Library

[5]

N. Brown, “Industrial-Strength Management Strategies,” IEEE Software, vol. 13, no. 4, pp. 94–103, July 1996.

Digital Library

[6]

W. Curtis, “By the Way, Did Anyone Study Any Real Programmers?” Empirical Studies of Programmers, R. Soloway and S. Iyengar, eds., pp. 256–262, Norwood, N.J.: Ablex Publishing, 1986.

Digital Library

[7]

K.M. Eisenhardt, “Building Theories from Case Study Research,” Academy of Management Review, vol. 14, pp. 532–550, 1989.

[8]

J.F. Gilgun, “Definitions, Methodologies, and Methods in Qualitative Family Research,” Qualitative Methods in Family Research. Thousand Oaks: Sage, 1992.

[9]

B.G. Glaser and A.L. Strauss, The Discovery of Grounded Theory: Strategies for Qualitative Research. Aldine Publishing, 1967.

[10]

J.T. Hackos and J.D. Redish, User and Task Analysis for Interface Design. pp. 258–259, New York: John Wiley & Sons, ch. 9, 1998.

Digital Library

[11]

C.M. Judd E.R. Smith and L.H. Kidder, Research Methods in Social Relations, sixth ed., ch. 13, pp. 298–320, Ft. Worth: Harcourt Brace Jovanovich, 1991.

[12]

Y.S. Lincoln and E.G. Guba, Naturalistic Inquiry. Thousand Oaks Calif.: Sage, 1985.

[13]

S. McConnell, Rapid Development: Taming Wild Software Schedules. ch. 11, pp. 249–272, Redmond, Washington: Microsoft Press, 1996.

Digital Library

[14]

M.B. Miles and A.M. Huberman, Qualitative Data Analysis: An Expanded Sourcebook, second ed. Thousand Oaks, Calif.: Sage, 1994.

[15]

A. Parra C. Seaman V.R. Basili S. Kraft S. Condon S. Burke and D. Yakimovich, “The Package-Based Development Process in the Flight Dynamics Division,” Proc. 22nd Software Eng. Workshop, pp. 21–56, NASA/Goddard Space Flight Center Software Eng. Laboratory (SEL), Dec. 1997.

[16]

D.E. Perry N.A. Staudenmayer and L.G. Votta, “People, Organizations, and Process Improvement,” IEEE Software, vol. 11, no. 4 pp. 36–45, July 1994.

Digital Library

[17]

C.B. Seaman and V.R. Basili, “Communication and Organization: An Empirical Study of Discussion in Inspection Meetings,” IEEE Trans. Software Eng., vol. 24, no. 7, pp. 559–572, July 1998.

Digital Library

[18]

B. Shneiderman, Designing the User Interface: Strategies for Effective Human-Computer Interaction, third ed., ch. 4, pp. 146–47, Reading, Mass.: Addison-Wesley, 1998.

Digital Library

[19]

S.J. Taylor and R. Bogdan, Introduction to Qualitative Research Methods. New York: John Wiley & Sons, 1984.

[20]

A. von Mayrhauser and A.M. Vans, “Identification of Dynamic Comprehension Processes during Large Scale Maintenance,” IEEE Trans. Software Eng., vol. 22, no. 6, pp. 424–437, June 1996.

Digital Library

[21]

W.F. Whyte, Learning from the Field: A Guide from Experience. Beverly Hills, Calif.: Sage, 1984.

Cited By

Freire SPacheco ARios NPérez BCastellanos CCorreal DRamač RMandić VTaušan NLópez GMendonça MFalessi DIzurieta CSeaman CSpínola R(2024)A Comprehensive View on TD Prevention Practices and Reasons for not Preventing ItACM Transactions on Computer-Human Interaction10.1145/3674727Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3674727
Coutinho MMarques LSantos ADahia MFrança Cde Souza Santos RAdams BZimmermann TOzkaya ILin DZhang J(2024)The Role of Generative AI in Software Development Productivity: A Pilot Case StudyProceedings of the 1st ACM International Conference on AI-Powered Software10.1145/3664646.3664773(131-138)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3664646.3664773
Islam MJha AAkhmetov INadi S(2024)Characterizing Python Library MigrationsProceedings of the ACM on Software Engineering10.1145/36437311:FSE(92-114)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3643731
Show More Cited By

Index Terms

Qualitative Methods in Empirical Studies of Software Engineering
1. Software and its engineering

Recommendations

Combining quantitative and qualitative studies in empirical software engineering research
ICSE-C '17: Proceedings of the 39th International Conference on Software Engineering Companion

This technical briefing provides an overview of how quantitative empirical research methods can be combined with qualitative ones generating the family of empirical software engineering approaches known as mixed-methods. The ultimate aim of such mixed-...
Using ethnographic methods in software engineering research
ICSE '10: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2

This tutorial provides an overview of the role of ethnography in Software Engineering research. It describes the use of ethnographic methods as a means to provide an in-depth understanding of the socio-technological realities surrounding everyday ...
Issues in applying empirical software engineering to software architecture
ECSA'07: Proceedings of the First European conference on Software Architecture

Empirical software engineering focuses on the evaluation of software engineering technologies, such as processes and tools, by comparing related sets of data. It has contributed a valuable body of knowledge in several areas such as Software Economics ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Software Engineering

IEEE Transactions on Software Engineering Volume 25, Issue 4

July 1999

168 pages

ISSN:0098-5589

Editor:
Richard A. Kemmerer
Univ. of California, Santa Barbara

Issue’s Table of Contents

Copyright © Copyright © 1999 IEEE. All Rights Reserved.

Publisher

IEEE Press

Publication History

Published: 01 July 1999

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

415
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Freire SPacheco ARios NPérez BCastellanos CCorreal DRamač RMandić VTaušan NLópez GMendonça MFalessi DIzurieta CSeaman CSpínola R(2024)A Comprehensive View on TD Prevention Practices and Reasons for not Preventing ItACM Transactions on Computer-Human Interaction10.1145/3674727Online publication date: 28-Jun-2024
https://dl.acm.org/doi/10.1145/3674727
Coutinho MMarques LSantos ADahia MFrança Cde Souza Santos RAdams BZimmermann TOzkaya ILin DZhang J(2024)The Role of Generative AI in Software Development Productivity: A Pilot Case StudyProceedings of the 1st ACM International Conference on AI-Powered Software10.1145/3664646.3664773(131-138)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3664646.3664773
Islam MJha AAkhmetov INadi S(2024)Characterizing Python Library MigrationsProceedings of the ACM on Software Engineering10.1145/36437311:FSE(92-114)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3643731
Tanzil MUddin GBarcomb AGraziotin DNolte A(2024)"How do people decide?": A Model for Software Library SelectionProceedings of the 2024 IEEE/ACM 17th International Conference on Cooperative and Human Aspects of Software Engineering10.1145/3641822.3641865(1-12)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3641822.3641865
Sporsem TRoychoudhury APaiva AAbreu RStorey M(2024)Discovering Explainability Requirements in ML-Based SoftwareProceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings10.1145/3639478.3639807(170-172)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639478.3639807
Liu ZChen CWang JChen MWu BHuang YHu JWang Q(2024)Unblind Text Inputs: Predicting Hint-text of Text Input in Mobile Apps via LLMProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642939(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642939
Caglayan DÖzcan-Top ÖHong JPark J(2024)Revisiting Technical Debt Types and Indicators for Software SystemsProceedings of the 39th ACM/SIGAPP Symposium on Applied Computing10.1145/3605098.3636043(834-841)Online publication date: 8-Apr-2024
https://dl.acm.org/doi/10.1145/3605098.3636043
Archela GMelo AGava VHong JPark J(2024)Technical Debt Management in Agile ContextProceedings of the 39th ACM/SIGAPP Symposium on Applied Computing10.1145/3605098.3635946(826-833)Online publication date: 8-Apr-2024
https://dl.acm.org/doi/10.1145/3605098.3635946
Zhang YQin MStol KZhou MLiu HRoychoudhury APaiva AAbreu RStorey M(2024)How Are Paid and Volunteer Open Source Developers Different? A Study of the Rust ProjectProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639197(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639197
OBrien DBiswas SImtiaz SAbdalkareem RShihab ERajan HRoychoudhury APaiva AAbreu RStorey M(2024)Are Prompt Engineering and TODO Comments Friends or Foes? An Evaluation on GitHub CopilotProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639176(1-13)Online publication date: 20-May-2024
https://dl.acm.org/doi/10.1145/3597503.3639176
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents