research-article

Obstacles to usability evaluation in practice: a survey of software development organizations

Authors:

Jakob Otkjær Bak,

Peter Risgaard,

Jan StageAuthors Info & Claims

NordiCHI '08: Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges

Pages 23 - 32

https://doi.org/10.1145/1463160.1463164

Published: 20 October 2008 Publication History

Abstract

This paper reports from a combined questionnaire survey and interview study of obstacles for deploying usability evaluation in software development organizations. It was conducted in a limited geographical area. The purpose of the questionnaire survey was to determine whether software development organizations in that area were evaluating the usability of their software and to identify key obstacles. It revealed that 29 of 39 software development organizations conducted some form of usability evaluation. The purpose of the interview study was to gain more insight into the obstacles that were expressed. It involved 10 of the 39 software development organizations. Our results show, that the understanding of usability evaluation is a major obstacle. Furthermore, the two most significant obstacles were resource demands and the mindset of developers. These obstacles were not only an obstacle for more organizations to deploy usability evaluation, but also a concern for the software development organizations, that had deployed usability evaluations in their development process.

References

[1]

Beu, A., Honold, P., and Yuan, X. 2000 How to Build Up an Infrastructure for Intercultural Usability Engineering. International Journal of Human-Computer Interaction 12, 3--4, 347--358.

[2]

Boedker, S. and Buur, J. 2002 The design collaboratorium: a place for usability design. ACM Transactions on Computer-Human Interaction 9, 2, 152--169.

Digital Library

[3]

Boivie, I., Aaborg, C., Persson, J., and Löfberg, M. 2003 Why usability gets lost or usability in in-house software development. Interacting with Computers 15, 4, 623--639.

[4]

Boivie, I., Gulliksen, J., and Göransson, B. 2006 Usability professionals' current practices and future development. Interacting with Computers 18, 4, 568--600.

Digital Library

[5]

Boivie, I., Gulliksen, J., and Göransson, B. 2006 The lonesome cowboy: A study of the usability designer role in systems development. Interacting with Computers 18, 4, 601--634.

Digital Library

[6]

Cajander, Aa., Gulliksen, J., and Boivie, I. 2006 Management perspectives on usability in a public authority: a case study. NordiCHI '06: Proceedings of the 4th Nordic conference on Human-computer interaction, 38--47. ACM Press.

Digital Library

[7]

Carter, J. 1999 Incorporating standards and guidelines in an approach that balances usability concerns for developers and end users. Interacting with Computers 12, 2, 179--206.

[8]

Card, S. K., Newell, A., and Moran, T. P. 1983 The Psychology of Human-Computer Interaction. Lawrence Erlbaum Associates.

Digital Library

[9]

Dillon, A., Sweeney, M., and Maguire, M. 1993 A survey of usability engineering within the European I.T. Industry Current practice and needs. Proceedings of CHI 93. ACM Press.

[10]

Dray, S. M. and Karat, C.-M. 1994 Human factors cost justification for an internal development project. Cost-justifying usability, 111--122. Academic Press.

Digital Library

[11]

Gulliksen, J., Göransson, B., Boivie, I., Blomkvist, S., Persson, J., and Cajander, Aa. 2003 Key principles for user-centred systems design. Behaviour & Information Technology 22, 6, 397--409.

[12]

Granollers, T. 2003 User Centred Design Process Model. Integration of Usability Engineering and Software Engineering. Proceedings of Interact 2003.

[13]

Grudin, J. 1991 Obstacles to user involvement in software product development, with implications for CSCW. International Journal of Man-Machine Studies 34, 3, 435--452.

Digital Library

[14]

Grudin, J. 1991 Systematic Sources of Suboptimal Interface Design in Large Product Development Organizations. Human-computer interaction 6, 2, 147--196.

Digital Library

[15]

Grudin, J. 1991 Interactive systems: bridging the gaps between developers and users. Computer 24, 4, 59--69.

Digital Library

[16]

Gould, J. D. and Lewis C. 1985 Designing for usability: key principles and what designers think. Communications of the ACM 28, 3, 300--311.

Digital Library

[17]

Gunther, R., Janis, J., and Butler S. The UCD Decision Matrix: How, When, and Where to Sell User-Centered Design into the Development Cycle. Found at: http://www.ovostudios.com/upa2001/surf.htm last seen April 11th 2007.

[18]

HTS-A's Regionalraad 2006 HTS-A Region Nordjylland The voice of the service sector (in Danish). HTS-A.

[19]

Iivari, N. 2006 Representing the User in software development. A cultural analysis of usability work in the product development context. Interacting with Computers 18, 4, 635--664.

Digital Library

[20]

Ji, Y. G. and Yun, M. H. 2006 Enhancing the Minority Discipline in the IT Industry: A Survey of Usability and User-Centered Design Practice. International Journal of Human-Computer Interaction 20, 2, 117--134.

[21]

Karat, C.-M. 1994 A Comparison of User Interface Evaluation Methods. Usability Inspection Methods, 203--233.

Digital Library

[22]

Karat. C.-M. 1997 Cost-justifying usability engineering in the software life cycle. Handbook of Human-Computer Interaction, 767--778.

[23]

Kujala, S. User involvement: a review of the benefits and challenges. Behaviour & Information Technology 22, 1, 1--16.

[24]

Kvale, S. 1998 Interview. An introduction to the qualitative research interview, 2^nd ed. Hans Reitzel.

[25]

Lauesen, S. Usability Engineering in Industrial Practice. Proceedings of Interact 97, 15--22.

Digital Library

[26]

Reed, S. 1992 Who defines usability? You do!. PC//Computing Dec., 220--232.

[27]

Rosenbaum, S., Bloomer, S., Rinehart, D., Rohn, J. A., Dye, K., Humburg, J., Nielsen, J., and Wixon D. 1999 What makes strategic usability fail?: lessons learned from the field. CHI '99 extended abstracts, 93--94. ACM Press.

Digital Library

[28]

Rosenbaum, S., Rohn, J. A., and Humburg, J. 2000 A Toolkit for Strategic Usability: Results from Workshops Panels, and Surveys. CHI 2000 Proceedings, ACM Inc.,. Found at: http://www.teced.com/PDFs/Chi2000sr.pdf last seen April 11th 2007.

Digital Library

[29]

Scapin, D. and Law, E. (2007) Proceedings of Mause International Workshop on Review, Report and Refine Usability Evaluation Methods (R3-UEM). COST 294 MAUSE.

[30]

Smith, A. and Dunckley, L. 2002 Prototype evaluation and redesign: structuring the design space through contextual techniques. Interacting with Computers 14, 6, 821--843.

[31]

Strauss, A. and Corbin, J. 1998 Basics of qualitative research. Techniques and Procedures for Developing Grounded Theory, 2.edition. SAGE Publications.

[32]

Strøm, G. How usability practitioners can get their suggestions implemented in industrial software design. Found at: http://www.georg.dk/usab-imple.pdf last seen April 11th 2007.

[33]

Videnskabsministeriet 2005 A sector that creates value? Focus on the Danish IT sector (in Danish) Ministeriet for Videnskab Teknologi og Udvikling.

[34]

Wilson, S., Bekker, M., Johnson, P., and Johnson, H. 1997 Helping and hindering user involvement. A tale of everyday design. Proceedings of CHI 97.

Digital Library

[35]

Wynekoop, J. L. and Conger, S. A. 1990 A Review of Computer Aided Software Engineering Research Methods. Proceedings of the IFIP TC8WG8.2 Working Conference on The Information Systems Research Arena of The 90s.

Cited By

Visescu ILárusdóttir MChoi W(2024)Exploration of UCD Practice LimitationsInteracting with Computers10.1093/iwc/iwae046Online publication date: 10-Oct-2024
https://doi.org/10.1093/iwc/iwae046
Azevedo DRamelot JLegay AKieffer S(2024)With or Without U(sers): A Journey to Integrate UX Activities in CybersecurityHCI for Cybersecurity, Privacy and Trust10.1007/978-3-031-61379-1_14(212-231)Online publication date: 1-Jun-2024
https://doi.org/10.1007/978-3-031-61379-1_14
Larusdottir MLanzilotti RPiccinno AVisescu ICostabile M(2023)UCD Sprint: A Fast Process to Involve Users in the Design Practices of Software CompaniesInternational Journal of Human–Computer Interaction10.1080/10447318.2023.227981640:23(8360-8377)Online publication date: 20-Nov-2023
https://doi.org/10.1080/10447318.2023.2279816
Show More Cited By

Index Terms

Obstacles to usability evaluation in practice: a survey of software development organizations
1. Social and professional topics
  1. Professional topics
    1. Management of computing and information systems
      1. Project and people management
        Systems development
      2. System management
        Quality assurance
2. Software and its engineering
  1. Software creation and management

Recommendations

Usability Evaluation of the Agile Software Process
IVIC 2013: Third International Visual Informatics Conference on Advances in Visual Informatics - Volume 8237

Agile development methods are most flexible approach for software development where the development team keeps on improving the software with ongoing involvement of user. Despite its flexibility approach in software development, agile methods are still ...
Integrating Usability Evaluations into Scrum: A Case Study Based on Remote Synchronous User Testing
16th International Conference on Human-Computer Interaction. Theories, Methods, and Tools - Volume 8510

The tendency to empower users in the software development process encourages the continuing search for ways to reconcile the interests of agile methodologies and Human-Computer Interaction HCI activities. The practice of agile methods, e.g. Scrum, is ...
Is usability evaluation important: the perspective of novice software developers
BCS-HCI '13: Proceedings of the 27th International BCS Human Computer Interaction Conference

In this paper we present the results of a study which aims to explore the perspective of novice software developers about usability evaluation. It is important for a software organization to understand how novice developers perceive the role and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

NordiCHI '08: Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges

October 2008

621 pages

ISBN:9781595937049

DOI:10.1145/1463160

General Chairs:
Konrad Tollmar
Lund University, Sweden
,
Bodil Jönsson
Lund University, Sweden

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Mangold International
Microsoft Dynamics

In-Cooperation

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 October 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

NordiCHI08

Sponsor:

NordiCHI08: 5th Nordic Conference on Human-Computer Interaction

October 20 - 22, 2008

Lund, Sweden

Acceptance Rates

Overall Acceptance Rate 379 of 1,572 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

58
Total Citations
View Citations
1,611
Total Downloads

Downloads (Last 12 months)58
Downloads (Last 6 weeks)7

Reflects downloads up to 31 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Visescu ILárusdóttir MChoi W(2024)Exploration of UCD Practice LimitationsInteracting with Computers10.1093/iwc/iwae046Online publication date: 10-Oct-2024
https://doi.org/10.1093/iwc/iwae046
Azevedo DRamelot JLegay AKieffer S(2024)With or Without U(sers): A Journey to Integrate UX Activities in CybersecurityHCI for Cybersecurity, Privacy and Trust10.1007/978-3-031-61379-1_14(212-231)Online publication date: 1-Jun-2024
https://doi.org/10.1007/978-3-031-61379-1_14
Larusdottir MLanzilotti RPiccinno AVisescu ICostabile M(2023)UCD Sprint: A Fast Process to Involve Users in the Design Practices of Software CompaniesInternational Journal of Human–Computer Interaction10.1080/10447318.2023.227981640:23(8360-8377)Online publication date: 20-Nov-2023
https://doi.org/10.1080/10447318.2023.2279816
Nielsen SOrdoñez RSkov MJochum E(2023)Strategies for strengthening UX competencies and cultivating corporate UX in a large organisation developing robotsBehaviour & Information Technology10.1080/0144929X.2023.222728443:9(1769-1797)Online publication date: 28-Jun-2023
https://doi.org/10.1080/0144929X.2023.2227284
Inal YClemmensen TRajanen DIivari NRizvanoglu KSivaji A(2022)Positive developments but challenges still aheadJournal of Usability Studies10.5555/3532713.353271615:4(210-246)Online publication date: 18-Apr-2022
https://dl.acm.org/doi/10.5555/3532713.3532716
Clemmensen TIivari NRajanen DSivaji A(2022)“Organized UX Professionalism” – An Empirical Study and Conceptual Tool for Scrutinizing UX Work of the FutureHuman Work Interaction Design. Artificial Intelligence and Designing for a Positive Work Experience in a Low Desire Society10.1007/978-3-031-02904-2_2(34-65)Online publication date: 1-Apr-2022
https://doi.org/10.1007/978-3-031-02904-2_2
Szabó BHercegfi K(2022)User‐centered approaches in software development processes: Qualitative research into the practice of Hungarian companiesJournal of Software: Evolution and Process10.1002/smr.250135:2Online publication date: 25-Aug-2022
https://doi.org/10.1002/smr.2501
Güncan DOnay Durdu P(2021)A user‐centered behavioral software development modelJournal of Software: Evolution and Process10.1002/smr.227433:2Online publication date: 14-Feb-2021
https://dl.acm.org/doi/10.1002/smr.2274
Kashfi PFeldt RNilsson A(2019)Integrating UX principles and practices into software development organizationsJournal of Systems and Software10.1016/j.jss.2019.03.066154:C(37-58)Online publication date: 1-Aug-2019
https://dl.acm.org/doi/10.1016/j.jss.2019.03.066
Li M(2019)Usability Problems and Obstacles to Addressing Them in Health Information Software ImplementationsInformation and Communication Technologies for Development. Strengthening Southern-Driven Cooperation as a Catalyst for ICT4D10.1007/978-3-030-19115-3_20(241-252)Online publication date: 27-Apr-2019
https://doi.org/10.1007/978-3-030-19115-3_20
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten