Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article

Embedding equality, diversity and inclusion in usability testing: : Recommendations and a research agenda

Published: 18 July 2024 Publication History

Highlights

Diversity in usability research participants is critical to ensure resulting technologies will be fit for purpose.
Giving participants a sense of value, trust and agency are likely universal principles for embedding equality, diversity and inclusion in research.
Organisations should facilitate an open and sharing culture where researchers can learn from mistakes.
Participants and researchers need to share expertise and perspectives to facilitate equality, diversity and inclusion.

Abstract

Technologies support our everyday lives, and to ensure that people are not routinely excluded they must be usable by the wider population. However, technologies are not commonly tested with participants from a range of backgrounds. This paper reports on interviews and roundtable discussions with people whose identities can be underrepresented in usability testing and usability researchers to discuss how equality, diversity and inclusion (EDI) can be embedded in usability testing.
Key findings include (1) when people participate in research they need a sense of value, trust and agency, and (2) challenges for researchers for embedding EDI in usability testing include organisational pressures, stakeholder culture, getting guidance and recruiting who you need. Recommendations are made to researchers, and to the organisations that employ them. Additionally, we propose a research agenda for a community of users, creators of services and products, usability researchers, and all those advocating for EDI in usability research.

References

[1]
K.R. Acharya, Promoting Social Justice Through Usability in Technical Communication: an Integrative Literature Review, Tech. Commun. 69 (2022) 6–26.
[2]
S.A. Alerasoul, G. Afeltra, H. Hakala, E. Minelli, F. Strozzi, Organisational learning, learning organisation, and learning orientation: an integrative review and framework, Hum. Resour. Manag. Rev. 32 (2022),.
[3]
C.A. Bohnert, R.M. Combs, E.J. Noonan, A.E. Weathers, L.A. Weingartner, Gender minorities in simulation: a mixed methods study of medical school standardized patient programs in the United States and Canada, Simul. Healthc. 16 (2021) e151–e158,.
[4]
J. Brooks, N. King, Template Analysis for Business and Management Students, 2016.
[5]
D. Caliz, L. Martínez, R. Caliz, Helping people with down syndrome through an usability testing guide proposal for mobile applications, Int. J. Comput. Sci. Eng. Inf. Technol. 7 (2017) 1–19,.
[6]
B. Colliver, Representation of Lgbtq communities in the grand theft auto series, Video Games, Crime and Next-Gen Deviance: Reorienting the Debate, 131–149, 2020,.
[7]
V.P. Cornet, T. Toscos, D. Bolchini, R.R. Ghahari, R. Ahmed, C. Daley, R.J. Holden, 2020. Untold stories in user-centered design of mobile health: practical challenges and strategies learned from the design and evaluation of an app for older adults with heart failure, JMIR mHealth uHealth 8 (2020).
[8]
J. Craven, H. Booth, Putting awareness into practice: practical steps for conducting usability tests, Libr. Rev. 55 (2006) 179–194,.
[9]
T. Darin, R. Andrade, J. Sánchez, Usability evaluation of multimodal interactive virtual environments for learners who are blind: an empirical investigation, Int. J. Hum. Comput. Stud. 158 (2022),.
[10]
H.T.O. Davies, S.M. Nutley, Developing learning organisations in the new NHS, Br. Med. J. 320 (2000) 998–1001,.
[11]
Disability Rights UK., n.d. About us [WWW Document]. Disabil. Rights UK. URL https://www.disabilityrightsuk.org/about-us/.
[13]
K. Elmore, L. Meloncon, C.A. James, Embracing interdependence: technology developers, autistic users, and technical communicators, Rhetorical Accessability: At the Intersection of Technical Communication and Disability Studies, Routledge, 2014, pp. 15–38.
[14]
I. Etikan, Comparison of Convenience Sampling and Purposive Sampling, Am. J. Theor. Appl. Stat. 5 (2016) 1,.
[15]
A. Fernandez, E. Insfran, S. Abrahão, Usability evaluation methods for the web: a systematic mapping study, Inf. Softw. Technol. 53 (2011) 789–817,.
[16]
S. Gagnon, T. Augustin, W. Cukier, Interplay for change in equality, diversity and inclusion studies, Hum. Relation. 75 (2022) 1327–1353,.
[17]
D.A. Garvin, Building a learning organization, Harv. Bus. Rev. 71 (1993) 78–91.
[18]
L. Hanna, K. Risden, K. Alexander, Guidelines for usability testing with children, interactions 4 (1997) 9–14,.
[19]
S. Harihareswara, User experience is a social justice issue. [WWW Document], Code4Lib J 28 (2015).
[20]
Henry, S.L., 2007. Just Ask: integrating Accessibility Throughout Design. [WWW Document]. Just Ask. URL www.uiAccess.com/JustAsk/.
[21]
L. Hutter, H.M. Lawrence, Promoting inclusive and accessible design in usability testing, Commun. Des. Q. 6 (2018) 21–30,.
[22]
C. Imison, M. Kaur, S. Dawson, Supporting Equity and Tackling Inequality: How can NIHR Promote Inclusion in public Partnerships? An Agenda for Action, 2022.
[23]
ISO, Ergonomics of Human-System Interaction - Part 11: Usability: Definitions and Concepts, 2018.
[24]
J. Johnstone, Technology as empowerment: a capability approach to computer ethics, Ethics Inf. Technol. 9 (2007) 73–87,.
[25]
A. Joyce, Usability Testing with Minors: 16 Tips. [WWW Document], NN/g Nielsen Norman Group, 2019, URL https://www.nngroup.com/articles/usability-testing-minors/.
[26]
J.L. Korte, L.E. Potter, S.H. Nielsen, An experience in requirements prototyping with young Deaf children, J. Usability Stud. 10 (2015) 195–214.
[27]
Á. López-Cabrales, J.C. Real, R. Valle, Relationships between human resource management practices and organizational learning capability: the mediating role of human capital, Pers. Rev. 40 (2011) 344–363,.
[28]
K. Mack, E. McDonnell, What do we mean by accessibility research? a literature survey of accessibility papers in chi and assets from 1994 to 2019, Conf. Hum. Factors Comput. Syst. - Proc. (2021),.
[29]
J. Mankoff, G.R. Hayes, D. Kasnitz, Disability studies as a source of critical inquiry for the field of assistive technology, in: ASSETS’10 - Proc. 12th Int. ACM SIGACCESS Conf. Comput. Access. 3–10, 2010,.
[30]
S. Marsh, Designing and researching with people with aphasia. User research in government, User Res. Gov. (2019) URL https://userresearch.blog.gov.uk/2019/08/30/designing-and-researching-with-people-with-aphasia/.
[31]
G. Meiselwitz, B. Wentz, J. Lazar, Universal usability: past, present, and future, Foundations and Trends in Human-Computer Interaction, 2009,.
[32]
A. Mihoc, Conducting remote research with people with access needs, User Res. Gov. (2020) URL https://userresearch.blog.gov.uk/2020/05/20/conducting-remote-research-with-people-with-access-needs/.
[33]
K. Moran, Usability Testing 101. NN/g Nielsen Norman Group. [WWW Document], Nielsen Norman Group logoNielsen Norman Group, 2019, URL.
[34]
J. Nielsen, Thinking aloud: The #1 Usability tool. [WWW Document], NN/g Nielsen Norman Group, 2012, URL https://www.nngroup.com/articles/thinking-aloud-the-1-usability-tool/.
[35]
J. Nielsen, Usability 101: Introduction to Usability. [WWW Document], NN/g Nielsen Norman Group, 2012, URL https://www.nngroup.com/articles/usability-101-introduction-to-usability/.
[36]
J. Nielsen, Why You Only Need to Test With 5 users. [WWW Document], NN/g Nielsen Norman Group, 2000, URL https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/.
[37]
M.J.M. Opiniano, Object-centered interviews in mixed methods: yielding the emotions of overseas migrant households in family financial socialization, Method. Psychol. 5 (2021),.
[38]
J. Paylor, C. McKevitt, The Possibilities and Limits of “Co-producing”, Research. Front. Sociol. 4 (2019) 1–5,.
[39]
K. Pernice, J. Nielsen, How to Conduct Usability Studies for Accessibility Methodology Guidelines for Testing Websites and Intranets With People Who Use Assistive Technology, 2012.
[40]
H. Petrie, F. Hamilton, N. King, P. Pavan, Remote usability evaluations with disabled people, Conf. Hum. Factors Comput. Syst. - Proc. 2 (2006) 1133–1141,.
[41]
D. Potnis, B. Gala, Best practices for conducting fieldwork with marginalized communities, Inf. Process. Manag. 57 (2020),.
[42]
S.A. Raza, Managing ethical requirements elicitation of complex socio-technical systems with critical systems thinking: a case of course-timetabling project, Technol. Soc. 66 (2021),.
[43]
F.H.A. Razak, H. Hafit, N. Sedi, N.A. Zubaidi, H. Haron, Usability testing with children: laboratory vs field studies, in: Proc. - 2010 Int. Conf. User Sci. Eng. i-USEr, 2010, 2010, pp. 104–109,.
[44]
J. Rubin, D. Chisnell, Handbook of Usability testing: How to plan, design, and Conduct Effective Tests, 2nd ed., Wiley Publisher, 2008.
[45]
A. Schlesinger, W.K. Edwards, R.E. Grinter, Intersectional HCI: engaging identity through gender, race, and class, in: Conf. Hum. Factors Comput. Syst. - Proc. 2017-May, 2017, pp. 5412–5427,.
[46]
B. Shneiderman, Universal usability, Commun. ACM 43 (2000) 84–91,.
[47]
H. Smith, A. Manzini, J. Ives, Inclusivity in TAS research: an example of EDI as RRI, J. Responsible Technol. 12 (2022),.
[48]
A. Strohmayer, R. Bellini, J. Meissner, E. Alabdulqader, A. Toombs, S.M. Finnigan, M. Balaam, CHIversity: implications for equality, diversity, and inclusion campaigns, Conf. Hum. Factors Comput. Syst. - Proc. (2018) 1–10,. 2018-April.
[49]
C.M. Sum, R. Alharbi, F. Spektor, C.L. Bennett, C.N. Harrington, K. Spiel, R.M. Williams, Dreaming disability justice in HCI, Conf. Hum. Factors Comput. Syst. - Proc. (2022),.
[50]
W.B. Tindale, P. Dimitri, MedTech innovation across the life course–the importance of users and usability, J. Med. Eng. Technol. 46 (2022) 427–432,.
[51]
M.K. Tornblad, K.S. Jones, F.A. Inan, Individuals with Autism spectrum disorder: a review of human factors considerations and usability testing guidelines, in: Proc. Hum. Factors Ergon. Soc. 2019-Octob, 2019, pp. 1575–1579,.
[52]
M.B. Twidale, D.M. Nichols, C.P. Lueg, Everyone everywhere: a distributed and embedded paradigm for usability, J. Assoc. Inf. Sci. Technol. 72 (2021) 1272–1284,.
[53]
UK Government, Running research sessions with disabled people, Gov.uK Serv. Man (2017) URL https://www.gov.uk/service-manual/user-research/running-research-sessions-with-people-with-disabilities.
[54]
A.C. van den Berg, S. Giest, W. Kraaij, Assessing inclusivity in online platforms through usability evaluation with Google Analytics, Policy and Internet 15 (2023) 55–77,.
[55]
T. Van Der Geest, Conducting usability studies with users who are elderly or have disabilities, Tech. Commun. 53 (2006) 23–31.
[56]
W3C WAI, 2016. Accessibility, Usability, and Inclusion. [WWW Document]. W3C WAI. URL https://www.w3.org/WAI/fundamentals/accessibility-usability-inclusion/.
[57]
D.M. Walls, User experience in social justice contexts, in: SIGDOC 2016 - 34th ACM Int. Conf. Des. Commun, 2016,.
[58]
G. Wheeler, N. Mills, U. Ankeny, P. Howsley, C. Bartlett, H. Elphick, P. Dimitri, Meaningful involvement of children and young people in health technology development, J. Med. Eng. Technol. 46 (2022) 462–471,.
[59]
P. Williams, Developing methods to evaluate web usability with people with learning difficulties, Br. J. Spec. Educ. 33 (2006) 173–179,.
[60]
R. Wood, E. Dixon, S. Elsayed-Ali, E. Shokeen, A. Lazar, J. Lazar, Investigating Best Practices for Remote Summative Usability Testing with People with Mild to Moderate Dementia, ACM Trans. Access. Comput. 14 (2021) 1–26,.
[61]
Y. Yoo, Qiarterjy, MIS Q. 34 (2010) 213–231.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image International Journal of Human-Computer Studies
International Journal of Human-Computer Studies  Volume 188, Issue C
Aug 2024
140 pages

Publisher

Academic Press, Inc.

United States

Publication History

Published: 18 July 2024

Author Tags

  1. Usability
  2. EDI
  3. Inclusive research
  4. Diverse participation
  5. Recruitment
  6. Evaluation

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 0
    Total Downloads
  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 16 Oct 2024

Other Metrics

Citations

View Options

View options

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media