Abstract
Providing accessibility in mobile applications is essential for the appropriate use by people with disabilities. Different evaluation methods yield different results, and professionals and researchers must be aware of the types of results obtained by user evaluations and inspections performed by professionals with different expertise levels. This study aimed to compare the results from manual inspection of mobile apps performed by two groups of professionals with different expertise levels and to compare them with user evaluations conducted by users with visual disabilities. The Saraiva and Receita Federal applications usability evaluations carried out with nine visually impaired users encountered 189 problems divided into 39 violations of accessibility guidelines. Then, the applications were inspected by two groups of professionals: 17 specialists in different areas of software development (full-stack developers, testers, and front-end developers) and ten specialists in Human-Computer Interaction (HCI) with previous experience in accessibility. The results indicated a difference between accessibility assessment methods. In accessibility inspections, there was a difference between software development specialists (DEV group) and HCI specialists (HCI group). The results indicated a difference in the number of violations the DEV group encounters compared to the HCI group. Inspections by HCI experts and testing with users with disabilities encountered a greater diversity of problem types. HCI professionals also showed a broader repertoire of accessibility inspection approaches. The results allow for an initial understanding of the extent to which evaluations by developers without HCI background can cover compared to inspections by HCI specialists and users with disabilities.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abascal, J., Arrue, M., Valencia, X.: Tools for web accessibility evaluation. In: Web Accessibility: A Foundation for Research, pp. 479–503 (2019)
Associação Brasileira de Normas Técnicas (ABNT): ABNT NBR 17060 - Acessibilidade em aplicativos de dispositivos móveis - Requisitos (Accessibility in Mobile Apps - requirements) (2022). shorturl.at/lvwJ2
BBC: HTML accessibility standards v2.0 (2014). http://www.bbc.co.uk/guidelines/futuremedia/accessibility/html/. Accessed 15 Feb 2021
Brajnik, G.: A comparative test of web accessibility evaluation methods. In: Proceedings of the 10th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 113–120 (2008)
Brajnik, G., Vigo, M., Yesilada, Y., Harper, S.: Group vs individual web accessibility evaluations: effects with novice evaluators. Interact. Comput. 28(6), 843–861 (2016)
Brajnik, G., Yesilada, Y., Harper, S.: Testability and validity of WCAG 2.0: the expertise effect. In: Proceedings of the 12th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 43–50 (2010)
Brajnik, G., Yesilada, Y., Harper, S.: The expertise effect on web accessibility evaluation methods. Hum.-Comput. Interact. 26(3), 246–283 (2011)
British Broadcasting Corporation - BBC: Mobile accessibility standards and guidelines v1.0 (2014). http://shorturl.at/wCPX3. Accessed 15 Feb 2021
Carvalho, M.C.N., Dias, F.S., Reis, A.G.S., Freire, A.P.: Accessibility and usability problems encountered on websites and applications in mobile devices by blind and normal-vision users. In: Proceedings of the 33rd Annual ACM Symposium on Applied Computing, SAC 2018, pp. 2022–2029. Association for Computing Machinery, New York (2018)
Dias, F.S.: Análise de causas técnicasde problemas de acessibilidade encontrados em aplicativos nativos para dispositivos móveis. UFLA, Lavras (2018)
Freire, A.P.: Disabled people and the Web: User-based measurement of accessibility. Ph.D. thesis, University of York (2012)
Governo Brasileiro: Lei brasileira de inclusão da pessoa com deficiência (2015). https://www.planalto.gov.br/ccivil_03/_ato2015-2018/2015/lei/l13146.htm. Accessed 31 Jan 2023
ISO - International Organization for Standardization: ISO 9241-11. usability: Definitions and concepts (2018). https://www.iso.org/obp/ui/iso:std:iso:9241:-11:ed-2:v1:en. Accessed 31 Jan 2023
Jaeger, P.T.: Assessing section 508 compliance on federal e-government web sites: a multi-method, user-centered evaluation of accessibility for persons with disabilities. Gov. Inf. Q. 23(2), 169–190 (2006)
Mankoff, J., Fait, H., Tran, T.: Is your web page accessible? A comparative study of methods for assessing web page accessibility for the blind. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 41–50 (2005)
Power, C., Freire, A., Petrie, H., Swallow, D.: Guidelines are only half of the story: accessibility problems encountered by blind users on the web. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 433–442 (2012)
Power, C., Petrie, H.: Working with participants. In: Web Accessibility: A Foundation for Research, pp. 153–168 (2019)
Rømen, D., Svanæs, D.: Validating WCAG versions 1.0 and 2.0 through usability testing with disabled users. Univers. Access Inf. Soc. 11(4), 375–385 (2012)
Statista: Market share of mobile operating systems in Brazil from January 2019 to August 2022 (2022). https://www.statista.com/statistics/262167/market-share-held-by-mobile-operating-systems-in-brazil/. Accessed 20 May 2023
Statista: Mobile OS share in North America 2018–2023 (2022). https://www.statista.com/statistics/1045192/share-of-mobile-operating-systems-in-north-america-by-month/. Accessed 20 May 2023
Vollenwyder, B., Petralito, S., Iten, G.H., Brühlmann, F., Opwis, K., Mekler, E.D.: How compliance with web accessibility standards shapes the experiences of users with and without disabilities. Int. J. Hum.-Comput. Stud. 170, 102956 (2023)
W3C: Web content accessibility guidelines 1.0, web accessibility initiative (1999). https://www.w3.org/TR/WCAG10/. Accessed 31 Jan 2023
W3C: W3c - world wide web consortium. web content accessibility guidelines (WCAG) 2.0 (2008). https://www.w3.org/TR/WCAG20/. Accessed 31 Jan 2023
W3C: W3c. web content accessibility guidelines (WCAG) 2.1 (2018). https://www.w3.org/TR/WCAG21/. Accessed 31 Jan 2023
W3C: W3c. web content accessibility guidelines (WCAG) 2.2 (2020). https://www.w3.org/TR/WCAG22/. Accessed 31 Jan 2023
W3C: W3c. web content accessibility guidelines (WCAG) 3.0 (2021). https://www.w3.org/TR/wcag-3.0/. Accessed 31 Jan 2023
Acknowledgements
We would like to thank all the participants who took part in this study for their valuable contribution. We also thank FAPEMIG, CNPq and FAPESP.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mateus, D.A., Ferreira, S.B.L., de Almeida Souza, M.R., Freire, A.P. (2023). Accessibility Inspections of Mobile Applications by Professionals with Different Expertise Levels: An Empirical Study Comparing with User Evaluations. In: Abdelnour Nocera, J., Kristín Lárusdóttir, M., Petrie, H., Piccinno, A., Winckler, M. (eds) Human-Computer Interaction – INTERACT 2023. INTERACT 2023. Lecture Notes in Computer Science, vol 14142. Springer, Cham. https://doi.org/10.1007/978-3-031-42280-5_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-42280-5_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-42279-9
Online ISBN: 978-3-031-42280-5
eBook Packages: Computer ScienceComputer Science (R0)
