research-article

Open access

Human-Centered AI Design in Reality: A Study of Developer Companies’ Practices : A study of Developer Companies’ Practices

Authors:

Maria Hartikainen,

Kaisa Väänänen,

Saara Ala-Luopa,

Thomas OlssonAuthors Info & Claims

NordiCHI '22: Nordic Human-Computer Interaction Conference

Article No.: 55, Pages 1 - 11

https://doi.org/10.1145/3546155.3546677

Published: 08 October 2022 Publication History

All formats PDF

Abstract

Human-Centered AI (HCAI) advocates the development of AI applications that are trustworthy, usable, and based on human needs. While the conceptual foundations of HCAI are extensively discussed in recent literature, the industry practices and methods appear to lag behind. To advance HCAI method development, current practices of AI developer companies need to be understood. To understand how HCAI principles manifest in the current practices of AI development, we conducted an interview study of practitioners from 12 AI developer companies in Finland, focusing on the early stages of AI application development. Our thematic analysis reveals current development practices and identifies four main challenges: (i) detachment of HCAI work from technical development, (ii) clients’ central role as the source of user requirements, (iii) uncertain nature of AI, and (iv) lack of value-based understanding of AI. The findings inform the development of HCAI methods and implementation of HCAI principles in AI application development.

References

[1]

Amershi, Saleema, Andrew Begel, Christian Bird, Robert DeLine, Harald Gall, Ece Kamar, Nachiappan Nagappan, Besmira Nushi, and Thomas Zimmermann. 2019. Software engineering for machine learning: A case study. In Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice. ACM, Montreal, Canada, 291-300

Digital Library

[2]

Amershi, Saleema, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh 2019. Guidelines for human-AI interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, Glasgow, 1-13.

Digital Library

[3]

Arrieta, Alejandro Barredo, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García 2020. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities, and challenges toward responsible AI. Information Fusion 58 (2020): 82-115.

Digital Library

[4]

Auernhammer, Jan. 2020. Human-centered AI: The role of Human-centered Design Research in the development of AI. In Synergy - DRS International Conference 2020. Held online.

[5]

Balaji, S., and M. Sundararajan Murugaiyan. 2012. Waterfall vs. V-Model vs. Agile: A comparative study on SDLC. International Journal of Information Technology and Business Management 2, 1 (2012), 26-30.

[6]

van Berkel, Niels, Tag, Benjamin, Goncalves, Jorge, and Hosio, Simo. 2020. Human-centred artificial intelligence: a contextual morality perspective. Behaviour & Information Technology. 1-17.

[7]

Braun, Virginia, and Clarke, Victoria. 2012. Thematic analysis. APA handbook of research methods in psychology, Vol. 2: Research designs: Quantitative, qualitative, neuropsychological, and biological. American Psychological Association, Washington.

[8]

Bond, Raymond R., Mulvenna, Maurice D., Wan, Hui, Finlay, Dewar D., Wong, Alexander, Ansgar, Koene, Brisk, Rob, Boger, Jennifer and Adel, Tameem. 2019. Human Centered Artificial Intelligence: Weaving UX into Algorithmic Decision Making. In Proceedings of 16th International Conference on Human-Computer Interaction. Bucharest, 2-9.

[9]

Bogers, Sander, Joep Frens, Janne Van Kollenburg, Eva Deckers, and Caroline Hummels. 2016. Connected Baby Bottle: A Design Case Study Towards a Framework for Data-Enabled Design. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems. ACM, 301-311.

Digital Library

[10]

Cantamessa, Marco, Montagna, Francesca, Altavilla, Stefania, and Casagrande-Seretti, Alessandro. 2020. Data-Driven Design: The New Challenges of Digitalization on Product Design and Development. Design Science, 6.

[11]

Cerejo, Joana. 2021. The design process of human-centered AI. Retrieved Feb 8, 2022, from https://bootcamp.uxdesign.cc/human-centered-ai-design-process-part-1-8cf7e3ce00

[12]

Chandra, Yanto, and Shang, Liang. 2019. Inductive coding. In Qualitative research using R: A systematic approach. Springer, Singapore.

[13]

Dellermann, Dominik, Adrian Calma, Nikolaus Lipusch, Thorsten Weber, Sascha Weigel, and Philipp Ebel. 2021. The future of human-AI collaboration: a taxonomy of design knowledge for hybrid intelligence systems. arXiv preprint arXiv:2105.03354.

[14]

Dignum, Virginia. 2019. HumanE AI. Project report. Retrieved Oct 8, 2021, from https://www.humane-ai.eu/wp-content/uploads/2019/11/D13-HumaneAI-framework-report.pdf

[15]

Dove, Graham, Kim Halskov, Jodi Forlizzi, and John Zimmerman. 2017. UX design innovation: Challenges for working with machine learning as a design material. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, Neew York, 278-288.

Digital Library

[16]

Feuerriegel, Stefan, Mateusz Dolata, and Gerhard Schwabe. 2020. Fair AI: Challenges and Opportunities. Business & information systems engineering 62, 4 (2020): 379-384.

[17]

Eiband, M., Völkel, S. T., Buschek, D., Cook, S., & Hussmann, H. (2019, March). When people and algorithms meet: User-reported problems in intelligent everyday applications. In Proceedings of the 24th international conference on intelligent user interfaces (pp. 96-106).

[18]

Ghallab, Malik. 2019. Responsible AI: requirements and challenges. AI Perspectives, 1:1. 1-7.

[19]

Giacomin, Joseph. 2014. What is human centred design? The Design Journal 17, 4 (2014): 606-623.

[20]

Girardin, Fabien, and Neal Lathia. 2017. When user experience designers partner with data scientists. In AAAI Spring Symposium Series on Designing the User Experience of Machine Learning Systems. The IEEE press, Palo Alto.

[21]

Guidotti, Riccardo, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2018. A survey of methods for explaining black box models. ACM computing surveys 51, 5 (2018). 1-42.

[22]

Hagendorff, Thilo. 2020. The ethics of AI ethics: An evaluation of guidelines. Minds and Machines 30, 1 (2020). 99-120.

Digital Library

[23]

Harper, Richard. 2019. The Role of HCI in the Age of AI. International Journal of Human–Computer Interaction, 35, 15 (2019). 1331-1344.

[24]

HAX. 2021. Microsoft Human-centered AI eXperience Toolkit. Retrieved Feb 8, 2022, from https://www.microsoft.com/en-us/haxtoolkit/

[25]

Herbsleb, James, David Zubrow, Dennis Goldenson, Will Hayes, and Mark Paulk. 1997. Software quality and the capability maturity model. Communications of the ACM 40, 6 (1997). 30-40.

[26]

Hesenius, Marc, Nils Schwenzfeier, Ole Meyer, Wilhelm Koop, and Volker Gruhn. 2019. Towards a software engineering process for developing data-driven applications. In Proceedings of 7th International Workshop on Realizing Artificial Intelligence Synergies in Software Engineering. ACM, Montreal, 35-41.

Digital Library

[27]

Hill, Charles, Bellamy, Rachel, Erickson, Thomas, and Burnett, Margaret. 2016. Trials and Tribulations of Developers of Intelligent Systems: A Field Study. In 2016 IEEE Symposium on Visual Languages and Human-Centric Computing.162-170.

[28]

Holmquist, Lars Erik. 2017. Intelligence on tap: artificial intelligence as a new design material. Interactions 24, no. 4 (2017). 28-33.

Digital Library

[29]

IBM Design for AI. 2019. Retrieved Oct 8, 2021 from https://www.ibm.com/design/ai/

[30]

Jin, Xiaoneng, Evans, Mark, Dong, Hua, and Yao, Anqi. 2021. Design Heuristics for Artificial Intelligence: Inspirational Design Stimuli for Supporting UX Designers in Generating AI-powered Ideas. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 1-8.

Digital Library

[31]

Lu, Jiahao, Ortega, Alejandra. G., Gonçalves, Milene, and Bourgeois, Jacky. 2021. The impact of data on the role of designers and their process. Proceedings of the Design Society, 1. 3021-3030.

[32]

Maguire, Martin. 2001. Methods to support human-centred design. International journal of human-computer studies 55, 4 (2001). 587-634.

Digital Library

[33]

Oppermann, Leif, Alexander Boden, Britta Hofmann, Wolfgang Prinz, and Stefan Decker. 2019. Beyond HCI and CSCW: Challenges and useful practices towards a human-centred vision of AI and IA. In Proceedings of the Halfway to the Future Symposium. ACM, 1-5.

Digital Library

[34]

PAIR with Google. People + AI Research. Retrieved Oct 8 from https://pair.withgoogle.com.

[35]

Paulk, Mark C., Curtis, Bill, Chrissis, Mary Beth, and Weber, Charles V. 1993. Capability maturity model, version 1.1. IEEE software 10, 4 (1993). 18-27.

Digital Library

[36]

Passi, Samir and Jackson, Steven J. 2018. Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects. Proceedings of the ACM on Human-Computer Interaction, 2. 1-28.

Digital Library

[37]

Piorkowski, David, Park, Soya, Wang, April Y., Wang, Dakuo, Muller, Michael, and Portnoy, Felix. 2021. How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study. Proceedings of the ACM on Human-Computer Interaction, 5. 1-25.

Digital Library

[38]

Saldaña, Johnny. 2014. Coding and analysis strategies. In The Oxford handbook of qualitative research.

[39]

Shneiderman, Ben. 2020. Bridging the gap between ethics and practice: Guidelines for reliable, safe, and trustworthy Human-Centered AI systems. ACM Transactions on Interactive Intelligent Systems, 10, 4 (2020). 1-31.

Digital Library

[40]

Shneiderman, Ben. 2020. Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy. International Journal of Human–Computer Interaction 36, 6 (2020). 495-504.

[41]

Silberg, Jake and Manyika, James. 2019. Notes from the AI frontier: Tackling bias in AI (and in humans). McKinsey Global Institute. 1-6.

[42]

Steen, Marc. 2011. Tensions in Human-Centred Design. CoDesign 7, 1 (2011). 45-60.

[43]

Subramonyam, Hariharan, Colleen Seifert, and Eytan Adar. 2021. ProtoAI: Model-Informed Prototyping for AI-Powered Interfaces. In Proceedigns of 26th International Conference on Intelligent User Interfaces. ACM, Virtual, 48–58.

Digital Library

[44]

Verheij, Bart. 2020. Artificial intelligence as law. Artificial Intelligence Law 28, 2 (2020). 181-206.

[45]

Wan, Zhiyuan, Xia, Xin, Lo, David, and Murphy, Gail C. 2019. How Does Machine Learning Change Software Development Practices? IEEE Transactions on Software Engineering, 47:9. 1857-1871

[46]

Wickramasinghe, Chathurika S., Marino, Daniel L., Grandio, Javier, and Manic, Milos. 2020. Trustworthy AI Development Guidelines for Human System Interaction. In International Conference on Human System Interaction. 130-136.

[47]

Windl, Maximiliane, Feger, Sebastian S., Zijlstra, Lara, Schmidt, Albrecht, and Wozniak, Pawel W. 2022. ‘It Is Not Always Discovery Time’: Four Pragmatic Approaches in Designing AI Systems. In CHI Conference on Human Factors in Computing Systems. 1-12.

Digital Library

[48]

van Wynsberghe, Aimee. 2021. Sustainable AI: AI for sustainability and the sustainability of AI. AI and Ethics, 1:3. 213-218.

[49]

Zhang, Jie M., Mark Harman, Lei Ma, and Yang Liu. 2020. Machine learning testing: Survey, landscapes, and horizons. IEEE Transactions on Software Engineering.

Digital Library

[50]

Xu, Wei. 2019. Toward Human-Centered AI: A Perspective from Human-Computer Interaction. Interactions 26, 4 (2019). 42-46.

Digital Library

[51]

Yang, Qian, Aaron Steinfeld, Carolyn Rosé, and John Zimmerman. 2020. Re-examining whether, why, and how human-AI interaction is uniquely difficult to design. In Proceedings of the CHI Conference on Human Factors in Computing Systems. ACM, Honolulu, 1-13.

Digital Library

Cited By

Dritsa Dvan Renswouw LColombo SVäänänen KBogers SMartinez AHolbrook JBrombacher A(2024)Designing (with) AI for WellbeingExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3636282(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3636282
Xu WGao Z(2024)Enabling Human-Centered AI: A Methodological Perspective2024 IEEE 4th International Conference on Human-Machine Systems (ICHMS)10.1109/ICHMS59971.2024.10555771(1-6)Online publication date: 15-May-2024
https://doi.org/10.1109/ICHMS59971.2024.10555771
Bach TKristiansen JBabic AJacovi A(2024)Unpacking Human-AI Interaction in Safety-Critical Industries: A Systematic Literature ReviewIEEE Access10.1109/ACCESS.2024.343719012(106385-106414)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3437190
Show More Cited By

Index Terms

Human-Centered AI Design in Reality: A Study of Developer Companies’ Practices : A study of Developer Companies’ Practices
1. Computing methodologies
  1. Artificial intelligence
2. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Towards a Human-Centred Artificial Intelligence Maturity Model
CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems

Artificial intelligence (AI) is becoming a central building block of computational systems. Following the long traditions of human-centered design, Human-Centered AI (HCAI) emphasises the importance of putting humans and various societal considerations ...
Recruitment Chatbot Acceptance in Company Practices: An Elicitation Study
CHItaly '23: Proceedings of the 15th Biannual Conference of the Italian SIGCHI Chapter

Recruitment bots are becoming increasingly popular, but there is limited research on how human-centered design principles guide their development. This study aims to address this gap by providing insights into the design of recruitment chatbots that ...
SIG on Data as Human-Centered Design Material
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems

Designers and HCI researchers from industry and academia have been exploring the opportunities that emerge from incorporating behavioral data into the design process. For this, designers employ and combine data from multiple sources, multiple scales, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

NordiCHI '22: Nordic Human-Computer Interaction Conference

October 2022

1091 pages

ISBN:9781450396998

DOI:10.1145/3546155

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 October 2022

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

NordiCHI '22

NordiCHI '22: Nordic Human-Computer Interaction Conference

October 8 - 12, 2022

Aarhus, Denmark

Acceptance Rates

Overall Acceptance Rate 379 of 1,572 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
1,429
Total Downloads

Downloads (Last 12 months)774
Downloads (Last 6 weeks)85

Reflects downloads up to 14 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Dritsa Dvan Renswouw LColombo SVäänänen KBogers SMartinez AHolbrook JBrombacher A(2024)Designing (with) AI for WellbeingExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613905.3636282(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3636282
Xu WGao Z(2024)Enabling Human-Centered AI: A Methodological Perspective2024 IEEE 4th International Conference on Human-Machine Systems (ICHMS)10.1109/ICHMS59971.2024.10555771(1-6)Online publication date: 15-May-2024
https://doi.org/10.1109/ICHMS59971.2024.10555771
Bach TKristiansen JBabic AJacovi A(2024)Unpacking Human-AI Interaction in Safety-Critical Industries: A Systematic Literature ReviewIEEE Access10.1109/ACCESS.2024.343719012(106385-106414)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3437190
Ala-Luopa SOlsson TVäänänen KHartikainen MMakkonen J(2024)Trusting Intelligent Automation in Expert Work: Accounting Practitioners’ Experiences and PerceptionsComputer Supported Cooperative Work (CSCW)10.1007/s10606-024-09499-6Online publication date: 30-Apr-2024
https://doi.org/10.1007/s10606-024-09499-6
Ryan M(2024)We’re only human after all: a critique of human-centred AIAI & SOCIETY10.1007/s00146-024-01976-2Online publication date: 4-Jun-2024
https://doi.org/10.1007/s00146-024-01976-2
Tahvanainen LTetri BAhonen O(2024)Exploring and Extending Human-Centered Design to Develop AI-Enabled Wellbeing Technology in HealthcareDigital Health and Wireless Solutions10.1007/978-3-031-59091-7_19(288-306)Online publication date: 5-May-2024
https://doi.org/10.1007/978-3-031-59091-7_19
Kurian NSaad C(2024)Where Technology Meets EmpathyCommunication and Education10.1002/9781119985280.ch10(148-163)Online publication date: 12-Jan-2024
https://doi.org/10.1002/9781119985280.ch10
Colley AVäänänen KHäkkilä J(2022)Tangible Explainable AI - an Initial Conceptual FrameworkProceedings of the 21st International Conference on Mobile and Ubiquitous Multimedia10.1145/3568444.3568456(22-27)Online publication date: 27-Nov-2022
https://dl.acm.org/doi/10.1145/3568444.3568456

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents