research-article

Designing Interactive Transfer Learning Tools for ML Non-Experts

Authors:

Jeffrey M RzeszotarskiAuthors Info & Claims

CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

Article No.: 364, Pages 1 - 15

https://doi.org/10.1145/3411764.3445096

Published: 07 May 2021 Publication History

Abstract

Interactive machine learning (iML) tools help to make ML accessible to users with limited ML expertise. However, gathering necessary training data and expertise for model-building remains challenging. Transfer learning, a process where learned representations from a model trained on potentially terabytes of data can be transferred to a new, related task, offers the possibility of providing ”building blocks” for non-expert users to quickly and effectively apply ML in their work. However, transfer learning largely remains an expert tool due to its high complexity. In this paper, we design a prototype to understand non-expert user behavior in an interactive environment that supports transfer learning. Our findings reveal a series of data- and perception-driven decision-making strategies non-expert users employ, to (in)effectively transfer elements using their domain expertise. Finally, we synthesize design implications which might inform future interactive transfer learning environments.

Supplementary Material

MP4 File (3411764.3445096_videofigure.mp4)

Supplemental video

Download
80.45 MB

References

[1]

Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, Manjunath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek G. Murray, Benoit Steiner, Paul Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2016. TensorFlow: A System for Large-Scale Machine Learning. In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation (Savannah, GA, USA) (OSDI’16). USENIX Association, USA, 265–283.

[2]

Y. Ahn and Y. R. Lin. 2020. FairSight: Visual Analytics for Fairness in Decision Making. IEEE Transactions on Visualization and Computer Graphics 26, 1(2020), 1086–1095. https://doi.org/10.1109/TVCG.2019.2934262

[3]

Ahmed Alqaraawi, Martin Schuessler, Philipp Weiß, Enrico Costanza, and Nadia Berthouze. 2020. Evaluating Saliency Map Explanations for Convolutional Neural Networks: A User Study. In Proceedings of the 25th International Conference on Intelligent User Interfaces (Cagliari, Italy) (IUI ’20). Association for Computing Machinery, New York, NY, USA, 275–285. https://doi.org/10.1145/3377325.3377519

Digital Library

[4]

Saleema Amershi, Maya Cakmak, William Bradley Knox, and Todd Kulesza. 2014. Power to the people: The role of humans in interactive machine learning. AI Magazine 35, 4 (2014), 105–120. https://doi.org/10.1609/aimag.v35i4.2513

Digital Library

[5]

Mykhaylo Andriluka, Jasper R. R. Uijlings, and Vittorio Ferrari. 2018. Fluid Annotation: A Human-Machine Collaboration Interface for Full Image Annotation. In Proceedings of the 26th ACM International Conference on Multimedia (Seoul, Republic of Korea) (MM ’18). Association for Computing Machinery, New York, NY, USA, 1957–1966. https://doi.org/10.1145/3240508.3241916

Digital Library

[6]

Jürgen Bernard, Marco Hutter, Matthias Zeppelzauer, Dieter Fellner, and Michael Sedlmair. 2017. Comparing visual-interactive labeling with active learning: An experimental study. IEEE transactions on visualization and computer graphics 24, 1(2017), 298–308.

[7]

Alsallakh Bilal, Amin Jourabloo, Mao Ye, Xiaoming Liu, and Liu Ren. 2018. Do convolutional neural networks learn class hierarchy?IEEE transactions on visualization and computer graphics 24, 1(2018), 152–162.

[8]

Nathan Bos, Kimberly Glasgow, John Gersh, Isaiah Harbison, and Celeste Lyn Paul. 2019. Mental models of AI-based systems: User predictions and explanations of image classification results. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 63, 1(2019), 184–188. https://doi.org/10.1177/1071181319631392arXiv:https://doi.org/10.1177/1071181319631392

[9]

M. Brooks, S. Amershi, B. Lee, S. M. Drucker, A. Kapoor, and P. Simard. 2015. FeatureInsight: Visual support for error-driven feature ideation in text classification. In 2015 IEEE Conference on Visual Analytics Science and Technology (VAST). IEEE Computer Society, Los Alamitos, CA, USA, 105–112. https://doi.org/10.1109/VAST.2015.7347637

[10]

Michelle Carney, Barron Webster, Irene Alvarado, Kyle Phillips, Noura Howell, Jordan Griffith, Jonas Jongejan, Amit Pitaru, and Alexander Chen. 2020. Teachable Machine: Approachable Web-Based Tool for Exploring Machine Learning Classification. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3334480.3382839

Digital Library

[11]

Davide Castelvecchi. 2016. Can we open the black box of AI?Nature News 538, 7623 (2016), 20.

[12]

Minsuk Choi, Cheonbok Park, Soyoung Yang, Yonggyu Kim, Jaegul Choo, and Sungsoo Ray Hong. 2019. AILA: Attentive Interactive Labeling Assistant for Document Classification through Attention-Based Deep Neural Networks. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300460

Digital Library

[13]

Gregory Cohen, Saeed Afshar, Jonathan Tapson, and André van Schaik. 2017. EMNIST: an extension of MNIST to handwritten letters. http://arxiv.org/abs/1702.05373

[14]

Graham Dove, Kim Halskov, Jodi Forlizzi, and John Zimmerman. 2017. UX Design Innovation: Challenges for Working with Machine Learning as a Design Material. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI ’17). Association for Computing Machinery, New York, NY, USA, 278–288. https://doi.org/10.1145/3025453.3025739

Digital Library

[15]

Steven P. Dow, Alana Glassco, Jonathan Kass, Melissa Schwarz, Daniel L. Schwartz, and Scott R. Klemmer. 2011. Parallel Prototyping Leads to Better Design Results, More Divergence, and Increased Self-Efficacy. ACM Trans. Comput.-Hum. Interact. 17, 4, Article 18 (Dec. 2011), 24 pages. https://doi.org/10.1145/1879831.1879836

Digital Library

[16]

Jerry Alan Fails and Dan R. Olsen. 2003. Interactive Machine Learning. In Proceedings of the 8th International Conference on Intelligent User Interfaces (Miami, Florida, USA) (IUI ’03). Association for Computing Machinery, New York, NY, USA, 39–45. https://doi.org/10.1145/604045.604056

Digital Library

[17]

Rebecca Fiebrink, Perry R. Cook, and Dan Trueman. 2011. Human Model Evaluation in Interactive Supervised Learning. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, BC, Canada) (CHI ’11). Association for Computing Machinery, New York, NY, USA, 147–156. https://doi.org/10.1145/1978942.1978965

Digital Library

[18]

Kunihiko Fukushima. 1980. Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological cybernetics 36, 4 (1980), 193–202.

[19]

Google. 2019. Teachable Machines. https://teachablemachine.withgoogle.com/

[20]

Google. 2020. Google ML KIT. https://developers.google.com/ml-kit

[21]

Google. 2020. ML-Kit App Lose It. https://developers.google.com/ml-kit/case-studies/lose-it

[22]

Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Advances in Psychology 52 (1988), 139–183. https://doi.org/10.1016/S0166-4115(08)62386-9

[23]

F. Hohman, M. Kahng, R. Pienta, and D. H. Chau. 2019. Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers. IEEE Transactions on Visualization and Computer Graphics 25, 8(2019), 2674–2693.

Digital Library

[24]

Fred Hohman, Kanit Wongsuphasawat, Mary Beth Kery, and Kayur Patel. 2020. Understanding and Visualizing Data Iteration in Machine Learning. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376177

Digital Library

[25]

Sungsoo Ray Hong, Jessica Hullman, and Enrico Bertini. 2020. Human Factors in Model Interpretability: Industry Practices, Challenges, and Needs. Proc. ACM Hum.-Comput. Interact. 4, CSCW, Article 068 (May 2020), 26 pages. https://doi.org/10.1145/3392878

Digital Library

[26]

IBM. 2019. TJBot IBM Research. https://www.research.ibm.com/tjbot/

[27]

Deloitte Insights. 2019. Technology, Media, and Telecommunications Predictions. https://www2.deloitte.com/insights/us/en/industry/technology/technology-media-and-telecom-predictions.html

[28]

Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional Architecture for Fast Feature Embedding. In Proceedings of the 22Nd ACM International Conference on Multimedia (Orlando, Florida, USA) (MM ’14). ACM, New York, NY, USA, 675–678. https://doi.org/10.1145/2647868.2654889

Digital Library

[29]

Liu Jiang, Shixia Liu, and Changjian Chen. 2019. Recent research advances on interactive machine learning. Journal of Visualization 22, 2 (2019), 401–417. https://doi.org/10.1007/s12650-018-0531-1

Digital Library

[30]

Minsuk Kahng, Pierre Y. Andrews, Aditya Kalro, and Duen Horng Chau. 2018. ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models. IEEE Transactions on Visualization and Computer Graphics 24 (2018), 88–97.

[31]

Ashish Kapoor, Bongshin Lee, Desney Tan, and Eric Horvitz. 2010. Interactive Optimization for Steering Machine Classification. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA) (CHI ’10). Association for Computing Machinery, New York, NY, USA, 1343–1352. https://doi.org/10.1145/1753326.1753529

Digital Library

[32]

Andrej Karpathy. 2016. Convnet JS Github. https://github.com/karpathy/convnetjs

[33]

Keras. 2019. Keras Library. https://keras.io/

[34]

J. Kim and C. Park. 2017. End-To-End Ego Lane Estimation Based on Sequential Transfer Learning for Self-Driving Cars. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, Honolulu, HI, USA, 1194–1202. https://doi.org/10.1109/CVPRW.2017.158

[35]

Todd Kulesza, Simone Stumpf, Margaret Burnett, and Irwin Kwan. 2012. Tell Me More? The Effects of Mental Model Soundness on Personalizing an Intelligent Agent. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12). Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/2207676.2207678

Digital Library

[36]

I. Lage, E. Chen, J. He, M. Narayanan, S. Gershman, B. Kim, and F. Doshi-Velez. 2018. An Evaluation of the Human-Interpretability of Explanation.

[37]

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature 521, 7553 (2015), 436–444.

[38]

Y. Liang, S. T. Monteiro, and E. S. Saber. 2016. Transfer learning for high resolution aerial image classification. In 2016 IEEE Applied Imagery Pattern Recognition Workshop (AIPR). IEEE, Washington, DC, USA, 1–8. https://doi.org/10.1109/AIPR.2016.8010600

[39]

Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, Nov (2008), 2579–2605.

[40]

S. J. Pan and Q. Yang. 2010. A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering 22, 10(2010), 1345–1359.

Digital Library

[41]

Aditya Parameswaran. 2019. Enabling Data Science for the Majority. Proc. VLDB Endow. 12, 12 (Aug. 2019), 2309–2322. https://doi.org/10.14778/3352063.3352148

Digital Library

[42]

Peter Pirolli and Stuart Card. 2005. The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In Proceedings of international conference on intelligence analysis, Vol. 5. McLean, VA, USA, VA,USA, 2–4.

[43]

Lorien Y. Pratt, Jack Mostow, and Candace A. Kamm. 1991. Direct Transfer of Learned Information among Neural Networks. In Proceedings of the Ninth National Conference on Artificial Intelligence - Volume 2(AAAI’91). AAAI Press, Anaheim, California, 584–589.

[44]

Donghao Ren, Saleema Amershi, Bongshin Lee, Jina Suh, and Jason D Williams. 2017. Squares: Supporting interactive performance analysis for multiclass classifiers. IEEE transactions on visualization and computer graphics 23, 1(2017), 61–70. https://doi.org/10.1109/TVCG.2016.2598828

Digital Library

[45]

Sebastian Ruder, Matthew E. Peters, Swabha Swayamdipta, and Thomas Wolf. 2019. Transfer Learning in Natural Language Processing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Tutorials. Association for Computational Linguistics, Minneapolis, Minnesota, 15–18. https://doi.org/10.18653/v1/N19-5004

[46]

Dominik Sacha, Michael Sedlmair, Leishi Zhang, John A Lee, Jaakko Peltonen, Daniel Weiskopf, Stephen C North, and Daniel A Keim. 2017. What you see is what you can change: Human-centered machine learning by interactive visualization. Neurocomputing 268(2017), 164–175.

Digital Library

[47]

Aécio Santos, Sonia Castelo, Cristian Felix, Jorge Piazentin Ono, Bowen Yu, Sungsoo Ray Hong, Cláudio T. Silva, Enrico Bertini, and Juliana Freire. 2019. Visus: An Interactive System for Automatic Machine Learning Model Building and Curation. In Proceedings of the Workshop on Human-In-the-Loop Data Analytics (Amsterdam, Netherlands) (HILDA’19). Association for Computing Machinery, New York, NY, USA, Article 6, 7 pages. https://doi.org/10.1145/3328519.3329134

Digital Library

[48]

Simone Stumpf, Vidya Rajaram, Lida Li, Weng-Keen Wong, Margaret Burnett, Thomas Dietterich, Erin Sullivan, and Jonathan Herlocker. 2009. Interacting meaningfully with machine learning systems: Three experiments. International Journal of Human-Computer Studies 67, 8 (2009), 639 – 662. https://doi.org/10.1016/j.ijhcs.2009.03.004

Digital Library

[49]

Jina Suh, Soroush Ghorashi, Gonzalo Ramos, Nan-Chen Chen, Steven Drucker, Johan Verwey, and Patrice Simard. 2019. AnchorViz: Facilitating Semantic Data Exploration and Concept Discovery for Interactive Machine Learning. ACM Trans. Interact. Intell. Syst. 10, 1, Article 7 (Aug. 2019), 38 pages. https://doi.org/10.1145/3241379

Digital Library

[50]

Justin Talbot, Bongshin Lee, Ashish Kapoor, and Desney S. Tan. 2009. EnsembleMatrix: Interactive Visualization to Support Machine Learning with Multiple Classifiers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Boston, MA, USA) (CHI ’09). Association for Computing Machinery, New York, NY, USA, 1283–1292. https://doi.org/10.1145/1518701.1518895

Digital Library

[51]

Lisa Torrey and Jude Shavlik. 2009. Transfer learning. Handbook of research on machine learning applications and trends: algorithms, methods, and techniques 1 (2009), 242–264.

[52]

Malcolm Ware, Eibe Frank, Geoffrey Holmes, Mark Hall, and Ian H. Witten. 2002. Interactive Machine Learning: Letting Users Build Classifiers. Int. J. Hum.-Comput. Stud. 56, 3 (March 2002), 281–292.

[53]

Karl Weiss, Taghi M Khoshgoftaar, and DingDing Wang. 2016. A survey of transfer learning. Journal of Big data 3, 1 (2016), 9. https://doi.org/10.1186/s40537-016-0043-6

[54]

Marcus Winter and Phil Jackson. 2020. Flatpack ML: How to Support Designers in Creating a New Generation of Customizable Machine Learning Applications. In Design, User Experience, and Usability. Design for Contemporary Interactive Environments, Aaron Marcus and Elizabeth Rosenzweig (Eds.). Springer International Publishing, Cham, 175–193.

[55]

Tongshuang Wu, Daniel S. Weld, and Jeffrey Heer. 2019. Local Decision Pitfalls in Interactive Machine Learning: An Investigation into Feature Selection in Sentiment Analysis. ACM Trans. Comput.-Hum. Interact. 26, 4, Article 24 (June 2019), 27 pages. https://doi.org/10.1145/3319616

Digital Library

[56]

Qian Yang, Alex Scuito, John Zimmerman, Jodi Forlizzi, and Aaron Steinfeld. 2018. Investigating How Experienced UX Designers Effectively Work with Machine Learning. In Proceedings of the 2018 Designing Interactive Systems Conference (Hong Kong, China) (DIS ’18). Association for Computing Machinery, New York, NY, USA, 585–596. https://doi.org/10.1145/3196709.3196730

Digital Library

[57]

Qian Yang, Jina Suh, Nan-Chen Chen, and Gonzalo Ramos. 2018. Grounding Interactive Machine Learning Tool Design in How Non-Experts Actually Build Models. In Proceedings of the 2018 Designing Interactive Systems Conference (Hong Kong, China) (DIS ’18). Association for Computing Machinery, New York, NY, USA, 573–584. https://doi.org/10.1145/3196709.3196729

Digital Library

[58]

Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. 2014. How Transferable Are Features in Deep Neural Networks?. In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2 (Montreal, Canada) (NIPS’14). MIT Press, Cambridge, MA, USA, 3320–3328.

Digital Library

[59]

Matthew D. Zeiler and Rob Fergus. 2014. Visualizing and Understanding Convolutional Networks. In Computer Vision – ECCV 2014, David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars (Eds.). Springer International Publishing, Cham, 818–833.

[60]

Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong, and Qing He. 2020. A comprehensive survey on transfer learning. Proc. IEEE 109, 1 (2020), 43–76.

Cited By

Lee CLee MMutlu B(2024)The AI-DEC: A Card-based Design Method for User-centered AI ExplanationsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661576(1010-1028)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661576
Zhang SYu JXu XYin CLu YYao BTory MPadilla LCaterino JZhang PWang D(2024)Rethinking Human-AI Collaboration in Complex Medical Decision Making: A Case Study in Sepsis DiagnosisProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642343(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642343
Yamani AAl-Shammare HBaslyman M(2024)Establishing Heuristics for Improving the Usability of GUI Machine Learning Tools for Novice UsersProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642087(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642087
Show More Cited By

Index Terms

Designing Interactive Transfer Learning Tools for ML Non-Experts
1. Computing methodologies
2. Human-centered computing
  1. Human computer interaction (HCI)

Index terms have been assigned to the content through auto-classification.

Recommendations

Big data, lifelong machine learning and transfer learning
WSDM '13: Proceedings of the sixth ACM international conference on Web search and data mining

A major challenge in today's world is the Big Data problem, which manifests itself in Web and Mobile domains as rapidly changing and heterogeneous data streams. A data-mining system must be able to cope with the influx of changing data in a continual ...
Investigation in transfer learning: better way to apply transfer learning between agents
MLDM'11: Proceedings of the 7th international conference on Machine learning and data mining in pattern recognition

This paper propose to investigate a better way to apply Transfer Learning (TL) between agents to speed up the Q-learning Reinforcement Learning algorithm and combines Case-Based Reasoning (CBR) and Heuristically Accelerated Reinforcement Learning (HARL) ...
Transfer in variable-reward hierarchical reinforcement learning

Transfer learning seeks to leverage previously learned tasks to achieve faster learning in a new task. In this paper, we consider transfer learning in the context of related but distinct Reinforcement Learning (RL) problems. In particular, our RL ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

May 2021

10862 pages

ISBN:9781450380966

DOI:10.1145/3411764

General Chairs:
Yoshifumi Kitamura
Tohoku University, Japan
,
Aaron Quigley
University of New South Wales, Australia
,
Program Chairs:
Katherine Isbister
University of California Santa Cruz, USA
,
Takeo Igarashi
The University of Tokyo, Japan
,
Publications Chairs:
Pernille Bjørn
University of Copenhagen, Denmark
,
Steven Drucker
Microsoft Research, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Best Paper

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

CHI '21

Sponsor:

SIGCHI

CHI '21: CHI Conference on Human Factors in Computing Systems

May 8 - 13, 2021

Yokohama, Japan

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
1,764
Total Downloads

Downloads (Last 12 months)252
Downloads (Last 6 weeks)20

Reflects downloads up to 26 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Lee CLee MMutlu B(2024)The AI-DEC: A Card-based Design Method for User-centered AI ExplanationsProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661576(1010-1028)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661576
Zhang SYu JXu XYin CLu YYao BTory MPadilla LCaterino JZhang PWang D(2024)Rethinking Human-AI Collaboration in Complex Medical Decision Making: A Case Study in Sepsis DiagnosisProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642343(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642343
Yamani AAl-Shammare HBaslyman M(2024)Establishing Heuristics for Improving the Usability of GUI Machine Learning Tools for Novice UsersProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642087(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642087
Raees MMeijerink ILykourentzou IKhan VPapangelis K(2024)From explainable to interactive AI: A literature review on current trends in human-AI interactionInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103301189(103301)Online publication date: Sep-2024
https://doi.org/10.1016/j.ijhcs.2024.103301
He YYang XChang CXie HIgarashi T(2023)Efficient Human-in-the-loop System for Guiding DNNs AttentionProceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581641.3584074(294-306)Online publication date: 27-Mar-2023
https://dl.acm.org/doi/10.1145/3581641.3584074
Feng KMcdonald D(2023)Addressing UX Practitioners’ Challenges in Designing ML Applications: an Interactive Machine Learning ApproachProceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581641.3584064(337-352)Online publication date: 27-Mar-2023
https://dl.acm.org/doi/10.1145/3581641.3584064
Mishra SRzeszotarski J(2023)Human Expectations and Perceptions of Learning in Machine TeachingProceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization10.1145/3565472.3595612(13-24)Online publication date: 18-Jun-2023
https://dl.acm.org/doi/10.1145/3565472.3595612
Feng KCoppock MMcDonald D(2023)How Do UX Practitioners Communicate AI as a Design Material? Artifacts, Conceptions, and PropositionsProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3596101(2263-2280)Online publication date: 10-Jul-2023
https://dl.acm.org/doi/10.1145/3563657.3596101
Shi CHu YWang SMa SZheng CMa XLuo Q(2023)RetroLens: A Human-AI Collaborative System for Multi-step Retrosynthetic Route PlanningProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581469(1-20)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581469
Capel TBrereton M(2023)What is Human-Centered about Human-Centered AI? A Map of the Research LandscapeProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580959(1-23)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580959
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents